5051-5100 of 10000 results (69ms)
2022-12-14 §
09:21 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host bast6001.wikimedia.org [production]
09:20 <jayme@deploy1002> helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
08:41 <hashar> Restarted Gerrit for a plugin update [production]
08:39 <hashar@deploy1002> Finished deploy [gerrit/gerrit@c0b0a70]: Add support for PipelineBot to the Checks API plugin - T214068 (duration: 00m 09s) [production]
08:39 <hashar@deploy1002> Started deploy [gerrit/gerrit@c0b0a70]: Add support for PipelineBot to the Checks API plugin - T214068 [production]
08:39 <aqu@deploy1002> Finished deploy [airflow-dags/analytics@353573b]: HDFS usage dataset pipeline deployment without superuser [airflow-dags@353573b] (duration: 00m 13s) [production]
08:39 <aqu@deploy1002> Started deploy [airflow-dags/analytics@353573b]: HDFS usage dataset pipeline deployment without superuser [airflow-dags@353573b] [production]
08:37 <aqu@deploy1002> Finished deploy [airflow-dags/analytics_test@353573b]: HDFS usage dataset pipeline deployment without superuser TEST [airflow-dags@353573b] (duration: 00m 10s) [production]
08:37 <aqu@deploy1002> Started deploy [airflow-dags/analytics_test@353573b]: HDFS usage dataset pipeline deployment without superuser TEST [airflow-dags@353573b] [production]
08:26 <hashar@deploy1002> Finished deploy [gerrit/gerrit@c0b0a70]: Add support for PipelineBot to the Checks API plugin - T214068 (duration: 00m 11s) [production]
08:25 <hashar@deploy1002> Started deploy [gerrit/gerrit@c0b0a70]: Add support for PipelineBot to the Checks API plugin - T214068 [production]
07:28 <phedenskog@deploy1002> Finished deploy [performance/navtiming@7ba179f]: (no justification provided) (duration: 00m 08s) [production]
07:28 <phedenskog@deploy1002> Started deploy [performance/navtiming@7ba179f]: (no justification provided) [production]
01:22 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host logstash2037.codfw.wmnet with OS bullseye [production]
01:18 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host logstash2036.codfw.wmnet with OS bullseye [production]
01:06 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on logstash2037.codfw.wmnet with reason: host reimage [production]
01:04 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on logstash2036.codfw.wmnet with reason: host reimage [production]
01:02 <cwhite@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on logstash2037.codfw.wmnet with reason: host reimage [production]
01:01 <cwhite@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on logstash2036.codfw.wmnet with reason: host reimage [production]
00:46 <cwhite@cumin2002> START - Cookbook sre.hosts.reimage for host logstash2037.codfw.wmnet with OS bullseye [production]
00:45 <cwhite@cumin2002> START - Cookbook sre.hosts.reimage for host logstash2036.codfw.wmnet with OS bullseye [production]
2022-12-13 §
22:57 <jdrewniak@deploy1002> Finished scap: Backport for [[gerrit:867618|Account for syntax errors in closest selector (T325113)]] (duration: 08m 29s) [production]
22:50 <jdrewniak@deploy1002> jdrewniak and jdlrobson: Backport for [[gerrit:867618|Account for syntax errors in closest selector (T325113)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet [production]
22:48 <jdrewniak@deploy1002> Started scap: Backport for [[gerrit:867618|Account for syntax errors in closest selector (T325113)]] [production]
22:42 <jdrewniak@deploy1002> Finished scap: Backport for [[gerrit:867617|Account for syntax errors in closest selector (T325113)]] (duration: 09m 20s) [production]
22:40 <dzahn@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "parse1002: failed -> active - dzahn@cumin2002" [production]
22:40 <mutante> netbox: set parse1002 status: failed -> active in web UI; ran cookbook 'sre.puppet.sync-netbox-hiera' to get data in sync - T324949 [production]
22:38 <dzahn@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "parse1002: failed -> active - dzahn@cumin2002" [production]
22:35 <dzahn@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
22:34 <jdrewniak@deploy1002> jdrewniak and jdlrobson: Backport for [[gerrit:867617|Account for syntax errors in closest selector (T325113)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet [production]
22:34 <dzahn@cumin2002> START - Cookbook sre.dns.netbox [production]
22:33 <jdrewniak@deploy1002> Started scap: Backport for [[gerrit:867617|Account for syntax errors in closest selector (T325113)]] [production]
22:23 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host logstash2035.codfw.wmnet with OS bullseye [production]
22:14 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host logstash2033.codfw.wmnet with OS bullseye [production]
22:12 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host logstash2034.codfw.wmnet with OS bullseye [production]
22:10 <aqu@deploy1002> Finished deploy [analytics/refinery@66736e1] (hadoop-test): HDFS FSImage conversion to XML script TEST [analytics/refinery@66736e1] (duration: 01m 11s) [production]
22:09 <aqu@deploy1002> Started deploy [analytics/refinery@66736e1] (hadoop-test): HDFS FSImage conversion to XML script TEST [analytics/refinery@66736e1] [production]
22:08 <aqu@deploy1002> Finished deploy [analytics/refinery@66736e1] (thin): HDFS FSImage conversion to XML script THIN [analytics/refinery@66736e1] (duration: 00m 07s) [production]
22:08 <aqu@deploy1002> Started deploy [analytics/refinery@66736e1] (thin): HDFS FSImage conversion to XML script THIN [analytics/refinery@66736e1] [production]
22:06 <aqu@deploy1002> Finished deploy [analytics/refinery@66736e1]: HDFS FSImage conversion to XML script [analytics/refinery@66736e1] (duration: 26m 32s) [production]
21:53 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on logstash2033.codfw.wmnet with reason: host reimage [production]
21:52 <reedy@deploy1002> Synchronized wmf-config/CommonSettings.php: extension distributor updates (duration: 06m 50s) [production]
21:50 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on logstash2034.codfw.wmnet with reason: host reimage [production]
21:48 <cwhite@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on logstash2033.codfw.wmnet with reason: host reimage [production]
21:47 <cwhite@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on logstash2034.codfw.wmnet with reason: host reimage [production]
21:43 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on logstash2035.codfw.wmnet with reason: host reimage [production]
21:41 <kindrobot> Finishing UTC late backport window [production]
21:40 <kindrobot@deploy1002> Finished scap: Backport for [[gerrit:863228|Start writing to cul_actor everywhere (T233004)]] (duration: 18m 47s) [production]
21:40 <aqu@deploy1002> Started deploy [analytics/refinery@66736e1]: HDFS FSImage conversion to XML script [analytics/refinery@66736e1] [production]
21:39 <cwhite@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on logstash2035.codfw.wmnet with reason: host reimage [production]