2901-2950 of 10000 results (97ms)
2023-11-27 ยง
17:52 <vgutierrez> upload ipip-multiqueue-optimizer 0.2 to apt.wm.o (bullseye) - T351069 [production]
17:48 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirtlocal1002.eqiad.wmnet with OS bookworm [production]
17:25 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirtlocal1002.eqiad.wmnet with reason: host reimage [production]
17:22 <andrew@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirtlocal1002.eqiad.wmnet with reason: host reimage [production]
17:13 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['ganeti2033'] [production]
17:07 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ganeti2033'] [production]
17:07 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['ganeti2033'] [production]
17:07 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['kubernetes2058'] [production]
17:07 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['kubernetes2057'] [production]
17:07 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ganeti2033'] [production]
17:06 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['kubernetes2058'] [production]
17:06 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['kubernetes2057'] [production]
17:06 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirtlocal1002.eqiad.wmnet with OS bookworm [production]
16:56 <jdrewniak@deploy2002> Synchronized portals: Wikimedia Portals Update: [[gerrit:977735| Bumping portals to master (T128546)]] (duration: 06m 31s) [production]
16:52 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirtlocal1001.eqiad.wmnet with OS bookworm [production]
16:50 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: dbbackups::monitoring [production]
16:50 <jdrewniak@deploy2002> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:977735| Bumping portals to master (T128546)]] (duration: 07m 06s) [production]
16:49 <pt1979@cumin1001> START - Cookbook sre.hosts.reimage for host an-worker1157.eqiad.wmnet with OS bullseye [production]
16:43 <pt1979@cumin1001> END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host an-worker1157.eqiad.wmnet [production]
16:39 <pt1979@cumin1001> START - Cookbook sre.hosts.dhcp for host an-worker1157.eqiad.wmnet [production]
16:31 <vgutierrez> upload tcp-mss-clamper 0.2+deb12u1 to apt.wm.o (bookworm) [production]
16:30 <jbond@cumin1001> START - Cookbook sre.puppet.migrate-role for role: dbbackups::monitoring [production]
16:28 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2033.mgmt.codfw.wmnet with reboot policy FORCED [production]
16:28 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host kubernetes2058.mgmt.codfw.wmnet with reboot policy FORCED [production]
16:27 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host kubernetes2057.mgmt.codfw.wmnet with reboot policy FORCED [production]
16:21 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti2034.mgmt.codfw.wmnet with reboot policy FORCED [production]
16:20 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['logging-hd2003'] [production]
16:20 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['logging-hd2002'] [production]
16:20 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['logging-hd2001'] [production]
16:20 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['elastic2109'] [production]
16:17 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2034.mgmt.codfw.wmnet with reboot policy FORCED [production]
16:17 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2033.mgmt.codfw.wmnet with reboot policy FORCED [production]
16:17 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host kubernetes2058.mgmt.codfw.wmnet with reboot policy FORCED [production]
16:17 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host kubernetes2057.mgmt.codfw.wmnet with reboot policy FORCED [production]
16:15 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['logging-hd2003'] [production]
16:15 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['logging-hd2002'] [production]
16:15 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['logging-hd2001'] [production]
16:15 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2109'] [production]
16:14 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['kubernetes2060'] [production]
16:14 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['kubernetes2059'] [production]
16:14 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['logging-hd2003'] [production]
16:14 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['logging-hd2002'] [production]
16:14 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['logging-hd2001'] [production]
16:13 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['elastic2109'] [production]
16:13 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['kubernetes2060'] [production]
16:13 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['kubernetes2059'] [production]
16:13 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['logging-hd2003'] [production]
16:13 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['logging-hd2002'] [production]
16:12 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['logging-hd2001'] [production]
16:12 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2109'] [production]