201-250 of 10000 results (108ms)
2025-07-11 §
02:09 <andrew@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cloudcephosd1039.eqiad.wmnet [production]
02:01 <andrew@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cloudcephosd1039.eqiad.wmnet [production]
01:17 <andrew@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts cloudcephosd1038.eqiad.wmnet [production]
01:17 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcephosd1038.eqiad.wmnet [production]
01:03 <andrew@cumin2002> START - Cookbook sre.hosts.reboot-single for host cloudcephosd1038.eqiad.wmnet [production]
00:57 <andrew@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cloudcephosd1038.eqiad.wmnet [production]
00:55 <krinkle@deploy1003> Finished scap sync-world: Backport for [[gerrit:1167985|beta: Change FileRepo zone URL to upload.wikimedia.beta.wmcloud.org (T289318)]] (duration: 11m 30s) [production]
00:50 <krinkle@deploy1003> krinkle: Continuing with sync [production]
00:47 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcephosd1037.eqiad.wmnet with OS bookworm [production]
00:45 <krinkle@deploy1003> krinkle: Backport for [[gerrit:1167985|beta: Change FileRepo zone URL to upload.wikimedia.beta.wmcloud.org (T289318)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
00:43 <krinkle@deploy1003> Started scap sync-world: Backport for [[gerrit:1167985|beta: Change FileRepo zone URL to upload.wikimedia.beta.wmcloud.org (T289318)]] [production]
00:34 <krinkle@deploy1003> Finished scap sync-world: Backport for [[gerrit:1167259|beta: Move beta wikipedia canonical to beta.wmcloud.org (T289318)]] (duration: 12m 13s) [production]
00:29 <krinkle@deploy1003> krinkle: Continuing with sync [production]
00:27 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1037.eqiad.wmnet with reason: host reimage [production]
00:24 <krinkle@deploy1003> krinkle: Backport for [[gerrit:1167259|beta: Move beta wikipedia canonical to beta.wmcloud.org (T289318)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
00:22 <krinkle@deploy1003> Started scap sync-world: Backport for [[gerrit:1167259|beta: Move beta wikipedia canonical to beta.wmcloud.org (T289318)]] [production]
00:21 <andrew@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1037.eqiad.wmnet with reason: host reimage [production]
2025-07-10 §
23:59 <andrew@cumin2002> START - Cookbook sre.hosts.reimage for host cloudcephosd1037.eqiad.wmnet with OS bookworm [production]
23:57 <andrew@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts cloudcephosd1037.eqiad.wmnet [production]
23:57 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcephosd1037.eqiad.wmnet [production]
23:44 <andrew@cumin2002> START - Cookbook sre.hosts.reboot-single for host cloudcephosd1037.eqiad.wmnet [production]
23:03 <andrew@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cloudcephosd1037.eqiad.wmnet [production]
23:02 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcephosd1036.eqiad.wmnet with OS bookworm [production]
22:43 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1036.eqiad.wmnet with reason: host reimage [production]
22:39 <andrew@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1036.eqiad.wmnet with reason: host reimage [production]
22:30 <zabe@deploy1003> Finished scap sync-world: Backport for [[gerrit:1167944|Fix categorylinks read new query for excluded categories (T385890)]] (duration: 07m 59s) [production]
22:25 <zabe@deploy1003> zabe: Continuing with sync [production]
22:24 <zabe@deploy1003> zabe: Backport for [[gerrit:1167944|Fix categorylinks read new query for excluded categories (T385890)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
22:22 <zabe@deploy1003> Started scap sync-world: Backport for [[gerrit:1167944|Fix categorylinks read new query for excluded categories (T385890)]] [production]
22:16 <andrew@cumin2002> START - Cookbook sre.hosts.reimage for host cloudcephosd1036.eqiad.wmnet with OS bookworm [production]
22:13 <andrew@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts cloudcephosd1036.eqiad.wmnet [production]
22:13 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcephosd1036.eqiad.wmnet [production]
22:00 <andrew@cumin2002> START - Cookbook sre.hosts.reboot-single for host cloudcephosd1036.eqiad.wmnet [production]
21:55 <andrew@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cloudcephosd1036.eqiad.wmnet [production]
21:32 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcephosd1035.eqiad.wmnet with OS bookworm [production]
21:12 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1035.eqiad.wmnet with reason: host reimage [production]
21:06 <andrew@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1035.eqiad.wmnet with reason: host reimage [production]
20:54 <jforrester@deploy1003> Finished scap sync-world: Backport for [[gerrit:1137480|Use `sul` dblist in InitialiseSettings]] (duration: 11m 43s) [production]
20:48 <jforrester@deploy1003> jforrester, bd808: Continuing with sync [production]
20:44 <andrew@cumin2002> START - Cookbook sre.hosts.reimage for host cloudcephosd1035.eqiad.wmnet with OS bookworm [production]
20:44 <jforrester@deploy1003> jforrester, bd808: Backport for [[gerrit:1137480|Use `sul` dblist in InitialiseSettings]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
20:42 <andrew@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts cloudcephosd1035.eqiad.wmnet [production]
20:42 <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1137480|Use `sul` dblist in InitialiseSettings]] [production]
20:41 <andrew@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cloudcephosd1035.eqiad.wmnet [production]
20:39 <robh@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts cloudcephosd1035.eqiad.wmnet [production]
20:39 <robh@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcephosd1035.eqiad.wmnet [production]
20:25 <root@cumin1003> END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts cloudcephosd1035.eqiad.wmnet [production]
20:25 <root@cumin1003> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cloudcephosd1035.eqiad.wmnet [production]
20:25 <robh@cumin2002> START - Cookbook sre.hosts.reboot-single for host cloudcephosd1035.eqiad.wmnet [production]
20:24 <root@cumin1003> END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts cloudcephosd1035.eqiad.wmnet [production]