451-500 of 10000 results (84ms)
2023-12-07 §
06:44 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission for hosts db1119.eqiad.wmnet [production]
06:35 <marostegui> Failover m5-master from dbproxy1021 to dbproxy1027 T351864 [production]
00:53 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host ms-be1082.eqiad.wmnet with OS bullseye [production]
00:53 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be1081.eqiad.wmnet with OS bullseye [production]
00:53 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be1080.eqiad.wmnet with OS bullseye [production]
00:53 <jclark@cumin1001> END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
00:53 <jclark@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
2023-12-06 §
23:56 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ms-be1082.eqiad.wmnet with OS bullseye [production]
23:47 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
23:42 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
23:25 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be1081.eqiad.wmnet with reason: host reimage [production]
23:23 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be1080.eqiad.wmnet with reason: host reimage [production]
23:20 <jclark@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be1081.eqiad.wmnet with reason: host reimage [production]
23:19 <jclark@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be1080.eqiad.wmnet with reason: host reimage [production]
23:03 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 34 days, 0:00:00 on wdqs1024.eqiad.wmnet with reason: T352878 [production]
23:03 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 34 days, 0:00:00 on wdqs1024.eqiad.wmnet with reason: T352878 [production]
22:50 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host ms-be1081.eqiad.wmnet with OS bullseye [production]
22:50 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host ms-be1082.eqiad.wmnet with OS bullseye [production]
22:50 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host ms-be1080.eqiad.wmnet with OS bullseye [production]
22:49 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['ms-be1080'] [production]
22:49 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['ms-be1082'] [production]
22:49 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['ms-be1081'] [production]
22:43 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ms-be1080'] [production]
22:43 <jclark@cumin1001> END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts ['ms-be1081'] [production]
22:43 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ms-be1081'] [production]
22:42 <jclark@cumin1001> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['ms-be1080'] [production]
22:42 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ms-be1082'] [production]
22:42 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ms-be1081'] [production]
22:42 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ms-be1080'] [production]
22:14 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be1082.mgmt.eqiad.wmnet with reboot policy FORCED [production]
22:11 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be1080.mgmt.eqiad.wmnet with reboot policy FORCED [production]
22:11 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be1081.mgmt.eqiad.wmnet with reboot policy FORCED [production]
21:56 <jclark@cumin1001> START - Cookbook sre.hosts.provision for host ms-be1082.mgmt.eqiad.wmnet with reboot policy FORCED [production]
21:52 <jclark@cumin1001> START - Cookbook sre.hosts.provision for host ms-be1080.mgmt.eqiad.wmnet with reboot policy FORCED [production]
21:51 <jclark@cumin1001> START - Cookbook sre.hosts.provision for host ms-be1081.mgmt.eqiad.wmnet with reboot policy FORCED [production]
21:51 <jclark@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:50 <jclark@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt ms-be - jclark@cumin1001" [production]
21:50 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt ms-be - jclark@cumin1001" [production]
21:47 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
21:45 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ms-be1082.mgmt.eqiad.wmnet with reboot policy FORCED [production]
21:43 <urbanecm@deploy2002> Finished scap: Backport for [[gerrit:980477|Correct links to beta feature (T352826)]], [[gerrit:980517|Beta Features: Allow Vector 2022 typography feature (T351339)]] (duration: 10m 51s) [production]
21:36 <urbanecm@deploy2002> urbanecm and jdlrobson: Continuing with sync [production]
21:35 <jclark@cumin1001> START - Cookbook sre.hosts.provision for host ms-be1082.mgmt.eqiad.wmnet with reboot policy FORCED [production]
21:35 <jclark@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:35 <jclark@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt ms-be - jclark@cumin1001" [production]
21:34 <urbanecm@deploy2002> urbanecm and jdlrobson: Backport for [[gerrit:980477|Correct links to beta feature (T352826)]], [[gerrit:980517|Beta Features: Allow Vector 2022 typography feature (T351339)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:34 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt ms-be - jclark@cumin1001" [production]
21:32 <urbanecm@deploy2002> Started scap: Backport for [[gerrit:980477|Correct links to beta feature (T352826)]], [[gerrit:980517|Beta Features: Allow Vector 2022 typography feature (T351339)]] [production]
21:32 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
21:31 <urbanecm@deploy2002> Finished scap: Backport for [[gerrit:980920|DiscussionTools: Rename config]] (duration: 10m 01s) [production]