1-50 of 10000 results (68ms)
2023-10-04 §
23:44 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1067.eqiad.wmnet with OS bullseye [production]
23:38 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1064.eqiad.wmnet with OS bullseye [production]
23:32 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1063.eqiad.wmnet with OS bullseye [production]
22:40 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1065.eqiad.wmnet with OS bullseye [production]
22:40 <jclark@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
22:39 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
22:24 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1065.eqiad.wmnet with reason: host reimage [production]
22:23 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1067.eqiad.wmnet with OS bullseye [production]
22:21 <jclark@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1065.eqiad.wmnet with reason: host reimage [production]
22:18 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1064.eqiad.wmnet with OS bullseye [production]
22:13 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host kubernetes2054.codfw.wmnet with OS bullseye [production]
22:11 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1063.eqiad.wmnet with OS bullseye [production]
22:06 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1067.eqiad.wmnet with OS bullseye [production]
22:06 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1064.eqiad.wmnet with OS bullseye [production]
22:06 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1063.eqiad.wmnet with OS bullseye [production]
22:05 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1065.eqiad.wmnet with OS bullseye [production]
22:02 <urandom> starting Cassandra rebuild, restbase1030-b — T346803 [production]
22:02 <brennen@deploy2002> Finished scap: Backport for [[gerrit:963351|Revert "Deprecate TOC mutation in OutputPageParserOutput hook" (T348134)]] (duration: 09m 13s) [production]
21:59 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host cloudvirt1065.eqiad.wmnet with OS bullseye [production]
21:58 <volans> uploaded spicerack_7.3.1 to apt.wikimedia.org bullseye-wikimedia [production]
21:56 <brennen@deploy2002> brennen and ssastry: Continuing with sync [production]
21:54 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1066.eqiad.wmnet with OS bullseye [production]
21:54 <jclark@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
21:54 <brennen@deploy2002> brennen and ssastry: Backport for [[gerrit:963351|Revert "Deprecate TOC mutation in OutputPageParserOutput hook" (T348134)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:53 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
21:53 <brennen@deploy2002> Started scap: Backport for [[gerrit:963351|Revert "Deprecate TOC mutation in OutputPageParserOutput hook" (T348134)]] [production]
21:46 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1065.eqiad.wmnet with OS bullseye [production]
21:44 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host cloudvirt1065.eqiad.wmnet with OS bullseye [production]
21:40 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1062.eqiad.wmnet with OS bullseye [production]
21:40 <jclark@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
21:38 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
21:38 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1066.eqiad.wmnet with reason: host reimage [production]
21:34 <jclark@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1066.eqiad.wmnet with reason: host reimage [production]
21:23 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1062.eqiad.wmnet with reason: host reimage [production]
21:20 <jclark@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1062.eqiad.wmnet with reason: host reimage [production]
21:02 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1062.eqiad.wmnet with OS bullseye [production]
20:59 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host cloudvirt1062.eqiad.wmnet with OS bullseye [production]
20:58 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host kubernetes2054.codfw.wmnet with OS bullseye [production]
20:54 <urbanecm@deploy2002> Finished scap: Backport for [[gerrit:963347|SpecialManageMentors: Skip OOUI initialization when transcluding (T346760)]], [[gerrit:963348|SpecialManageMentors: Skip OOUI initialization when transcluding (T346760)]], [[gerrit:963349|Fix phan for GrowthExperiments (T347571)]] (duration: 07m 49s) [production]
20:48 <urbanecm@deploy2002> urbanecm: Continuing with sync [production]
20:48 <urbanecm@deploy2002> urbanecm: Backport for [[gerrit:963347|SpecialManageMentors: Skip OOUI initialization when transcluding (T346760)]], [[gerrit:963348|SpecialManageMentors: Skip OOUI initialization when transcluding (T346760)]], [[gerrit:963349|Fix phan for GrowthExperiments (T347571)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:46 <urbanecm@deploy2002> Started scap: Backport for [[gerrit:963347|SpecialManageMentors: Skip OOUI initialization when transcluding (T346760)]], [[gerrit:963348|SpecialManageMentors: Skip OOUI initialization when transcluding (T346760)]], [[gerrit:963349|Fix phan for GrowthExperiments (T347571)]] [production]
20:46 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1067.eqiad.wmnet with OS bullseye [production]
20:46 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1066.eqiad.wmnet with OS bullseye [production]
20:46 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1065.eqiad.wmnet with OS bullseye [production]
20:46 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1064.eqiad.wmnet with OS bullseye [production]
20:45 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1063.eqiad.wmnet with OS bullseye [production]
20:45 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1062.eqiad.wmnet with OS bullseye [production]
20:45 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host cloudvirt1065.eqiad.wmnet with OS bullseye [production]
20:21 <eileen> re-enable process control (more better hopefully) config revision changed from 89231b1b to d66626f6 [production]