3751-3800 of 10000 results (127ms)
2024-06-27 ยง
14:56 <cgoubert@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:55 <cgoubert@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming mw1365 to wikikube-worker1023 - cgoubert@cumin1002" [production]
14:54 <cgoubert@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming mw1365 to wikikube-worker1023 - cgoubert@cumin1002" [production]
14:53 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host deploy1003.eqiad.wmnet with OS bullseye [production]
14:52 <brennen@deploy1002> Finished deploy [phabricator/deployment@0df351e]: deploy phab1004 for minor update (duration: 00m 32s) [production]
14:52 <cgoubert@cumin1002> START - Cookbook sre.dns.netbox [production]
14:52 <cgoubert@cumin1002> START - Cookbook sre.hosts.rename from mw1365 to wikikube-worker1023 [production]
14:52 <brennen@deploy1002> Started deploy [phabricator/deployment@0df351e]: deploy phab1004 for minor update [production]
14:51 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from mw1359 to wikikube-worker1022 [production]
14:51 <cgoubert@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1022 [production]
14:51 <brennen@deploy1002> Finished deploy [phabricator/deployment@0df351e]: test deploy phab2002 (duration: 00m 34s) [production]
14:50 <brennen@deploy1002> Started deploy [phabricator/deployment@0df351e]: test deploy phab2002 [production]
14:50 <cgoubert@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1022 [production]
14:50 <cgoubert@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:50 <cgoubert@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming mw1359 to wikikube-worker1022 - cgoubert@cumin1002" [production]
14:48 <cgoubert@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming mw1359 to wikikube-worker1022 - cgoubert@cumin1002" [production]
14:46 <cmooney@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:50:00 on lsw1-e7-eqiad.mgmt with reason: prep JunOS upgrade lsw1-e7-eqiad [production]
14:46 <dcaro@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1007.eqiad.wmnet with reason: host reimage [production]
14:46 <cmooney@cumin1002> START - Cookbook sre.hosts.downtime for 0:50:00 on lsw1-e7-eqiad.mgmt with reason: prep JunOS upgrade lsw1-e7-eqiad [production]
14:46 <cgoubert@cumin1002> START - Cookbook sre.dns.netbox [production]
14:46 <cgoubert@cumin1002> START - Cookbook sre.hosts.rename from mw1359 to wikikube-worker1022 [production]
14:43 <dcaro@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1007.eqiad.wmnet with reason: host reimage [production]
14:38 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on es1037.eqiad.wmnet with reason: T365988 [production]
14:38 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on es1037.eqiad.wmnet with reason: T365988 [production]
14:37 <arnaudb@cumin1002> dbctl commit (dc=all): 'T365988 - depool es1037', diff saved to https://phabricator.wikimedia.org/P65531 and previous config saved to /var/cache/conftool/dbconfig/20240627-143741-arnaudb.json [production]
14:15 <fnegri@cumin1002> conftool action : set/pooled=no; selector: name=clouddb1015.eqiad.wmnet,service=s4 [production]
14:12 <dcaro@cumin1002> START - Cookbook sre.hosts.reimage for host cloudcephosd1007.eqiad.wmnet with OS bullseye [production]
13:54 <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:1043043|CommonSettings: Mark REL1_42 as stable (T359850)]] (duration: 08m 10s) [production]
13:48 <btullis@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
13:46 <urbanecm@deploy1002> Started scap: Backport for [[gerrit:1043043|CommonSettings: Mark REL1_42 as stable (T359850)]] [production]
13:46 <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:1050309|ptwiki: Enable CommunityConfiguration (T368310)]] (duration: 08m 58s) [production]
13:42 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirtlocal1001.eqiad.wmnet with OS bookworm [production]
13:41 <urbanecm@deploy1002> urbanecm: Continuing with sync [production]
13:40 <urbanecm> Run `mwscript extensions/GrowthExperiments/maintenance/migrateCommunityConfig.php --wiki=ptwiki --force` via mwdebug1001 (T368310) [production]
13:40 <jclark@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host deploy1003 [production]
13:39 <urbanecm@deploy1002> urbanecm: Backport for [[gerrit:1050309|ptwiki: Enable CommunityConfiguration (T368310)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:38 <jclark@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host deploy1003 [production]
13:37 <urbanecm@deploy1002> Started scap: Backport for [[gerrit:1050309|ptwiki: Enable CommunityConfiguration (T368310)]] [production]
13:36 <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:1042430|Enable local uploads for Gilaki Wikipedia (T364673)]], [[gerrit:1048855|[noop] Remove $wgRedirectScript, not used since MediaWiki 1.22]], [[gerrit:1048419|CommunityConfiguration: Log info and higher]] (duration: 10m 22s) [production]
13:35 <btullis@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
13:31 <urbanecm@deploy1002> urbanecm, tgr, nmw03: Continuing with sync [production]
13:28 <urbanecm@deploy1002> urbanecm, tgr, nmw03: Backport for [[gerrit:1042430|Enable local uploads for Gilaki Wikipedia (T364673)]], [[gerrit:1048855|[noop] Remove $wgRedirectScript, not used since MediaWiki 1.22]], [[gerrit:1048419|CommunityConfiguration: Log info and higher]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:25 <urbanecm@deploy1002> Started scap: Backport for [[gerrit:1042430|Enable local uploads for Gilaki Wikipedia (T364673)]], [[gerrit:1048855|[noop] Remove $wgRedirectScript, not used since MediaWiki 1.22]], [[gerrit:1048419|CommunityConfiguration: Log info and higher]] [production]
13:24 <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:1038742|[CheckUser] Stop writing old for event tables migration on all wikis (T360685)]], [[gerrit:1049970|testwiki: use shellbox-video for scaling video (T356241)]], [[gerrit:1049886|Add VK namespace alias to Azerbaijani Wikibooks (T368237)]] (duration: 16m 48s) [production]
13:19 <urbanecm@deploy1002> urbanecm, dreamrimmer, hnowlan, dreamyjazz: Continuing with sync [production]
13:12 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirtlocal1001.eqiad.wmnet with reason: host reimage [production]
13:10 <urbanecm@deploy1002> urbanecm, dreamrimmer, hnowlan, dreamyjazz: Backport for [[gerrit:1038742|[CheckUser] Stop writing old for event tables migration on all wikis (T360685)]], [[gerrit:1049970|testwiki: use shellbox-video for scaling video (T356241)]], [[gerrit:1049886|Add VK namespace alias to Azerbaijani Wikibooks (T368237)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:08 <andrew@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirtlocal1001.eqiad.wmnet with reason: host reimage [production]
13:08 <urbanecm@deploy1002> Started scap: Backport for [[gerrit:1038742|[CheckUser] Stop writing old for event tables migration on all wikis (T360685)]], [[gerrit:1049970|testwiki: use shellbox-video for scaling video (T356241)]], [[gerrit:1049886|Add VK namespace alias to Azerbaijani Wikibooks (T368237)]] [production]
13:02 <sukhe> A:dnsbox: remove 10.3.0.2/32 from /e/n/i [production]