1601-1650 of 10000 results (109ms)
2024-10-29 ยง
13:57 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2036.codfw.wmnet [production]
13:57 <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1084112|AuthManagerStatsdHandler: Add label for wiki (T375505)]], [[gerrit:1084111|AuthManagerStatsdHandler: Add label for wiki (T375505)]] [production]
13:56 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2036.codfw.wmnet [production]
13:48 <jforrester@deploy2002> Finished scap sync-world: Backport for [[gerrit:1084110|fix ibawiki's tagline svg path]] (duration: 07m 41s) [production]
13:47 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 16347 [production]
13:46 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'configure' for AS: 16347 [production]
13:45 <ayounsi@cumin1002> END (FAIL) - Cookbook sre.network.peering (exit_code=99) with action 'email' for AS: 16347 [production]
13:45 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 16347 [production]
13:43 <jforrester@deploy2002> jforrester, hamishz: Continuing with sync [production]
13:42 <jforrester@deploy2002> jforrester, hamishz: Backport for [[gerrit:1084110|fix ibawiki's tagline svg path]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:42 <moritzm> installing ghoscript security updates [production]
13:40 <jforrester@deploy2002> Started scap sync-world: Backport for [[gerrit:1084110|fix ibawiki's tagline svg path]] [production]
13:38 <jforrester@deploy2002> Finished scap sync-world: Backport for [[gerrit:1082878|Allow admins on testwiki to grant and remove upwizcampeditors (T378067)]], [[gerrit:1082444|nlwiki, commonswiki, wikidata: lift IP cap for edit-a-thon (T377930)]] (duration: 08m 03s) [production]
13:34 <jforrester@deploy2002> dreamrimmer, superzerocool, jforrester: Continuing with sync [production]
13:33 <jforrester@deploy2002> dreamrimmer, superzerocool, jforrester: Backport for [[gerrit:1082878|Allow admins on testwiki to grant and remove upwizcampeditors (T378067)]], [[gerrit:1082444|nlwiki, commonswiki, wikidata: lift IP cap for edit-a-thon (T377930)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:31 <jforrester@deploy2002> Started scap sync-world: Backport for [[gerrit:1082878|Allow admins on testwiki to grant and remove upwizcampeditors (T378067)]], [[gerrit:1082444|nlwiki, commonswiki, wikidata: lift IP cap for edit-a-thon (T377930)]] [production]
13:31 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2211 (re)pooling @ 100%: post clone repool', diff saved to https://phabricator.wikimedia.org/P70612 and previous config saved to /var/cache/conftool/dbconfig/20241029-132956-arnaudb.json [production]
13:30 <mszabo@deploy2002> helmfile [codfw] DONE helmfile.d/services/ipoid: apply [production]
13:30 <mszabo@deploy2002> helmfile [codfw] START helmfile.d/services/ipoid: apply [production]
13:29 <mszabo@deploy2002> helmfile [eqiad] DONE helmfile.d/services/ipoid: apply [production]
13:28 <jforrester@deploy2002> Finished scap sync-world: Backport for [[gerrit:1083952|annwiki: Add logo (T377535)]], [[gerrit:1084100|kgewiki: Add logo (T377075)]], [[gerrit:1084097|shnwikinews: Add logo (T377543)]], [[gerrit:1084094|gorwikiquote: Add logo (T377542)]], [[gerrit:1084090|moswiki: Add logo (T377539)]], [[gerrit:1084088|ibawiki: Add logo (T377538)]], [[gerrit:1084069|rskwiki: Add logo (T377536)]], [[gerrit:10840 [production]
13:28 <mszabo@deploy2002> helmfile [eqiad] START helmfile.d/services/ipoid: apply [production]
13:27 <mszabo@deploy2002> helmfile [staging] DONE helmfile.d/services/ipoid: apply [production]
13:26 <mszabo@deploy2002> helmfile [staging] START helmfile.d/services/ipoid: apply [production]
13:24 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2035.codfw.wmnet [production]
13:23 <jforrester@deploy2002> jforrester, hamishz: Continuing with sync [production]
13:22 <jforrester@deploy2002> jforrester, hamishz: Backport for [[gerrit:1083952|annwiki: Add logo (T377535)]], [[gerrit:1084100|kgewiki: Add logo (T377075)]], [[gerrit:1084097|shnwikinews: Add logo (T377543)]], [[gerrit:1084094|gorwikiquote: Add logo (T377542)]], [[gerrit:1084090|moswiki: Add logo (T377539)]], [[gerrit:1084088|ibawiki: Add logo (T377538)]], [[gerrit:1084069|rskwiki: Add logo (T377536)]], [[gerrit:1084079|td [production]
13:20 <jforrester@deploy2002> Started scap sync-world: Backport for [[gerrit:1083952|annwiki: Add logo (T377535)]], [[gerrit:1084100|kgewiki: Add logo (T377075)]], [[gerrit:1084097|shnwikinews: Add logo (T377543)]], [[gerrit:1084094|gorwikiquote: Add logo (T377542)]], [[gerrit:1084090|moswiki: Add logo (T377539)]], [[gerrit:1084088|ibawiki: Add logo (T377538)]], [[gerrit:1084069|rskwiki: Add logo (T377536)]], [[gerrit:108407 [production]
13:19 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2035.codfw.wmnet [production]
13:17 <jforrester@deploy2002> Finished scap sync-world: Backport for [[gerrit:1082256|ExtensionDistributor: Mark 1.43 as beta (T372322)]], [[gerrit:1084096|ExtensionDistributor: Remove EOL 1.40 (T364989)]], [[gerrit:1083294|enwiktionary: Enable mobile page tabs for non logged in users (T377648)]] (duration: 12m 41s) [production]
13:14 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2211 (re)pooling @ 75%: post clone repool', diff saved to https://phabricator.wikimedia.org/P70610 and previous config saved to /var/cache/conftool/dbconfig/20241029-131451-arnaudb.json [production]
13:11 <jforrester@deploy2002> zabe, macfan4000, hamishz, jforrester: Continuing with sync [production]
13:10 <jforrester@deploy2002> zabe, macfan4000, hamishz, jforrester: Backport for [[gerrit:1082256|ExtensionDistributor: Mark 1.43 as beta (T372322)]], [[gerrit:1084096|ExtensionDistributor: Remove EOL 1.40 (T364989)]], [[gerrit:1083294|enwiktionary: Enable mobile page tabs for non logged in users (T377648)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:10 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2035.codfw.wmnet [production]
13:07 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2035.codfw.wmnet [production]
13:05 <jforrester@deploy2002> Started scap sync-world: Backport for [[gerrit:1082256|ExtensionDistributor: Mark 1.43 as beta (T372322)]], [[gerrit:1084096|ExtensionDistributor: Remove EOL 1.40 (T364989)]], [[gerrit:1083294|enwiktionary: Enable mobile page tabs for non logged in users (T377648)]] [production]
12:59 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2211 (re)pooling @ 50%: post clone repool', diff saved to https://phabricator.wikimedia.org/P70607 and previous config saved to /var/cache/conftool/dbconfig/20241029-125945-arnaudb.json [production]
12:50 <moritzm> installing Apache security updates [production]
12:44 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2211 (re)pooling @ 25%: post clone repool', diff saved to https://phabricator.wikimedia.org/P70606 and previous config saved to /var/cache/conftool/dbconfig/20241029-124440-arnaudb.json [production]
12:43 <claime> Manually relaunched import-wikitech.sh on wikitech-static - T374114 [production]
12:42 <claime> Killed dead and stacked import-wikitech.sh processes on wikitech-static - T374114 [production]
12:28 <andrewtavis-wmde@deploy2002> Finished deploy [airflow-dags/wmde@d85a93c]: (no justification provided) (duration: 00m 30s) [production]
12:27 <andrewtavis-wmde@deploy2002> Started deploy [airflow-dags/wmde@d85a93c]: (no justification provided) [production]
12:08 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2015.codfw.wmnet [production]
12:04 <cgoubert@deploy2002> Finished scap sync-world: T377958 - full mediawiki image rebuild and deployment to add helper scripts for mwcron, mwscript (duration: 29m 44s) [production]
11:39 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2044.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:39 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2044.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:36 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2041.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:36 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2041.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:36 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2040.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]