1951-2000 of 10000 results (107ms)
2024-10-29 ยง
13:30 <mszabo@deploy2002> helmfile [codfw] DONE helmfile.d/services/ipoid: apply [production]
13:30 <mszabo@deploy2002> helmfile [codfw] START helmfile.d/services/ipoid: apply [production]
13:29 <mszabo@deploy2002> helmfile [eqiad] DONE helmfile.d/services/ipoid: apply [production]
13:28 <jforrester@deploy2002> Finished scap sync-world: Backport for [[gerrit:1083952|annwiki: Add logo (T377535)]], [[gerrit:1084100|kgewiki: Add logo (T377075)]], [[gerrit:1084097|shnwikinews: Add logo (T377543)]], [[gerrit:1084094|gorwikiquote: Add logo (T377542)]], [[gerrit:1084090|moswiki: Add logo (T377539)]], [[gerrit:1084088|ibawiki: Add logo (T377538)]], [[gerrit:1084069|rskwiki: Add logo (T377536)]], [[gerrit:10840 [production]
13:28 <mszabo@deploy2002> helmfile [eqiad] START helmfile.d/services/ipoid: apply [production]
13:27 <mszabo@deploy2002> helmfile [staging] DONE helmfile.d/services/ipoid: apply [production]
13:26 <mszabo@deploy2002> helmfile [staging] START helmfile.d/services/ipoid: apply [production]
13:24 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2035.codfw.wmnet [production]
13:23 <jforrester@deploy2002> jforrester, hamishz: Continuing with sync [production]
13:22 <jforrester@deploy2002> jforrester, hamishz: Backport for [[gerrit:1083952|annwiki: Add logo (T377535)]], [[gerrit:1084100|kgewiki: Add logo (T377075)]], [[gerrit:1084097|shnwikinews: Add logo (T377543)]], [[gerrit:1084094|gorwikiquote: Add logo (T377542)]], [[gerrit:1084090|moswiki: Add logo (T377539)]], [[gerrit:1084088|ibawiki: Add logo (T377538)]], [[gerrit:1084069|rskwiki: Add logo (T377536)]], [[gerrit:1084079|td [production]
13:20 <jforrester@deploy2002> Started scap sync-world: Backport for [[gerrit:1083952|annwiki: Add logo (T377535)]], [[gerrit:1084100|kgewiki: Add logo (T377075)]], [[gerrit:1084097|shnwikinews: Add logo (T377543)]], [[gerrit:1084094|gorwikiquote: Add logo (T377542)]], [[gerrit:1084090|moswiki: Add logo (T377539)]], [[gerrit:1084088|ibawiki: Add logo (T377538)]], [[gerrit:1084069|rskwiki: Add logo (T377536)]], [[gerrit:108407 [production]
13:19 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2035.codfw.wmnet [production]
13:17 <jforrester@deploy2002> Finished scap sync-world: Backport for [[gerrit:1082256|ExtensionDistributor: Mark 1.43 as beta (T372322)]], [[gerrit:1084096|ExtensionDistributor: Remove EOL 1.40 (T364989)]], [[gerrit:1083294|enwiktionary: Enable mobile page tabs for non logged in users (T377648)]] (duration: 12m 41s) [production]
13:14 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2211 (re)pooling @ 75%: post clone repool', diff saved to https://phabricator.wikimedia.org/P70610 and previous config saved to /var/cache/conftool/dbconfig/20241029-131451-arnaudb.json [production]
13:11 <jforrester@deploy2002> zabe, macfan4000, hamishz, jforrester: Continuing with sync [production]
13:10 <jforrester@deploy2002> zabe, macfan4000, hamishz, jforrester: Backport for [[gerrit:1082256|ExtensionDistributor: Mark 1.43 as beta (T372322)]], [[gerrit:1084096|ExtensionDistributor: Remove EOL 1.40 (T364989)]], [[gerrit:1083294|enwiktionary: Enable mobile page tabs for non logged in users (T377648)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:10 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2035.codfw.wmnet [production]
13:07 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2035.codfw.wmnet [production]
13:05 <jforrester@deploy2002> Started scap sync-world: Backport for [[gerrit:1082256|ExtensionDistributor: Mark 1.43 as beta (T372322)]], [[gerrit:1084096|ExtensionDistributor: Remove EOL 1.40 (T364989)]], [[gerrit:1083294|enwiktionary: Enable mobile page tabs for non logged in users (T377648)]] [production]
12:59 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2211 (re)pooling @ 50%: post clone repool', diff saved to https://phabricator.wikimedia.org/P70607 and previous config saved to /var/cache/conftool/dbconfig/20241029-125945-arnaudb.json [production]
12:50 <moritzm> installing Apache security updates [production]
12:44 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2211 (re)pooling @ 25%: post clone repool', diff saved to https://phabricator.wikimedia.org/P70606 and previous config saved to /var/cache/conftool/dbconfig/20241029-124440-arnaudb.json [production]
12:43 <claime> Manually relaunched import-wikitech.sh on wikitech-static - T374114 [production]
12:42 <claime> Killed dead and stacked import-wikitech.sh processes on wikitech-static - T374114 [production]
12:28 <andrewtavis-wmde@deploy2002> Finished deploy [airflow-dags/wmde@d85a93c]: (no justification provided) (duration: 00m 30s) [production]
12:27 <andrewtavis-wmde@deploy2002> Started deploy [airflow-dags/wmde@d85a93c]: (no justification provided) [production]
12:08 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2015.codfw.wmnet [production]
12:04 <cgoubert@deploy2002> Finished scap sync-world: T377958 - full mediawiki image rebuild and deployment to add helper scripts for mwcron, mwscript (duration: 29m 44s) [production]
11:39 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2044.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:39 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2044.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:36 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2041.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:36 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2041.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:36 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2040.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:36 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2040.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:35 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2039.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:35 <cgoubert@deploy2002> Started scap sync-world: T377958 - full mediawiki image rebuild and deployment to add helper scripts for mwcron, mwscript [production]
11:35 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2039.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:35 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2038.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:34 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2038.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:33 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2037.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:33 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2037.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:32 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2036.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:32 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti2036.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:30 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1052.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:30 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti1052.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:29 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1051.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:29 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti1051.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:29 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1050.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:29 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host ganeti1050.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:28 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1049.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]