4951-5000 of 10000 results (94ms)
2024-03-12 ยง
23:03 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
23:02 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
23:00 <tstarling@deploy2002> tstarling: Backport for [[gerrit:1010218|migrateBlocks.php: Skip existing IDs (T355034)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
22:57 <tstarling@deploy2002> Started scap: Backport for [[gerrit:1010218|migrateBlocks.php: Skip existing IDs (T355034)]] [production]
22:50 <tzatziki> removing 2 files for legal compliance [production]
19:58 <hashar@deploy2002> Finished scap: Backport for [[gerrit:1010568|WikiModule: Fix data structure when preloading title info (T359939)]] (duration: 13m 14s) [production]
19:48 <hashar@deploy2002> hashar and jforrester: Continuing with sync [production]
19:47 <hashar@deploy2002> hashar and jforrester: Backport for [[gerrit:1010568|WikiModule: Fix data structure when preloading title info (T359939)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
19:46 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
19:46 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
19:45 <hashar@deploy2002> Started scap: Backport for [[gerrit:1010568|WikiModule: Fix data structure when preloading title info (T359939)]] [production]
15:32 <eoghan@cumin1002> END (FAIL) - Cookbook sre.gitlab.failover (exit_code=93) Failover of gitlab from gitlab1004.wikimedia.org to gitlab1003.wikimedia.org [production]
14:58 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host dbprov2005.codfw.wmnet with OS bullseye [production]
14:58 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host dbprov2006.codfw.wmnet with OS bullseye [production]
14:43 <ladsgroup@cumin1002> dbctl commit (dc=all): 'db2146 (re)pooling @ 100%: Maint over', diff saved to https://phabricator.wikimedia.org/P58774 and previous config saved to /var/cache/conftool/dbconfig/20240312-144307-ladsgroup.json [production]
14:35 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
14:34 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
14:28 <ladsgroup@cumin1002> dbctl commit (dc=all): 'db2146 (re)pooling @ 75%: Maint over', diff saved to https://phabricator.wikimedia.org/P58773 and previous config saved to /var/cache/conftool/dbconfig/20240312-142801-ladsgroup.json [production]
14:26 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
14:26 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
14:22 <kharlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/ipoid: apply [production]
14:22 <kharlan@deploy2002> helmfile [codfw] START helmfile.d/services/ipoid: apply [production]
14:21 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
14:21 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
14:20 <kharlan@deploy2002> helmfile [eqiad] DONE helmfile.d/services/ipoid: apply [production]
14:20 <kharlan@deploy2002> helmfile [eqiad] START helmfile.d/services/ipoid: apply [production]
14:18 <kharlan@deploy2002> helmfile [staging] DONE helmfile.d/services/ipoid: apply [production]
14:18 <kharlan@deploy2002> helmfile [staging] START helmfile.d/services/ipoid: apply [production]
14:16 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on db1246.eqiad.wmnet with reason: db1246 depooled [production]
14:16 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on db1246.eqiad.wmnet with reason: db1246 depooled [production]
14:14 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
14:14 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
14:12 <ladsgroup@cumin1002> dbctl commit (dc=all): 'db2146 (re)pooling @ 25%: Maint over', diff saved to https://phabricator.wikimedia.org/P58772 and previous config saved to /var/cache/conftool/dbconfig/20240312-141255-ladsgroup.json [production]
14:00 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1246.eqiad.wmnet with reason: db1246 depooled [production]
14:00 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1246.eqiad.wmnet with reason: db1246 depooled [production]
13:58 <Lucas_WMDE> UTC afternoon backport+config window done [production]
13:57 <ladsgroup@cumin1002> dbctl commit (dc=all): 'db2146 (re)pooling @ 10%: Maint over', diff saved to https://phabricator.wikimedia.org/P58770 and previous config saved to /var/cache/conftool/dbconfig/20240312-135750-ladsgroup.json [production]
13:57 <logmsgbot> lucaswerkmeister-wmde@deploy2002 Finished scap: Backport for [[gerrit:1002639|Enable native math rendering options by default (T358803)]] (duration: 15m 05s) [production]
13:57 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Down', diff saved to https://phabricator.wikimedia.org/P58769 and previous config saved to /var/cache/conftool/dbconfig/20240312-135715-ladsgroup.json [production]
13:55 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1246', diff saved to https://phabricator.wikimedia.org/P58768 and previous config saved to /var/cache/conftool/dbconfig/20240312-135537-arnaudb.json [production]
13:52 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:52 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:47 <logmsgbot> lucaswerkmeister-wmde@deploy2002 lucaswerkmeister-wmde and physikerwelt: Continuing with sync [production]
13:45 <logmsgbot> lucaswerkmeister-wmde@deploy2002 lucaswerkmeister-wmde and physikerwelt: Backport for [[gerrit:1002639|Enable native math rendering options by default (T358803)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:42 <logmsgbot> lucaswerkmeister-wmde@deploy2002 Started scap: Backport for [[gerrit:1002639|Enable native math rendering options by default (T358803)]] [production]
13:39 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:39 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:38 <logmsgbot> lucaswerkmeister-wmde@deploy2002 Finished scap: Backport for [[gerrit:1009729|Add `suppressredirect` right to pagemover and filemover user groups in azwiki (T359614)]] (duration: 14m 13s) [production]
13:35 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:35 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]