3051-3100 of 10000 results (76ms)
2023-10-04 §
03:55 <kartik@deploy2002> helmfile [codfw] START helmfile.d/services/cxserver: apply [production]
03:51 <kartik@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cxserver: apply [production]
03:51 <kartik@deploy2002> helmfile [eqiad] START helmfile.d/services/cxserver: apply [production]
03:48 <kartik@deploy2002> helmfile [staging] DONE helmfile.d/services/cxserver: apply [production]
03:48 <kartik@deploy2002> helmfile [staging] START helmfile.d/services/cxserver: apply [production]
2023-10-03 §
23:43 <arnaudb@cumin1001> dbctl commit (dc=all): 'Depooling db2162 (T343198)', diff saved to https://phabricator.wikimedia.org/P52812 and previous config saved to /var/cache/conftool/dbconfig/20231003-234343-arnaudb.json [production]
23:43 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2162.codfw.wmnet with reason: Maintenance [production]
23:43 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2162.codfw.wmnet with reason: Maintenance [production]
23:43 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T343198)', diff saved to https://phabricator.wikimedia.org/P52811 and previous config saved to /var/cache/conftool/dbconfig/20231003-234322-arnaudb.json [production]
23:28 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P52810 and previous config saved to /var/cache/conftool/dbconfig/20231003-232815-arnaudb.json [production]
23:13 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P52809 and previous config saved to /var/cache/conftool/dbconfig/20231003-231309-arnaudb.json [production]
22:58 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T343198)', diff saved to https://phabricator.wikimedia.org/P52808 and previous config saved to /var/cache/conftool/dbconfig/20231003-225803-arnaudb.json [production]
22:22 <jdrewniak@deploy2002> Finished scap: Backport for [[gerrit:963043|Web typography prototype survey (T347208)]], [[gerrit:963137|Correct a recently-added message]], [[gerrit:963138|[Prototype] Change i18n message (T347208)]] (duration: 39m 08s) [production]
22:11 <jdrewniak@deploy2002> jdrewniak: Continuing with sync [production]
22:01 <jdrewniak@deploy2002> jdrewniak: Backport for [[gerrit:963043|Web typography prototype survey (T347208)]], [[gerrit:963137|Correct a recently-added message]], [[gerrit:963138|[Prototype] Change i18n message (T347208)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:43 <jdrewniak@deploy2002> Started scap: Backport for [[gerrit:963043|Web typography prototype survey (T347208)]], [[gerrit:963137|Correct a recently-added message]], [[gerrit:963138|[Prototype] Change i18n message (T347208)]] [production]
21:32 <jdrewniak@deploy2002> Finished scap: Backport for [[gerrit:962684|Promote several Wikipedias to Vector 2022 as default skin (T347321)]] (duration: 09m 26s) [production]
21:26 <jdrewniak@deploy2002> jdlrobson and jdrewniak: Continuing with sync [production]
21:24 <jdrewniak@deploy2002> jdlrobson and jdrewniak: Backport for [[gerrit:962684|Promote several Wikipedias to Vector 2022 as default skin (T347321)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:23 <jdrewniak@deploy2002> Started scap: Backport for [[gerrit:962684|Promote several Wikipedias to Vector 2022 as default skin (T347321)]] [production]
20:56 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1004.eqiad.wmnet with OS bullseye [production]
20:56 <eileen> tools upgraded from 130ca87e to 2e19cd39 [production]
20:50 <jdrewniak@deploy2002> Finished scap: Backport for [[gerrit:944978|Re-enable Extension:ParserMigration on labs (T333179)]] (duration: 38m 52s) [production]
20:49 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1003.eqiad.wmnet with OS bullseye [production]
20:35 <jdrewniak@deploy2002> jdrewniak and sbailey: Continuing with sync [production]
20:34 <jdrewniak@deploy2002> jdrewniak and sbailey: Backport for [[gerrit:944978|Re-enable Extension:ParserMigration on labs (T333179)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:16 <fabfur> merged https://gerrit.wikimedia.org/r/c/operations/puppet/+/963081 (T347837). `purged` daemon will be restarted by puppet in eqsin in the next 30m [production]
20:11 <jdrewniak@deploy2002> Started scap: Backport for [[gerrit:944978|Re-enable Extension:ParserMigration on labs (T333179)]] [production]
19:41 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bullseye [production]
19:38 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1004.eqiad.wmnet with OS bullseye [production]
19:16 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bullseye [production]
19:16 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1003.eqiad.wmnet with OS bullseye [production]
19:16 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1004.eqiad.wmnet with OS bullseye [production]
19:15 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1003.eqiad.wmnet with OS bullseye [production]
19:15 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bullseye [production]
19:15 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1003.eqiad.wmnet with OS bullseye [production]
18:48 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1003.eqiad.wmnet with OS bullseye [production]
18:25 <jhuneidi@deploy2002> rebuilt and synchronized wikiversions files: group0 wikis to 1.41.0-wmf.29 refs T347080 [production]
18:13 <jhuneidi@deploy2002> Pruned MediaWiki: 1.41.0-wmf.27 (duration: 02m 14s) [production]
18:11 <jhuneidi@deploy2002> Finished scap: testwikis wikis to 1.41.0-wmf.29 refs T347080 (duration: 43m 24s) [production]
17:34 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1004.eqiad.wmnet with OS bullseye [production]
17:34 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bullseye [production]
17:33 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1004.eqiad.wmnet with OS bullseye [production]
17:33 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bullseye [production]
17:33 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1003.eqiad.wmnet with OS bullseye [production]
17:28 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for restbase1027.eqiad.wmnet [production]
17:28 <eevans@cumin1001> START - Cookbook sre.hosts.remove-downtime for restbase1027.eqiad.wmnet [production]
17:27 <jhuneidi@deploy2002> Started scap: testwikis wikis to 1.41.0-wmf.29 refs T347080 [production]
17:17 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1027.eqiad.wmnet with OS bullseye [production]
17:09 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1003.eqiad.wmnet with OS bullseye [production]