5451-5500 of 10000 results (70ms)
2022-05-25 ยง
08:32 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
08:32 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
08:32 <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:799265|Set templatelinks migration to read new everywhere except three wikis (T306673)]] (duration: 00m 50s) [production]
08:29 <kharlan@deploy1002> Synchronized php-1.39.0-wmf.12/extensions/GrowthExperiments/modules/ext.growthExperiments.StructuredTask/addimage: Backport: [[gerrit:799008|Add an image: Attach view image details button to .mw-ge-recommendedImage-imageWrapper (T309152)]] (duration: 00m 51s) [production]
08:27 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host schema2003.codfw.wmnet [production]
08:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P28481 and previous config saved to /var/cache/conftool/dbconfig/20220525-081908-ladsgroup.json [production]
08:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1132 to install 10.6.8', diff saved to https://phabricator.wikimedia.org/P28480 and previous config saved to /var/cache/conftool/dbconfig/20220525-081901-marostegui.json [production]
08:18 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host schema2003.codfw.wmnet [production]
08:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1130 (T298560)', diff saved to https://phabricator.wikimedia.org/P28479 and previous config saved to /var/cache/conftool/dbconfig/20220525-080829-ladsgroup.json [production]
08:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P28478 and previous config saved to /var/cache/conftool/dbconfig/20220525-080403-ladsgroup.json [production]
08:00 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:59 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:57 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2021.codfw.wmnet [production]
07:51 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2021.codfw.wmnet [production]
07:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1156 (T303603)', diff saved to https://phabricator.wikimedia.org/P28477 and previous config saved to /var/cache/conftool/dbconfig/20220525-074856-ladsgroup.json [production]
07:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1156 (T303603)', diff saved to https://phabricator.wikimedia.org/P28476 and previous config saved to /var/cache/conftool/dbconfig/20220525-074205-ladsgroup.json [production]
07:42 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
07:42 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
07:41 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1156.eqiad.wmnet with reason: Maintenance [production]
07:41 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1156.eqiad.wmnet with reason: Maintenance [production]
07:36 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
07:36 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
07:36 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
07:36 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
07:31 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
07:31 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
07:31 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
07:30 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
07:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
07:29 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
07:23 <kart_> Config: [[gerrit:798389|Enable Section Translation for Hindi in testwiki (T308834)]] [production]
07:11 <kart_> Config: [[gerrit:797977|Enable Content and Section Translation in Serbian and Zulu Wikipedias (T304834 T304858)]] [production]
06:38 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 100%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28475 and previous config saved to /var/cache/conftool/dbconfig/20220525-063856-root.json [production]
06:23 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 75%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28474 and previous config saved to /var/cache/conftool/dbconfig/20220525-062352-root.json [production]
06:20 <elukey> `elukey@an-tool1011:~$ sudo systemctl reset-failed ifup@ens13.service` - T273026 [production]
06:08 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 50%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28473 and previous config saved to /var/cache/conftool/dbconfig/20220525-060848-root.json [production]
05:53 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 25%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28472 and previous config saved to /var/cache/conftool/dbconfig/20220525-055344-root.json [production]
05:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1130 (T298560)', diff saved to https://phabricator.wikimedia.org/P28471 and previous config saved to /var/cache/conftool/dbconfig/20220525-054135-ladsgroup.json [production]
05:41 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1130.eqiad.wmnet with reason: Maintenance [production]
05:41 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1130.eqiad.wmnet with reason: Maintenance [production]
05:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T298560)', diff saved to https://phabricator.wikimedia.org/P28470 and previous config saved to /var/cache/conftool/dbconfig/20220525-054127-ladsgroup.json [production]
05:38 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 10%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28469 and previous config saved to /var/cache/conftool/dbconfig/20220525-053840-root.json [production]
05:30 <marostegui> Rename revision_actor_temp on s1 T307906 [production]
05:26 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P28468 and previous config saved to /var/cache/conftool/dbconfig/20220525-052622-ladsgroup.json [production]
05:23 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 5%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28467 and previous config saved to /var/cache/conftool/dbconfig/20220525-052336-root.json [production]
05:18 <marostegui> Rename revision_actor_temp on s5 T307906 [production]
05:14 <marostegui> Rename revision_actor_temp on s7 T307906 [production]
05:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P28466 and previous config saved to /var/cache/conftool/dbconfig/20220525-051117-ladsgroup.json [production]
05:08 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 1%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28465 and previous config saved to /var/cache/conftool/dbconfig/20220525-050833-root.json [production]
05:06 <marostegui> Install 10.4.25 on db1143 T308915 [production]