|
2022-05-25
ยง
|
| 08:19 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1132 to install 10.6.8', diff saved to https://phabricator.wikimedia.org/P28480 and previous config saved to /var/cache/conftool/dbconfig/20220525-081901-marostegui.json |
[production] |
| 08:18 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host schema2003.codfw.wmnet |
[production] |
| 08:08 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1130 (T298560)', diff saved to https://phabricator.wikimedia.org/P28479 and previous config saved to /var/cache/conftool/dbconfig/20220525-080829-ladsgroup.json |
[production] |
| 08:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P28478 and previous config saved to /var/cache/conftool/dbconfig/20220525-080403-ladsgroup.json |
[production] |
| 08:00 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:59 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:57 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2021.codfw.wmnet |
[production] |
| 07:51 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2021.codfw.wmnet |
[production] |
| 07:48 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1156 (T303603)', diff saved to https://phabricator.wikimedia.org/P28477 and previous config saved to /var/cache/conftool/dbconfig/20220525-074856-ladsgroup.json |
[production] |
| 07:42 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1156 (T303603)', diff saved to https://phabricator.wikimedia.org/P28476 and previous config saved to /var/cache/conftool/dbconfig/20220525-074205-ladsgroup.json |
[production] |
| 07:42 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:42 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:41 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1156.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:41 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1156.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:36 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1102.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:36 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1102.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:36 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1133.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:36 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db1133.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:31 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1139.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:31 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1139.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:31 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 07:30 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 07:30 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 07:29 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 07:23 |
<kart_> |
Config: [[gerrit:798389|Enable Section Translation for Hindi in testwiki (T308834)]] |
[production] |
| 07:11 |
<kart_> |
Config: [[gerrit:797977|Enable Content and Section Translation in Serbian and Zulu Wikipedias (T304834 T304858)]] |
[production] |
| 06:38 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 100%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28475 and previous config saved to /var/cache/conftool/dbconfig/20220525-063856-root.json |
[production] |
| 06:23 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 75%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28474 and previous config saved to /var/cache/conftool/dbconfig/20220525-062352-root.json |
[production] |
| 06:20 |
<elukey> |
`elukey@an-tool1011:~$ sudo systemctl reset-failed ifup@ens13.service` - T273026 |
[production] |
| 06:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 50%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28473 and previous config saved to /var/cache/conftool/dbconfig/20220525-060848-root.json |
[production] |
| 05:53 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 25%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28472 and previous config saved to /var/cache/conftool/dbconfig/20220525-055344-root.json |
[production] |
| 05:41 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1130 (T298560)', diff saved to https://phabricator.wikimedia.org/P28471 and previous config saved to /var/cache/conftool/dbconfig/20220525-054135-ladsgroup.json |
[production] |
| 05:41 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1130.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:41 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1130.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:41 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T298560)', diff saved to https://phabricator.wikimedia.org/P28470 and previous config saved to /var/cache/conftool/dbconfig/20220525-054127-ladsgroup.json |
[production] |
| 05:38 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 10%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28469 and previous config saved to /var/cache/conftool/dbconfig/20220525-053840-root.json |
[production] |
| 05:30 |
<marostegui> |
Rename revision_actor_temp on s1 T307906 |
[production] |
| 05:26 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P28468 and previous config saved to /var/cache/conftool/dbconfig/20220525-052622-ladsgroup.json |
[production] |
| 05:23 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 5%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28467 and previous config saved to /var/cache/conftool/dbconfig/20220525-052336-root.json |
[production] |
| 05:18 |
<marostegui> |
Rename revision_actor_temp on s5 T307906 |
[production] |
| 05:14 |
<marostegui> |
Rename revision_actor_temp on s7 T307906 |
[production] |
| 05:11 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P28466 and previous config saved to /var/cache/conftool/dbconfig/20220525-051117-ladsgroup.json |
[production] |
| 05:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 1%: After migrating to 10.4.25', diff saved to https://phabricator.wikimedia.org/P28465 and previous config saved to /var/cache/conftool/dbconfig/20220525-050833-root.json |
[production] |
| 05:06 |
<marostegui> |
Install 10.4.25 on db1143 T308915 |
[production] |
| 05:05 |
<marostegui@cumin2002> |
dbctl commit (dc=all): 'Depool db1143', diff saved to https://phabricator.wikimedia.org/P28464 and previous config saved to /var/cache/conftool/dbconfig/20220525-050538-marostegui.json |
[production] |
| 05:03 |
<marostegui> |
Rename revision_actor_temp on s2 T307906 |
[production] |
| 04:57 |
<marostegui> |
Rename revision_actor_temp on s4 T307906 |
[production] |
| 04:56 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T298560)', diff saved to https://phabricator.wikimedia.org/P28463 and previous config saved to /var/cache/conftool/dbconfig/20220525-045612-ladsgroup.json |
[production] |
| 02:02 |
<ebernhardson> |
restart elasticsearch_6@production-search-psi-eqiad to resolve CirrusSearchJVMGCOldPoolFlatlined alert |
[production] |
| 00:15 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1096:3315 (T298560)', diff saved to https://phabricator.wikimedia.org/P28462 and previous config saved to /var/cache/conftool/dbconfig/20220525-001552-ladsgroup.json |
[production] |