|
2022-07-14
ยง
|
| 15:20 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db1158.eqiad.wmnet with reason: Maintenance |
[production] |
| 15:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1101:3317 (T312977)', diff saved to https://phabricator.wikimedia.org/P31081 and previous config saved to /var/cache/conftool/dbconfig/20220714-152040-marostegui.json |
[production] |
| 15:15 |
<hnowlan@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/image-suggestion: sync |
[production] |
| 15:15 |
<hnowlan@deploy1002> |
helmfile [eqiad] START helmfile.d/services/image-suggestion: sync |
[production] |
| 15:14 |
<hnowlan@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/image-suggestion: sync |
[production] |
| 15:14 |
<hnowlan@deploy1002> |
helmfile [codfw] START helmfile.d/services/image-suggestion: sync |
[production] |
| 15:13 |
<hnowlan@deploy1002> |
helmfile [staging] DONE helmfile.d/services/image-suggestion: sync |
[production] |
| 15:13 |
<hnowlan@deploy1002> |
helmfile [staging] START helmfile.d/services/image-suggestion: sync |
[production] |
| 15:12 |
<ebysans@deploy1002> |
Finished deploy [airflow-dags/analytics@b8f66e9]: (no justification provided) (duration: 00m 10s) |
[production] |
| 15:11 |
<ebysans@deploy1002> |
Started deploy [airflow-dags/analytics@b8f66e9]: (no justification provided) |
[production] |
| 15:10 |
<ejegg> |
updated payments-wiki from 6a8aa302 to be11fac2 |
[production] |
| 15:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1101:3317', diff saved to https://phabricator.wikimedia.org/P31080 and previous config saved to /var/cache/conftool/dbconfig/20220714-150535-marostegui.json |
[production] |
| 14:57 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1113:3316 (T312984)', diff saved to https://phabricator.wikimedia.org/P31079 and previous config saved to /var/cache/conftool/dbconfig/20220714-145736-ladsgroup.json |
[production] |
| 14:57 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1113.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:57 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db1113.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:57 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1165 (T312984)', diff saved to https://phabricator.wikimedia.org/P31078 and previous config saved to /var/cache/conftool/dbconfig/20220714-145716-ladsgroup.json |
[production] |
| 14:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1101:3317', diff saved to https://phabricator.wikimedia.org/P31077 and previous config saved to /var/cache/conftool/dbconfig/20220714-145030-marostegui.json |
[production] |
| 14:42 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1165', diff saved to https://phabricator.wikimedia.org/P31076 and previous config saved to /var/cache/conftool/dbconfig/20220714-144211-ladsgroup.json |
[production] |
| 14:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1101:3317 (T312977)', diff saved to https://phabricator.wikimedia.org/P31075 and previous config saved to /var/cache/conftool/dbconfig/20220714-143525-marostegui.json |
[production] |
| 14:27 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1165', diff saved to https://phabricator.wikimedia.org/P31074 and previous config saved to /var/cache/conftool/dbconfig/20220714-142706-ladsgroup.json |
[production] |
| 14:19 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1101:3317 (T312977)', diff saved to https://phabricator.wikimedia.org/P31073 and previous config saved to /var/cache/conftool/dbconfig/20220714-141917-marostegui.json |
[production] |
| 14:19 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1101.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:19 |
<papaul> |
on going PDU maintenance in rack A6 codfw |
[production] |
| 14:19 |
<papaul> |
on going PU maintenance in rack A6 codfw |
[production] |
| 14:18 |
<papaul> |
on going PU maintenance in rack A6 codfw |
[production] |
| 14:18 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db1101.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1098:3317 (T312977)', diff saved to https://phabricator.wikimedia.org/P31072 and previous config saved to /var/cache/conftool/dbconfig/20220714-141846-marostegui.json |
[production] |
| 14:12 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1165 (T312984)', diff saved to https://phabricator.wikimedia.org/P31071 and previous config saved to /var/cache/conftool/dbconfig/20220714-141201-ladsgroup.json |
[production] |
| 14:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1098:3317', diff saved to https://phabricator.wikimedia.org/P31070 and previous config saved to /var/cache/conftool/dbconfig/20220714-140341-marostegui.json |
[production] |
| 14:02 |
<matthiasmullie> |
UTC afternoon backport window done |
[production] |
| 13:53 |
<mlitn@deploy1002> |
Finished scap: Backport: [[gerrit:813829|Improve maint script output & update i18n messages]] (duration: 16m 05s) |
[production] |
| 13:50 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1165 (T312984)', diff saved to https://phabricator.wikimedia.org/P31069 and previous config saved to /var/cache/conftool/dbconfig/20220714-135038-ladsgroup.json |
[production] |
| 13:50 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on clouddb[1015,1019,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:50 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 20:00:00 on clouddb[1015,1019,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:50 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1165.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:50 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db1165.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:50 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1131 (T312984)', diff saved to https://phabricator.wikimedia.org/P31068 and previous config saved to /var/cache/conftool/dbconfig/20220714-135000-ladsgroup.json |
[production] |
| 13:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1098:3317', diff saved to https://phabricator.wikimedia.org/P31067 and previous config saved to /var/cache/conftool/dbconfig/20220714-134836-marostegui.json |
[production] |
| 13:41 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:40 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:40 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:39 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:37 |
<mlitn@deploy1002> |
Started scap: Backport: [[gerrit:813829|Improve maint script output & update i18n messages]] |
[production] |
| 13:34 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1131', diff saved to https://phabricator.wikimedia.org/P31065 and previous config saved to /var/cache/conftool/dbconfig/20220714-133455-ladsgroup.json |
[production] |
| 13:34 |
<mlitn@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:813881|Update boosts for weighted_tags]] (duration: 02m 45s) |
[production] |
| 13:34 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1098:3317 (T312977)', diff saved to https://phabricator.wikimedia.org/P31064 and previous config saved to /var/cache/conftool/dbconfig/20220714-133331-marostegui.json |
[production] |
| 13:33 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:32 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:31 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |