|
2021-12-13
ยง
|
| 10:51 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on 7 hosts with reason: Maintenance |
[production] |
| 10:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1172', diff saved to https://phabricator.wikimedia.org/P18151 and previous config saved to /var/cache/conftool/dbconfig/20211213-101707-marostegui.json |
[production] |
| 10:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1172 (T277354)', diff saved to https://phabricator.wikimedia.org/P18150 and previous config saved to /var/cache/conftool/dbconfig/20211213-101427-marostegui.json |
[production] |
| 10:14 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1172.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:14 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1172.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:14 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1171.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:14 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1171.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1126', diff saved to https://phabricator.wikimedia.org/P18149 and previous config saved to /var/cache/conftool/dbconfig/20211213-101013-marostegui.json |
[production] |
| 10:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1126 (T277354)', diff saved to https://phabricator.wikimedia.org/P18148 and previous config saved to /var/cache/conftool/dbconfig/20211213-100238-marostegui.json |
[production] |
| 10:02 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1126.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:02 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1126.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:02 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1116.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:02 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1116.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1114', diff saved to https://phabricator.wikimedia.org/P18147 and previous config saved to /var/cache/conftool/dbconfig/20211213-100143-marostegui.json |
[production] |
| 09:59 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db2130 and db2074', diff saved to https://phabricator.wikimedia.org/P18146 and previous config saved to /var/cache/conftool/dbconfig/20211213-095949-marostegui.json |
[production] |
| 09:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1114 (T277354)', diff saved to https://phabricator.wikimedia.org/P18145 and previous config saved to /var/cache/conftool/dbconfig/20211213-095851-marostegui.json |
[production] |
| 09:58 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1114.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:58 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1114.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1111', diff saved to https://phabricator.wikimedia.org/P18144 and previous config saved to /var/cache/conftool/dbconfig/20211213-095728-marostegui.json |
[production] |
| 09:49 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1111 (T277354)', diff saved to https://phabricator.wikimedia.org/P18143 and previous config saved to /var/cache/conftool/dbconfig/20211213-094900-marostegui.json |
[production] |
| 09:49 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1111.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:48 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1111.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1104 (T277354)', diff saved to https://phabricator.wikimedia.org/P18142 and previous config saved to /var/cache/conftool/dbconfig/20211213-094853-marostegui.json |
[production] |
| 09:45 |
<urbanecm> |
Staging at mwdebug1001 ended |
[production] |
| 09:43 |
<urbanecm> |
pwnwiki: Create DB tables for GrowthExperiments |
[production] |
| 09:43 |
<dcausse@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . |
[production] |
| 09:42 |
<urbanecm> |
Stagging at mwdebug1001 |
[production] |
| 09:38 |
<Amir1> |
revoking DROP from centralauth grant of wikiadmin (T249683) |
[production] |
| 09:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1104', diff saved to https://phabricator.wikimedia.org/P18141 and previous config saved to /var/cache/conftool/dbconfig/20211213-093348-marostegui.json |
[production] |
| 09:32 |
<dcausse@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . |
[production] |
| 09:31 |
<elukey@deploy1002> |
helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 09:31 |
<elukey@deploy1002> |
helmfile [staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 09:31 |
<elukey@deploy1002> |
helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 09:31 |
<elukey@deploy1002> |
helmfile [staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 09:29 |
<dcausse> |
restarting blazegraph on wdqs1012 (jvm stuck for 6h) |
[production] |
| 09:29 |
<elukey@deploy1002> |
helmfile [ml-serve-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 09:28 |
<elukey@deploy1002> |
helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 09:25 |
<elukey@deploy1002> |
helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. |
[production] |
| 09:25 |
<elukey@deploy1002> |
helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. |
[production] |
| 09:24 |
<dcausse@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . |
[production] |
| 09:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1104', diff saved to https://phabricator.wikimedia.org/P18140 and previous config saved to /var/cache/conftool/dbconfig/20211213-091844-marostegui.json |
[production] |
| 09:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1104 (T277354)', diff saved to https://phabricator.wikimedia.org/P18139 and previous config saved to /var/cache/conftool/dbconfig/20211213-090339-marostegui.json |
[production] |
| 08:48 |
<Amir1> |
removing grant of '%a%' on db1123 (s3) T296537 |
[production] |
| 08:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1104 (T277354)', diff saved to https://phabricator.wikimedia.org/P18138 and previous config saved to /var/cache/conftool/dbconfig/20211213-084729-marostegui.json |
[production] |
| 08:47 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1104.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:47 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1104.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1101:3318 (T277354)', diff saved to https://phabricator.wikimedia.org/P18137 and previous config saved to /var/cache/conftool/dbconfig/20211213-084721-marostegui.json |
[production] |
| 08:45 |
<Amir1> |
fixing centralauth grants of wikiuser on all of s7 T296537 |
[production] |
| 08:32 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1101:3318', diff saved to https://phabricator.wikimedia.org/P18136 and previous config saved to /var/cache/conftool/dbconfig/20211213-083217-marostegui.json |
[production] |
| 08:27 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |