|
2022-06-27
ยง
|
| 07:39 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:39 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:39 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30293 and previous config saved to /var/cache/conftool/dbconfig/20220627-073929-ladsgroup.json |
[production] |
| 07:35 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 07:33 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 07:33 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 07:30 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 07:24 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30292 and previous config saved to /var/cache/conftool/dbconfig/20220627-072424-ladsgroup.json |
[production] |
| 07:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1157 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30291 and previous config saved to /var/cache/conftool/dbconfig/20220627-071539-root.json |
[production] |
| 07:15 |
<mlitn@deploy1002> |
Synchronized php-1.39.0-wmf.17/extensions/ImageSuggestions/maintenance/SendNotificationsForUnillustratedWatchedTitles.php: Backport: [[gerrit:808120|Echo tables can live in a different db]] (duration: 03m 45s) |
[production] |
| 07:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1144 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30290 and previous config saved to /var/cache/conftool/dbconfig/20220627-071506-root.json |
[production] |
| 07:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1109 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30289 and previous config saved to /var/cache/conftool/dbconfig/20220627-071434-root.json |
[production] |
| 07:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1105 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30288 and previous config saved to /var/cache/conftool/dbconfig/20220627-071414-root.json |
[production] |
| 07:13 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1121 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30287 and previous config saved to /var/cache/conftool/dbconfig/20220627-071304-root.json |
[production] |
| 07:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1112 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30286 and previous config saved to /var/cache/conftool/dbconfig/20220627-071255-root.json |
[production] |
| 07:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3318 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30285 and previous config saved to /var/cache/conftool/dbconfig/20220627-071226-root.json |
[production] |
| 07:09 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30284 and previous config saved to /var/cache/conftool/dbconfig/20220627-070919-ladsgroup.json |
[production] |
| 07:00 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 06:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1121 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30283 and previous config saved to /var/cache/conftool/dbconfig/20220627-065800-root.json |
[production] |
| 06:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1112 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30282 and previous config saved to /var/cache/conftool/dbconfig/20220627-065751-root.json |
[production] |
| 06:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30281 and previous config saved to /var/cache/conftool/dbconfig/20220627-065729-root.json |
[production] |
| 06:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3318 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30280 and previous config saved to /var/cache/conftool/dbconfig/20220627-065722-root.json |
[production] |
| 06:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30279 and previous config saved to /var/cache/conftool/dbconfig/20220627-065716-root.json |
[production] |
| 06:56 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 06:56 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 06:54 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30278 and previous config saved to /var/cache/conftool/dbconfig/20220627-065414-ladsgroup.json |
[production] |
| 06:53 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 06:52 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 8:00:00 on 14 hosts with reason: Maintenance |
[production] |
| 06:52 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 3 days, 8:00:00 on 14 hosts with reason: Maintenance |
[production] |
| 06:52 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 6:00:00 on db2103.codfw.wmnet with reason: Maintenance |
[production] |
| 06:52 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 6:00:00 on db2103.codfw.wmnet with reason: Maintenance |
[production] |
| 06:50 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on 14 hosts with reason: Maintenance |
[production] |
| 06:50 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on 14 hosts with reason: Maintenance |
[production] |
| 06:50 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2103.codfw.wmnet with reason: Maintenance |
[production] |
| 06:50 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2103.codfw.wmnet with reason: Maintenance |
[production] |
| 06:50 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30277 and previous config saved to /var/cache/conftool/dbconfig/20220627-065009-ladsgroup.json |
[production] |
| 06:50 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:50 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1121 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30276 and previous config saved to /var/cache/conftool/dbconfig/20220627-064256-root.json |
[production] |
| 06:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1112 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30275 and previous config saved to /var/cache/conftool/dbconfig/20220627-064247-root.json |
[production] |
| 06:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30274 and previous config saved to /var/cache/conftool/dbconfig/20220627-064225-root.json |
[production] |
| 06:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3318 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30273 and previous config saved to /var/cache/conftool/dbconfig/20220627-064219-root.json |
[production] |
| 06:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30272 and previous config saved to /var/cache/conftool/dbconfig/20220627-064212-root.json |
[production] |
| 06:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1121 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30271 and previous config saved to /var/cache/conftool/dbconfig/20220627-062752-root.json |
[production] |
| 06:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1112 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30270 and previous config saved to /var/cache/conftool/dbconfig/20220627-062743-root.json |
[production] |
| 06:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30269 and previous config saved to /var/cache/conftool/dbconfig/20220627-062721-root.json |
[production] |
| 06:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1099:3318 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30268 and previous config saved to /var/cache/conftool/dbconfig/20220627-062715-root.json |
[production] |
| 06:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30267 and previous config saved to /var/cache/conftool/dbconfig/20220627-062708-root.json |
[production] |
| 06:13 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 06:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1121 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30266 and previous config saved to /var/cache/conftool/dbconfig/20220627-061249-root.json |
[production] |