|
2026-03-03
ยง
|
| 06:25 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2147 (T418465)', diff saved to https://phabricator.wikimedia.org/P89642 and previous config saved to /var/cache/conftool/dbconfig/20260303-062507-marostegui.json |
[production] |
| 05:58 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db2147 (T418465)', diff saved to https://phabricator.wikimedia.org/P89639 and previous config saved to /var/cache/conftool/dbconfig/20260303-055834-marostegui.json |
[production] |
| 05:58 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2147.codfw.wmnet with reason: Maintenance |
[production] |
| 05:57 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1150.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:48 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2212.codfw.wmnet with reason: Maintenance |
[production] |
| 05:48 |
<marostegui@cumin1003> |
START - Cookbook sre.mysql.pool db2240 gradually with 4 steps - repool after schema change |
[production] |
| 05:48 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1163.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:01 |
<mwpresync@deploy2002> |
Pruned MediaWiki: 1.46.0-wmf.15 (duration: 01m 10s) |
[production] |
| 04:43 |
<mwpresync@deploy2002> |
Finished scap sync-world: testwikis to 1.46.0-wmf.18 refs T413809 (duration: 39m 43s) |
[production] |
| 04:03 |
<mwpresync@deploy2002> |
Started scap sync-world: testwikis to 1.46.0-wmf.18 refs T413809 |
[production] |
| 03:58 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance |
[production] |
| 03:57 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1251 (T418465)', diff saved to https://phabricator.wikimedia.org/P89637 and previous config saved to /var/cache/conftool/dbconfig/20260303-035746-marostegui.json |
[production] |
| 03:42 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1251', diff saved to https://phabricator.wikimedia.org/P89636 and previous config saved to /var/cache/conftool/dbconfig/20260303-034239-marostegui.json |
[production] |
| 03:27 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1251', diff saved to https://phabricator.wikimedia.org/P89635 and previous config saved to /var/cache/conftool/dbconfig/20260303-032731-marostegui.json |
[production] |
| 03:12 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1251 (T418465)', diff saved to https://phabricator.wikimedia.org/P89634 and previous config saved to /var/cache/conftool/dbconfig/20260303-031224-marostegui.json |
[production] |
| 03:02 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db1251 (T418465)', diff saved to https://phabricator.wikimedia.org/P89633 and previous config saved to /var/cache/conftool/dbconfig/20260303-030217-marostegui.json |
[production] |
| 03:02 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1251.eqiad.wmnet with reason: Maintenance |
[production] |
| 02:16 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1240.eqiad.wmnet with reason: Maintenance |
[production] |
| 02:08 |
<mwpresync@deploy2002> |
Finished scap build-images: Publishing wmf/next image (duration: 08m 00s) |
[production] |
| 02:08 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1239.eqiad.wmnet with reason: Maintenance |
[production] |
| 02:08 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1235 (T418465)', diff saved to https://phabricator.wikimedia.org/P89632 and previous config saved to /var/cache/conftool/dbconfig/20260303-020817-marostegui.json |
[production] |
| 02:00 |
<mwpresync@deploy2002> |
Started scap build-images: Publishing wmf/next image |
[production] |
| 01:53 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1235', diff saved to https://phabricator.wikimedia.org/P89631 and previous config saved to /var/cache/conftool/dbconfig/20260303-015309-marostegui.json |
[production] |
| 01:42 |
<herron@cumin1003> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mwlog2003.codfw.wmnet with OS trixie |
[production] |
| 01:38 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1235', diff saved to https://phabricator.wikimedia.org/P89630 and previous config saved to /var/cache/conftool/dbconfig/20260303-013802-marostegui.json |
[production] |
| 01:37 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2216 (T418465)', diff saved to https://phabricator.wikimedia.org/P89629 and previous config saved to /var/cache/conftool/dbconfig/20260303-013719-marostegui.json |
[production] |
| 01:22 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1235 (T418465)', diff saved to https://phabricator.wikimedia.org/P89628 and previous config saved to /var/cache/conftool/dbconfig/20260303-012254-marostegui.json |
[production] |
| 01:22 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2216', diff saved to https://phabricator.wikimedia.org/P89627 and previous config saved to /var/cache/conftool/dbconfig/20260303-012211-marostegui.json |
[production] |
| 01:19 |
<herron@cumin1003> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mwlog2003.codfw.wmnet with reason: host reimage |
[production] |
| 01:11 |
<herron@cumin1003> |
START - Cookbook sre.hosts.downtime for 2:00:00 on mwlog2003.codfw.wmnet with reason: host reimage |
[production] |
| 01:11 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db1235 (T418465)', diff saved to https://phabricator.wikimedia.org/P89626 and previous config saved to /var/cache/conftool/dbconfig/20260303-011151-marostegui.json |
[production] |
| 01:11 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1235.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:11 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1234 (T418465)', diff saved to https://phabricator.wikimedia.org/P89625 and previous config saved to /var/cache/conftool/dbconfig/20260303-011128-marostegui.json |
[production] |
| 01:07 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2216', diff saved to https://phabricator.wikimedia.org/P89624 and previous config saved to /var/cache/conftool/dbconfig/20260303-010703-marostegui.json |
[production] |
| 00:59 |
<zabe@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1247189|Revert "ImageListPager: Properly support file schema migration read new"]] (duration: 08m 12s) |
[production] |
| 00:56 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1234', diff saved to https://phabricator.wikimedia.org/P89623 and previous config saved to /var/cache/conftool/dbconfig/20260303-005620-marostegui.json |
[production] |
| 00:56 |
<zabe@deploy2002> |
zabe: Continuing with sync |
[production] |
| 00:54 |
<herron@cumin1003> |
START - Cookbook sre.hosts.reimage for host mwlog2003.codfw.wmnet with OS trixie |
[production] |
| 00:53 |
<zabe@deploy2002> |
zabe: Backport for [[gerrit:1247189|Revert "ImageListPager: Properly support file schema migration read new"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 00:53 |
<herron@cumin1003> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host mwlog2003.codfw.wmnet with OS trixie |
[production] |
| 00:51 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2216 (T418465)', diff saved to https://phabricator.wikimedia.org/P89622 and previous config saved to /var/cache/conftool/dbconfig/20260303-005156-marostegui.json |
[production] |
| 00:51 |
<zabe@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1247189|Revert "ImageListPager: Properly support file schema migration read new"]] |
[production] |
| 00:41 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1234', diff saved to https://phabricator.wikimedia.org/P89621 and previous config saved to /var/cache/conftool/dbconfig/20260303-004112-marostegui.json |
[production] |
| 00:40 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db2216 (T418465)', diff saved to https://phabricator.wikimedia.org/P89620 and previous config saved to /var/cache/conftool/dbconfig/20260303-004056-marostegui.json |
[production] |
| 00:40 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2216.codfw.wmnet with reason: Maintenance |
[production] |
| 00:40 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2203 (T418465)', diff saved to https://phabricator.wikimedia.org/P89619 and previous config saved to /var/cache/conftool/dbconfig/20260303-004033-marostegui.json |
[production] |
| 00:31 |
<herron@cumin1003> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mwlog1003.eqiad.wmnet with OS trixie |
[production] |
| 00:26 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1234 (T418465)', diff saved to https://phabricator.wikimedia.org/P89618 and previous config saved to /var/cache/conftool/dbconfig/20260303-002604-marostegui.json |
[production] |
| 00:25 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2203', diff saved to https://phabricator.wikimedia.org/P89617 and previous config saved to /var/cache/conftool/dbconfig/20260303-002525-marostegui.json |
[production] |
| 00:20 |
<zabe@deploy2002> |
helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply |
[production] |