|
2026-05-27
§
|
| 01:02 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Depooling db2172 (T426633)', diff saved to https://phabricator.wikimedia.org/P93121 and previous config saved to /var/cache/conftool/dbconfig/20260527-010234-fceratto.json |
[production] |
| 01:02 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2172.codfw.wmnet with reason: Maintenance |
[production] |
| 01:02 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2155 (T426633)', diff saved to https://phabricator.wikimedia.org/P93120 and previous config saved to /var/cache/conftool/dbconfig/20260527-010205-fceratto.json |
[production] |
| 00:51 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P93119 and previous config saved to /var/cache/conftool/dbconfig/20260527-005157-fceratto.json |
[production] |
| 00:41 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P93118 and previous config saved to /var/cache/conftool/dbconfig/20260527-004149-fceratto.json |
[production] |
| 00:31 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2155 (T426633)', diff saved to https://phabricator.wikimedia.org/P93117 and previous config saved to /var/cache/conftool/dbconfig/20260527-003141-fceratto.json |
[production] |
| 00:23 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Depooling db2155 (T426633)', diff saved to https://phabricator.wikimedia.org/P93116 and previous config saved to /var/cache/conftool/dbconfig/20260527-002309-fceratto.json |
[production] |
| 00:23 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2155.codfw.wmnet with reason: Maintenance |
[production] |
| 00:22 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2166 (T426633)', diff saved to https://phabricator.wikimedia.org/P93115 and previous config saved to /var/cache/conftool/dbconfig/20260527-002228-fceratto.json |
[production] |
| 00:12 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2166', diff saved to https://phabricator.wikimedia.org/P93114 and previous config saved to /var/cache/conftool/dbconfig/20260527-001220-fceratto.json |
[production] |
| 00:02 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2166', diff saved to https://phabricator.wikimedia.org/P93113 and previous config saved to /var/cache/conftool/dbconfig/20260527-000209-fceratto.json |
[production] |
|
2026-05-26
§
|
| 23:52 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2166 (T426633)', diff saved to https://phabricator.wikimedia.org/P93112 and previous config saved to /var/cache/conftool/dbconfig/20260526-235201-fceratto.json |
[production] |
| 23:44 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Depooling db2166 (T426633)', diff saved to https://phabricator.wikimedia.org/P93111 and previous config saved to /var/cache/conftool/dbconfig/20260526-234451-fceratto.json |
[production] |
| 23:44 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2166.codfw.wmnet with reason: Maintenance |
[production] |
| 23:44 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2165 (T426633)', diff saved to https://phabricator.wikimedia.org/P93110 and previous config saved to /var/cache/conftool/dbconfig/20260526-234421-fceratto.json |
[production] |
| 23:34 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2165', diff saved to https://phabricator.wikimedia.org/P93109 and previous config saved to /var/cache/conftool/dbconfig/20260526-233414-fceratto.json |
[production] |
| 23:27 |
<brett@puppetserver1001> |
conftool action : set/pooled=yes; selector: name=cp5026.* |
[production] |
| 23:24 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2165', diff saved to https://phabricator.wikimedia.org/P93108 and previous config saved to /var/cache/conftool/dbconfig/20260526-232406-fceratto.json |
[production] |
| 23:13 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2165 (T426633)', diff saved to https://phabricator.wikimedia.org/P93107 and previous config saved to /var/cache/conftool/dbconfig/20260526-231358-fceratto.json |
[production] |
| 23:07 |
<brett@puppetserver1001> |
conftool action : set/pooled=no; selector: name=cp5026.* |
[production] |
| 23:06 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Depooling db2165 (T426633)', diff saved to https://phabricator.wikimedia.org/P93106 and previous config saved to /var/cache/conftool/dbconfig/20260526-230650-fceratto.json |
[production] |
| 23:06 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2165.codfw.wmnet with reason: Maintenance |
[production] |
| 23:06 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2164 (T426633)', diff saved to https://phabricator.wikimedia.org/P93105 and previous config saved to /var/cache/conftool/dbconfig/20260526-230620-fceratto.json |
[production] |
| 22:56 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2164', diff saved to https://phabricator.wikimedia.org/P93104 and previous config saved to /var/cache/conftool/dbconfig/20260526-225612-fceratto.json |
[production] |
| 22:46 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2164', diff saved to https://phabricator.wikimedia.org/P93103 and previous config saved to /var/cache/conftool/dbconfig/20260526-224604-fceratto.json |
[production] |
| 22:35 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2164 (T426633)', diff saved to https://phabricator.wikimedia.org/P93101 and previous config saved to /var/cache/conftool/dbconfig/20260526-223556-fceratto.json |
[production] |
| 22:28 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Depooling db2164 (T426633)', diff saved to https://phabricator.wikimedia.org/P93100 and previous config saved to /var/cache/conftool/dbconfig/20260526-222848-fceratto.json |
[production] |
| 22:28 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2164.codfw.wmnet with reason: Maintenance |
[production] |
| 22:28 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2163 (T426633)', diff saved to https://phabricator.wikimedia.org/P93099 and previous config saved to /var/cache/conftool/dbconfig/20260526-222828-fceratto.json |
[production] |
| 22:23 |
<robh@cumin2002> |
END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts cp6015.drmrs.wmnet |
[production] |
| 22:18 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2163', diff saved to https://phabricator.wikimedia.org/P93098 and previous config saved to /var/cache/conftool/dbconfig/20260526-221819-fceratto.json |
[production] |
| 22:10 |
<bking@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host relforge1009.eqiad.wmnet with OS trixie |
[production] |
| 22:08 |
<bking@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host relforge1008.eqiad.wmnet with OS trixie |
[production] |
| 22:08 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2163', diff saved to https://phabricator.wikimedia.org/P93097 and previous config saved to /var/cache/conftool/dbconfig/20260526-220811-fceratto.json |
[production] |
| 22:04 |
<egardner@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1293701|MultimediaViewer: enable image carousel as a beta feature on testwiki (T426799)]] (duration: 09m 30s) |
[production] |
| 22:03 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on relforge1009.eqiad.wmnet with reason: host reimage |
[production] |
| 22:00 |
<egardner@deploy1003> |
egardner, mfossati: Continuing with deployment |
[production] |
| 21:59 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on relforge1008.eqiad.wmnet with reason: host reimage |
[production] |
| 21:58 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2163 (T426633)', diff saved to https://phabricator.wikimedia.org/P93096 and previous config saved to /var/cache/conftool/dbconfig/20260526-215803-fceratto.json |
[production] |
| 21:57 |
<egardner@deploy1003> |
egardner, mfossati: Backport for [[gerrit:1293701|MultimediaViewer: enable image carousel as a beta feature on testwiki (T426799)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 21:56 |
<robh@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp6015.drmrs.wmnet |
[production] |
| 21:56 |
<bking@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host relforge1010.eqiad.wmnet with OS trixie |
[production] |
| 21:56 |
<robh@cumin2002> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts cp6015.drmrs.wmnet |
[production] |
| 21:55 |
<egardner@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1293701|MultimediaViewer: enable image carousel as a beta feature on testwiki (T426799)]] |
[production] |
| 21:54 |
<bking@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on relforge1009.eqiad.wmnet with reason: host reimage |
[production] |
| 21:51 |
<bking@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on relforge1008.eqiad.wmnet with reason: host reimage |
[production] |
| 21:50 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Depooling db2163 (T426633)', diff saved to https://phabricator.wikimedia.org/P93095 and previous config saved to /var/cache/conftool/dbconfig/20260526-215043-fceratto.json |
[production] |
| 21:50 |
<fceratto@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2163.codfw.wmnet with reason: Maintenance |
[production] |
| 21:50 |
<fceratto@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2222 (T426633)', diff saved to https://phabricator.wikimedia.org/P93094 and previous config saved to /var/cache/conftool/dbconfig/20260526-215011-fceratto.json |
[production] |
| 21:49 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on relforge1010.eqiad.wmnet with reason: host reimage |
[production] |