|
2026-01-30
ยง
|
| 14:11 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2163', diff saved to https://phabricator.wikimedia.org/P88270 and previous config saved to /var/cache/conftool/dbconfig/20260130-141139-marostegui.json |
[production] |
| 14:11 |
<tappof> |
puppet enabled and services repooled on titan2001 T410152 |
[production] |
| 14:10 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:10 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1259 (T415786)', diff saved to https://phabricator.wikimedia.org/P88269 and previous config saved to /var/cache/conftool/dbconfig/20260130-141026-marostegui.json |
[production] |
| 13:56 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2163', diff saved to https://phabricator.wikimedia.org/P88268 and previous config saved to /var/cache/conftool/dbconfig/20260130-135627-marostegui.json |
[production] |
| 13:55 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1259', diff saved to Unable to send diff to phaste and previous config saved to /var/cache/conftool/dbconfig/20260130-135513-marostegui.json |
[production] |
| 13:41 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2163 (T415786)', diff saved to https://phabricator.wikimedia.org/P88266 and previous config saved to /var/cache/conftool/dbconfig/20260130-134117-marostegui.json |
[production] |
| 13:40 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1259', diff saved to https://phabricator.wikimedia.org/P88265 and previous config saved to /var/cache/conftool/dbconfig/20260130-134001-marostegui.json |
[production] |
| 13:31 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db2163 (T415786)', diff saved to https://phabricator.wikimedia.org/P88264 and previous config saved to /var/cache/conftool/dbconfig/20260130-133111-marostegui.json |
[production] |
| 13:31 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2163.codfw.wmnet with reason: Maintenance |
[production] |
| 13:30 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2154 (T415786)', diff saved to https://phabricator.wikimedia.org/P88263 and previous config saved to /var/cache/conftool/dbconfig/20260130-133045-marostegui.json |
[production] |
| 13:24 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1259 (T415786)', diff saved to https://phabricator.wikimedia.org/P88262 and previous config saved to /var/cache/conftool/dbconfig/20260130-132452-marostegui.json |
[production] |
| 13:15 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P88261 and previous config saved to /var/cache/conftool/dbconfig/20260130-131533-marostegui.json |
[production] |
| 13:00 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P88260 and previous config saved to /var/cache/conftool/dbconfig/20260130-130024-marostegui.json |
[production] |
| 12:45 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2154 (T415786)', diff saved to https://phabricator.wikimedia.org/P88259 and previous config saved to /var/cache/conftool/dbconfig/20260130-124515-marostegui.json |
[production] |
| 12:35 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db2154 (T415786)', diff saved to Unable to send diff to phaste and previous config saved to /var/cache/conftool/dbconfig/20260130-123510-marostegui.json |
[production] |
| 12:35 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2154.codfw.wmnet with reason: Maintenance |
[production] |
| 12:34 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2152 (T415786)', diff saved to https://phabricator.wikimedia.org/P88257 and previous config saved to /var/cache/conftool/dbconfig/20260130-123442-marostegui.json |
[production] |
| 12:19 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2152', diff saved to https://phabricator.wikimedia.org/P88256 and previous config saved to /var/cache/conftool/dbconfig/20260130-121934-marostegui.json |
[production] |
| 12:18 |
<marostegui@cumin1003> |
END (PASS) - Cookbook sre.mysql.newpool (exit_code=0) pool db1209: After schema change |
[production] |
| 12:04 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2152', diff saved to https://phabricator.wikimedia.org/P88253 and previous config saved to /var/cache/conftool/dbconfig/20260130-120423-marostegui.json |
[production] |
| 11:49 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2152 (T415786)', diff saved to https://phabricator.wikimedia.org/P88251 and previous config saved to /var/cache/conftool/dbconfig/20260130-114913-marostegui.json |
[production] |
| 11:47 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db1259 (T415786)', diff saved to https://phabricator.wikimedia.org/P88249 and previous config saved to /var/cache/conftool/dbconfig/20260130-114719-marostegui.json |
[production] |
| 11:47 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1259.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:46 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1254 (T415786)', diff saved to https://phabricator.wikimedia.org/P88248 and previous config saved to /var/cache/conftool/dbconfig/20260130-114654-marostegui.json |
[production] |
| 11:39 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db2152 (T415786)', diff saved to https://phabricator.wikimedia.org/P88246 and previous config saved to /var/cache/conftool/dbconfig/20260130-113900-marostegui.json |
[production] |
| 11:38 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2152.codfw.wmnet with reason: Maintenance |
[production] |
| 11:33 |
<marostegui@cumin1003> |
START - Cookbook sre.mysql.newpool pool db1209: After schema change |
[production] |
| 11:33 |
<marostegui@cumin1003> |
END (ERROR) - Cookbook sre.mysql.newpool (exit_code=97) pool db1209: After schema change |
[production] |
| 11:31 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1254', diff saved to https://phabricator.wikimedia.org/P88244 and previous config saved to /var/cache/conftool/dbconfig/20260130-113146-marostegui.json |
[production] |
| 11:30 |
<marostegui@cumin1003> |
START - Cookbook sre.mysql.newpool pool db1209: After schema change |
[production] |
| 11:16 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1254', diff saved to Unable to send diff to phaste and previous config saved to /var/cache/conftool/dbconfig/20260130-111633-marostegui.json |
[production] |
| 11:01 |
<fnegri@cumin1003> |
END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) for database pplwiki (T415050) |
[production] |
| 11:01 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1254 (T415786)', diff saved to https://phabricator.wikimedia.org/P88242 and previous config saved to /var/cache/conftool/dbconfig/20260130-110122-marostegui.json |
[production] |
| 10:36 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2238 (T415786)', diff saved to Unable to send diff to phaste and previous config saved to /var/cache/conftool/dbconfig/20260130-103610-marostegui.json |
[production] |
| 10:21 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2238', diff saved to https://phabricator.wikimedia.org/P88240 and previous config saved to /var/cache/conftool/dbconfig/20260130-102102-marostegui.json |
[production] |
| 10:05 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2238', diff saved to https://phabricator.wikimedia.org/P88238 and previous config saved to /var/cache/conftool/dbconfig/20260130-100552-marostegui.json |
[production] |
| 10:04 |
<fnegri@cumin1003> |
START - Cookbook sre.wikireplicas.add-wiki for database pplwiki (T415050) |
[production] |
| 09:50 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2238 (T415786)', diff saved to https://phabricator.wikimedia.org/P88237 and previous config saved to /var/cache/conftool/dbconfig/20260130-095042-marostegui.json |
[production] |
| 09:27 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db1254 (T415786)', diff saved to https://phabricator.wikimedia.org/P88236 and previous config saved to /var/cache/conftool/dbconfig/20260130-092729-marostegui.json |
[production] |
| 09:27 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1254.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:16 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db2238 (T415786)', diff saved to https://phabricator.wikimedia.org/P88235 and previous config saved to /var/cache/conftool/dbconfig/20260130-081629-marostegui.json |
[production] |
| 08:16 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2238.codfw.wmnet with reason: Maintenance |
[production] |
| 08:16 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2226 (T415786)', diff saved to https://phabricator.wikimedia.org/P88234 and previous config saved to /var/cache/conftool/dbconfig/20260130-081559-marostegui.json |
[production] |
| 08:00 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2226', diff saved to https://phabricator.wikimedia.org/P88233 and previous config saved to /var/cache/conftool/dbconfig/20260130-080051-marostegui.json |
[production] |
| 07:57 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1239.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:57 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1233 (T415786)', diff saved to https://phabricator.wikimedia.org/P88232 and previous config saved to /var/cache/conftool/dbconfig/20260130-075721-marostegui.json |
[production] |
| 07:45 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2226', diff saved to https://phabricator.wikimedia.org/P88231 and previous config saved to /var/cache/conftool/dbconfig/20260130-074540-marostegui.json |
[production] |
| 07:42 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1233', diff saved to https://phabricator.wikimedia.org/P88230 and previous config saved to /var/cache/conftool/dbconfig/20260130-074213-marostegui.json |
[production] |
| 07:30 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2226 (T415786)', diff saved to https://phabricator.wikimedia.org/P88229 and previous config saved to /var/cache/conftool/dbconfig/20260130-073032-marostegui.json |
[production] |