|
2024-01-23
§
|
| 23:04 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1218', diff saved to https://phabricator.wikimedia.org/P55426 and previous config saved to /var/cache/conftool/dbconfig/20240123-230453-marostegui.json |
[production] |
| 22:49 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1218', diff saved to https://phabricator.wikimedia.org/P55425 and previous config saved to /var/cache/conftool/dbconfig/20240123-224946-marostegui.json |
[production] |
| 22:34 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1218 (T354336)', diff saved to https://phabricator.wikimedia.org/P55424 and previous config saved to /var/cache/conftool/dbconfig/20240123-223439-marostegui.json |
[production] |
| 22:32 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1218 (T354336)', diff saved to https://phabricator.wikimedia.org/P55423 and previous config saved to /var/cache/conftool/dbconfig/20240123-223215-marostegui.json |
[production] |
| 22:32 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1218.eqiad.wmnet with reason: Maintenance |
[production] |
| 22:32 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1218.eqiad.wmnet with reason: Maintenance |
[production] |
| 22:31 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207 (T354336)', diff saved to https://phabricator.wikimedia.org/P55422 and previous config saved to /var/cache/conftool/dbconfig/20240123-223153-marostegui.json |
[production] |
| 22:16 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P55421 and previous config saved to /var/cache/conftool/dbconfig/20240123-221646-marostegui.json |
[production] |
| 22:03 |
<kostajh> |
UTC late deploys done |
[production] |
| 22:02 |
<kostajh> |
T355695 running mwscript resetAuthenticationThrottle.php --wiki=enwikibooks --signup --ip 195.70.81.86 |
[production] |
| 22:02 |
<kostajh> |
T355695 running mwscript resetAuthenticationThrottle.php --wiki=enwikibooks --signup --ip 62.232.9.14 |
[production] |
| 22:01 |
<kostajh> |
T355695 running mwscript resetAuthenticationThrottle.php --wiki=enwiki --signup --ip 195.70.81.86 |
[production] |
| 22:01 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P55420 and previous config saved to /var/cache/conftool/dbconfig/20240123-220140-marostegui.json |
[production] |
| 22:01 |
<kostajh> |
T355695 running mwscript resetAuthenticationThrottle.php --wiki=enwiki --signup --ip 62.232.9.14 |
[production] |
| 21:59 |
<kharlan@deploy2002> |
Finished scap: Backport for [[gerrit:992461|[knwiki] Removing the temporary logo (already reverted) (T338136)]], [[gerrit:992466|[itwiki] Add the 'abusefilter-bypass-blocked-external-domains' right to botadmins (T355694)]], [[gerrit:992471|[enwiki] and [enwikibooks] Throttle exemption for event (T355695)]] (duration: 15m 33s) |
[production] |
| 21:53 |
<kharlan@deploy2002> |
superpes and kharlan: Continuing with sync |
[production] |
| 21:46 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207 (T354336)', diff saved to https://phabricator.wikimedia.org/P55419 and previous config saved to /var/cache/conftool/dbconfig/20240123-214633-marostegui.json |
[production] |
| 21:45 |
<kharlan@deploy2002> |
superpes and kharlan: Backport for [[gerrit:992461|[knwiki] Removing the temporary logo (already reverted) (T338136)]], [[gerrit:992466|[itwiki] Add the 'abusefilter-bypass-blocked-external-domains' right to botadmins (T355694)]], [[gerrit:992471|[enwiki] and [enwikibooks] Throttle exemption for event (T355695)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:44 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1207 (T354336)', diff saved to https://phabricator.wikimedia.org/P55418 and previous config saved to /var/cache/conftool/dbconfig/20240123-214413-marostegui.json |
[production] |
| 21:44 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1207.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:43 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1207.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:43 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1206 (T354336)', diff saved to https://phabricator.wikimedia.org/P55417 and previous config saved to /var/cache/conftool/dbconfig/20240123-214351-marostegui.json |
[production] |
| 21:43 |
<kharlan@deploy2002> |
Started scap: Backport for [[gerrit:992461|[knwiki] Removing the temporary logo (already reverted) (T338136)]], [[gerrit:992466|[itwiki] Add the 'abusefilter-bypass-blocked-external-domains' right to botadmins (T355694)]], [[gerrit:992471|[enwiki] and [enwikibooks] Throttle exemption for event (T355695)]] |
[production] |
| 21:36 |
<kharlan@deploy2002> |
Finished scap: Backport for [[gerrit:992506|revertrisk: Fix i18n message reference (T348298)]], [[gerrit:992507|revertrisk: Fix i18n messages (T348298)]] (duration: 30m 51s) |
[production] |
| 21:28 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1206', diff saved to https://phabricator.wikimedia.org/P55416 and previous config saved to /var/cache/conftool/dbconfig/20240123-212845-marostegui.json |
[production] |
| 21:26 |
<kharlan@deploy2002> |
kharlan: Continuing with sync |
[production] |
| 21:25 |
<kharlan@deploy2002> |
kharlan: Backport for [[gerrit:992506|revertrisk: Fix i18n message reference (T348298)]], [[gerrit:992507|revertrisk: Fix i18n messages (T348298)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:13 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1206', diff saved to https://phabricator.wikimedia.org/P55415 and previous config saved to /var/cache/conftool/dbconfig/20240123-211338-marostegui.json |
[production] |
| 21:05 |
<kharlan@deploy2002> |
Started scap: Backport for [[gerrit:992506|revertrisk: Fix i18n message reference (T348298)]], [[gerrit:992507|revertrisk: Fix i18n messages (T348298)]] |
[production] |
| 20:58 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1206 (T354336)', diff saved to https://phabricator.wikimedia.org/P55414 and previous config saved to /var/cache/conftool/dbconfig/20240123-205832-marostegui.json |
[production] |
| 20:56 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1206 (T354336)', diff saved to https://phabricator.wikimedia.org/P55413 and previous config saved to /var/cache/conftool/dbconfig/20240123-205611-marostegui.json |
[production] |
| 20:56 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1206.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:55 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1206.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:55 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1196 (T354336)', diff saved to https://phabricator.wikimedia.org/P55412 and previous config saved to /var/cache/conftool/dbconfig/20240123-205549-marostegui.json |
[production] |
| 20:40 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P55411 and previous config saved to /var/cache/conftool/dbconfig/20240123-204043-marostegui.json |
[production] |
| 20:25 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P55410 and previous config saved to /var/cache/conftool/dbconfig/20240123-202536-marostegui.json |
[production] |
| 20:23 |
<cstone> |
payments-wiki upgraded from c2138768 to a3691a8e |
[production] |
| 20:23 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T347624, test data xfer) xfer categories from wdqs2024.codfw.wmnet -> wdqs2025.codfw.wmnet w/ force delete existing files, repooling both afterwards |
[production] |
| 20:12 |
<bking@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T347624, test data xfer) xfer categories from wdqs2024.codfw.wmnet -> wdqs2025.codfw.wmnet w/ force delete existing files, repooling both afterwards |
[production] |
| 20:10 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1196 (T354336)', diff saved to https://phabricator.wikimedia.org/P55409 and previous config saved to /var/cache/conftool/dbconfig/20240123-201030-marostegui.json |
[production] |
| 20:08 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T347624, test data xfer) xfer categories from wdqs2024.codfw.wmnet -> wdqs2025.codfw.wmnet w/ force delete existing files, repooling both afterwards |
[production] |
| 20:08 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1196 (T354336)', diff saved to https://phabricator.wikimedia.org/P55408 and previous config saved to /var/cache/conftool/dbconfig/20240123-200809-marostegui.json |
[production] |
| 20:08 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 16:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:07 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 16:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:07 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1196.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:07 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1196.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:07 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1186 (T354336)', diff saved to https://phabricator.wikimedia.org/P55407 and previous config saved to /var/cache/conftool/dbconfig/20240123-200726-marostegui.json |
[production] |
| 19:57 |
<bking@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T347624, test data xfer) xfer categories from wdqs2024.codfw.wmnet -> wdqs2025.codfw.wmnet w/ force delete existing files, repooling both afterwards |
[production] |
| 19:57 |
<bking@cumin2002> |
END (ERROR) - Cookbook sre.wdqs.data-transfer (exit_code=97) (T347624, test data xfer) xfer categories from wdqs2024.codfw.wmnet -> wdqs2025.codfw.wmnet, repooling both afterwards |
[production] |
| 19:52 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1186', diff saved to https://phabricator.wikimedia.org/P55406 and previous config saved to /var/cache/conftool/dbconfig/20240123-195220-marostegui.json |
[production] |