|
2024-04-04
ยง
|
| 15:36 |
<sukhe@cumin2002> |
START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on db2214.codfw.wmnet with reason: depooled, see T361851 |
[production] |
| 15:36 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1203.eqiad.wmnet with reason: Maintenance |
[production] |
| 15:36 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1193 (T355609)', diff saved to https://phabricator.wikimedia.org/P59517 and previous config saved to /var/cache/conftool/dbconfig/20240404-153603-marostegui.json |
[production] |
| 15:32 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1174', diff saved to https://phabricator.wikimedia.org/P59516 and previous config saved to /var/cache/conftool/dbconfig/20240404-153200-arnaudb.json |
[production] |
| 15:30 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2213 (re)pooling @ 16%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59515 and previous config saved to /var/cache/conftool/dbconfig/20240404-153043-arnaudb.json |
[production] |
| 15:30 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2207 (re)pooling @ 16%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59514 and previous config saved to /var/cache/conftool/dbconfig/20240404-153030-arnaudb.json |
[production] |
| 15:30 |
<sukhe@cumin2002> |
dbctl commit (dc=all): 'depool db2214', diff saved to https://phabricator.wikimedia.org/P59513 and previous config saved to /var/cache/conftool/dbconfig/20240404-153023-sukhe.json |
[production] |
| 15:23 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1206', diff saved to https://phabricator.wikimedia.org/P59512 and previous config saved to /var/cache/conftool/dbconfig/20240404-152341-marostegui.json |
[production] |
| 15:23 |
<jgiannelos@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 15:22 |
<jgiannelos@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mobileapps: apply |
[production] |
| 15:22 |
<logmsgbot> |
dreamyjazz Deployed security patch for T361296 |
[production] |
| 15:20 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P59511 and previous config saved to /var/cache/conftool/dbconfig/20240404-152056-marostegui.json |
[production] |
| 15:18 |
<jgiannelos@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 15:16 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1174', diff saved to https://phabricator.wikimedia.org/P59510 and previous config saved to /var/cache/conftool/dbconfig/20240404-151653-arnaudb.json |
[production] |
| 15:16 |
<jgiannelos@deploy1002> |
helmfile [codfw] START helmfile.d/services/mobileapps: apply |
[production] |
| 15:16 |
<jgiannelos@deploy1002> |
helmfile [staging] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 15:16 |
<jgiannelos@deploy1002> |
helmfile [staging] START helmfile.d/services/mobileapps: apply |
[production] |
| 15:15 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2113 (re)pooling @ 100%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59509 and previous config saved to /var/cache/conftool/dbconfig/20240404-151547-arnaudb.json |
[production] |
| 15:15 |
<jgiannelos@deploy1002> |
helmfile [codfw] START helmfile.d/services/mobileapps: apply |
[production] |
| 15:15 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2213 (re)pooling @ 8%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59508 and previous config saved to /var/cache/conftool/dbconfig/20240404-151537-arnaudb.json |
[production] |
| 15:15 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2207 (re)pooling @ 8%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59507 and previous config saved to /var/cache/conftool/dbconfig/20240404-151524-arnaudb.json |
[production] |
| 15:11 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2107 (re)pooling @ 100%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59506 and previous config saved to /var/cache/conftool/dbconfig/20240404-151123-arnaudb.json |
[production] |
| 15:10 |
<herron> |
beginning rolling hardware upgrades on titan200[12] T361229 |
[production] |
| 15:08 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1206 (T356166)', diff saved to https://phabricator.wikimedia.org/P59505 and previous config saved to /var/cache/conftool/dbconfig/20240404-150833-marostegui.json |
[production] |
| 15:08 |
<jgiannelos@deploy1002> |
helmfile [staging] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 15:08 |
<jgiannelos@deploy1002> |
helmfile [staging] START helmfile.d/services/mobileapps: apply |
[production] |
| 15:05 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P59504 and previous config saved to /var/cache/conftool/dbconfig/20240404-150549-marostegui.json |
[production] |
| 15:03 |
<logmsgbot> |
dreamyjazz Deployed security patch for T361295 |
[production] |
| 15:01 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1174 (T360332)', diff saved to https://phabricator.wikimedia.org/P59503 and previous config saved to /var/cache/conftool/dbconfig/20240404-150145-arnaudb.json |
[production] |
| 15:00 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2113 (re)pooling @ 70%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59502 and previous config saved to /var/cache/conftool/dbconfig/20240404-150041-arnaudb.json |
[production] |
| 15:00 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2213 (re)pooling @ 4%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59501 and previous config saved to /var/cache/conftool/dbconfig/20240404-150032-arnaudb.json |
[production] |
| 15:00 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2207 (re)pooling @ 4%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59500 and previous config saved to /var/cache/conftool/dbconfig/20240404-150017-arnaudb.json |
[production] |
| 14:57 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db1174 (T360332)', diff saved to https://phabricator.wikimedia.org/P59499 and previous config saved to /var/cache/conftool/dbconfig/20240404-145714-arnaudb.json |
[production] |
| 14:57 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1174.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:56 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1174.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:56 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1171.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:56 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2107 (re)pooling @ 70%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59498 and previous config saved to /var/cache/conftool/dbconfig/20240404-145617-arnaudb.json |
[production] |
| 14:56 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1171.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:56 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1170 (T360332)', diff saved to https://phabricator.wikimedia.org/P59497 and previous config saved to /var/cache/conftool/dbconfig/20240404-145613-arnaudb.json |
[production] |
| 14:50 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1193 (T355609)', diff saved to https://phabricator.wikimedia.org/P59496 and previous config saved to /var/cache/conftool/dbconfig/20240404-145041-marostegui.json |
[production] |
| 14:48 |
<dreamyjazz@deploy1002> |
Finished scap: Backport for [[gerrit:1017062|Remove sk translation of centralauth-rightslog-name (T361695)]] (duration: 14m 49s) |
[production] |
| 14:45 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2113 (re)pooling @ 50%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59495 and previous config saved to /var/cache/conftool/dbconfig/20240404-144536-arnaudb.json |
[production] |
| 14:45 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2213 (re)pooling @ 2%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59494 and previous config saved to /var/cache/conftool/dbconfig/20240404-144526-arnaudb.json |
[production] |
| 14:45 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2207 (re)pooling @ 2%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59493 and previous config saved to /var/cache/conftool/dbconfig/20240404-144511-arnaudb.json |
[production] |
| 14:41 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2107 (re)pooling @ 50%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59492 and previous config saved to /var/cache/conftool/dbconfig/20240404-144111-arnaudb.json |
[production] |
| 14:41 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1170', diff saved to https://phabricator.wikimedia.org/P59491 and previous config saved to /var/cache/conftool/dbconfig/20240404-144105-arnaudb.json |
[production] |
| 14:36 |
<dreamyjazz@deploy1002> |
dreamyjazz: Continuing with sync |
[production] |
| 14:36 |
<dreamyjazz@deploy1002> |
dreamyjazz: Backport for [[gerrit:1017062|Remove sk translation of centralauth-rightslog-name (T361695)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 14:33 |
<dreamyjazz@deploy1002> |
Started scap: Backport for [[gerrit:1017062|Remove sk translation of centralauth-rightslog-name (T361695)]] |
[production] |
| 14:30 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2113 (re)pooling @ 30%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59490 and previous config saved to /var/cache/conftool/dbconfig/20240404-143030-arnaudb.json |
[production] |