|
2022-05-26
ยง
|
| 10:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Remove db1164 from dbctl', diff saved to https://phabricator.wikimedia.org/P28588 and previous config saved to /var/cache/conftool/dbconfig/20220526-101250-marostegui.json |
[production] |
| 10:05 |
<marostegui> |
Stop mysql on db1117:3321 to clone db1164 T309286 |
[production] |
| 10:00 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P28586 and previous config saved to /var/cache/conftool/dbconfig/20220526-100020-ladsgroup.json |
[production] |
| 10:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1164 T309286', diff saved to https://phabricator.wikimedia.org/P28585 and previous config saved to /var/cache/conftool/dbconfig/20220526-100013-marostegui.json |
[production] |
| 09:45 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157 (T298560)', diff saved to https://phabricator.wikimedia.org/P28583 and previous config saved to /var/cache/conftool/dbconfig/20220526-094509-ladsgroup.json |
[production] |
| 09:21 |
<volans> |
uploaded spicerack_2.5.0 to apt.wikimedia.org bullseye-wikimedia |
[production] |
| 09:19 |
<elukey@deploy1002> |
helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . |
[production] |
| 09:16 |
<elukey@deploy1002> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . |
[production] |
| 08:56 |
<elukey@deploy1002> |
helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . |
[production] |
| 08:55 |
<elukey@deploy1002> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . |
[production] |
| 08:40 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es1032 T309265', diff saved to https://phabricator.wikimedia.org/P28582 and previous config saved to /var/cache/conftool/dbconfig/20220526-084009-marostegui.json |
[production] |
| 08:30 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:29 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:29 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1099:3311 (T298555)', diff saved to https://phabricator.wikimedia.org/P28581 and previous config saved to /var/cache/conftool/dbconfig/20220526-082951-ladsgroup.json |
[production] |
| 08:14 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1099:3311', diff saved to https://phabricator.wikimedia.org/P28580 and previous config saved to /var/cache/conftool/dbconfig/20220526-081446-ladsgroup.json |
[production] |
| 07:59 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1099:3311', diff saved to https://phabricator.wikimedia.org/P28579 and previous config saved to /var/cache/conftool/dbconfig/20220526-075941-ladsgroup.json |
[production] |
| 07:55 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1157 (T298560)', diff saved to https://phabricator.wikimedia.org/P28578 and previous config saved to /var/cache/conftool/dbconfig/20220526-075525-ladsgroup.json |
[production] |
| 07:55 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1157.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:55 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1157.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:44 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1099:3311 (T298555)', diff saved to https://phabricator.wikimedia.org/P28577 and previous config saved to /var/cache/conftool/dbconfig/20220526-074436-ladsgroup.json |
[production] |
| 07:31 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 07:30 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 07:30 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 07:29 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 07:25 |
<ariel@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:793799|Add namespaces to Punjabi wikisource default search (T287887)]] (duration: 00m 50s) |
[production] |
| 07:24 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 07:23 |
<elukey@deploy1002> |
helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . |
[production] |
| 07:20 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 07:20 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 07:19 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 07:18 |
<elukey@deploy1002> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . |
[production] |
| 07:15 |
<ariel@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:796385|Enable Realtime Preview on more pilot wikis: huwiki and fiwiki (T303961)]] (duration: 00m 51s) |
[production] |
| 07:14 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 07:13 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 07:13 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 07:12 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 06:15 |
<kart_> |
Updated cxserver to 2022-05-26-052433-production (T309161, T308829, T308834) |
[production] |
| 06:13 |
<kartik@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/cxserver: apply |
[production] |
| 06:12 |
<kartik@deploy1002> |
helmfile [eqiad] START helmfile.d/services/cxserver: apply |
[production] |
| 06:11 |
<kartik@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/cxserver: apply |
[production] |
| 06:10 |
<kartik@deploy1002> |
helmfile [codfw] START helmfile.d/services/cxserver: apply |
[production] |
| 06:07 |
<kartik@deploy1002> |
helmfile [staging] DONE helmfile.d/services/cxserver: apply |
[production] |
| 06:06 |
<kartik@deploy1002> |
helmfile [staging] START helmfile.d/services/cxserver: apply |
[production] |
| 05:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1111', diff saved to https://phabricator.wikimedia.org/P28576 and previous config saved to /var/cache/conftool/dbconfig/20220526-053155-marostegui.json |
[production] |
| 05:16 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1099:3311 (T298555)', diff saved to https://phabricator.wikimedia.org/P28575 and previous config saved to /var/cache/conftool/dbconfig/20220526-051649-ladsgroup.json |
[production] |
| 05:16 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1099.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:16 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db1099.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:16 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1119 (T298555)', diff saved to https://phabricator.wikimedia.org/P28574 and previous config saved to /var/cache/conftool/dbconfig/20220526-051641-ladsgroup.json |
[production] |
| 05:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1119', diff saved to https://phabricator.wikimedia.org/P28573 and previous config saved to /var/cache/conftool/dbconfig/20220526-050136-ladsgroup.json |
[production] |
| 04:31 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1119 (T298555)', diff saved to https://phabricator.wikimedia.org/P28571 and previous config saved to /var/cache/conftool/dbconfig/20220526-043126-ladsgroup.json |
[production] |