|
2022-03-10
ยง
|
| 22:08 |
<bking@cumin1001> |
END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 |
[production] |
| 22:05 |
<bking@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 |
[production] |
| 22:04 |
<bking@cumin1001> |
END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 |
[production] |
| 22:04 |
<bking@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 |
[production] |
| 22:02 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 |
[production] |
| 22:02 |
<bking@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 |
[production] |
| 21:43 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:42 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:42 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:41 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:41 |
<rzl> |
UTC late B&C training window done |
[production] |
| 21:39 |
<rzl@deploy1002> |
Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:769779|CommonSettings: Update comment about Image Suggestions API (T294362)]] (duration: 00m 48s) |
[production] |
| 21:34 |
<rzl@deploy1002> |
Synchronized php-1.38.0-wmf.25/extensions/DiscussionTools/modules/controller.js: Backport: [[gerrit:769559|Fix highlighting of comments when reloading (T303261)]] (duration: 00m 47s) |
[production] |
| 21:33 |
<rzl@deploy1002> |
Synchronized php-1.38.0-wmf.25/extensions/VisualEditor/modules/ve-mw: Backport: [[gerrit:769558|Preserve classes on media wrapper links (T292657 T303469)]] (duration: 00m 49s) |
[production] |
| 21:25 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:25 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:25 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:24 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:19 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:18 |
<cstone> |
update Donation Interface revision changed from ca37a93e to 5db12b21 |
[production] |
| 21:18 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:18 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:17 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:13 |
<rzl@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:766307|Remove centralauth-oversight from the config (T302675)]] (duration: 00m 49s) |
[production] |
| 21:12 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:11 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:11 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:09 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 20:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1167 (T300775)', diff saved to https://phabricator.wikimedia.org/P22356 and previous config saved to /var/cache/conftool/dbconfig/20220310-205114-marostegui.json |
[production] |
| 20:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P22355 and previous config saved to /var/cache/conftool/dbconfig/20220310-203608-marostegui.json |
[production] |
| 20:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P22354 and previous config saved to /var/cache/conftool/dbconfig/20220310-202103-marostegui.json |
[production] |
| 20:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1167 (T300775)', diff saved to https://phabricator.wikimedia.org/P22353 and previous config saved to /var/cache/conftool/dbconfig/20220310-200558-marostegui.json |
[production] |
| 19:49 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 19:48 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 19:48 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 19:47 |
<volans> |
installed spicerack v2.3.2 on the cumin hosts |
[production] |
| 19:46 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 19:46 |
<volans@cumin2002> |
END (PASS) - Cookbook sre.misc-clusters.sretest (exit_code=0) rolling restart_daemons on A:sretest |
[production] |
| 19:46 |
<volans@cumin2002> |
START - Cookbook sre.misc-clusters.sretest rolling restart_daemons on A:sretest |
[production] |
| 19:44 |
<dancy@deploy1002> |
rebuilt and synchronized wikiversions files: group2 wikis to 1.38.0-wmf.25 refs T300201 |
[production] |
| 19:44 |
<volans> |
uploaded spicerack_2.3.2 to apt.wikimedia.org buster-wikimedia,bullseye-wikimedia |
[production] |
| 19:33 |
<dduvall@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/blubberoid: apply |
[production] |
| 19:32 |
<dduvall@deploy1002> |
helmfile [eqiad] START helmfile.d/services/blubberoid: apply |
[production] |
| 19:32 |
<dduvall@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/blubberoid: apply |
[production] |
| 19:31 |
<dduvall@deploy1002> |
helmfile [codfw] START helmfile.d/services/blubberoid: apply |
[production] |
| 19:29 |
<dduvall@deploy1002> |
helmfile [staging] DONE helmfile.d/services/blubberoid: apply |
[production] |
| 19:29 |
<dduvall@deploy1002> |
helmfile [staging] START helmfile.d/services/blubberoid: apply |
[production] |
| 19:21 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 19:21 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 19:21 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |