|
2025-03-31
ยง
|
| 14:25 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4007.ulsfo.wmnet |
[production] |
| 14:23 |
<dcausse@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:22 |
<dcausse@deploy1003> |
helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:08 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster cloudelastic: apply new plugins - bking@cumin2002 - T390100 |
[production] |
| 14:02 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: relforge1004* for ban relforge1004 prior to service restart and decom T390565 - bking@cumin2002 - T390100 |
[production] |
| 14:01 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.ban Banning hosts: relforge1004* for ban relforge1004 prior to service restart and decom T390565 - bking@cumin2002 - T390100 |
[production] |
| 13:58 |
<godog> |
move k8s instances from prometheus1005 to prometheus1007 - T383232 |
[production] |
| 13:57 |
<cgoubert@deploy1003> |
Finished scap sync-world: Deploy mediawiki chart 0.8.8 (duration: 02m 19s) |
[production] |
| 13:57 |
<cgoubert@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/mw-cron: apply |
[production] |
| 13:56 |
<cgoubert@deploy1003> |
helmfile [eqiad] START helmfile.d/services/mw-cron: apply |
[production] |
| 13:56 |
<cgoubert@deploy1003> |
cgoubert: Continuing with sync |
[production] |
| 13:55 |
<cgoubert@deploy1003> |
cgoubert: Deploy mediawiki chart 0.8.8 synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:55 |
<cgoubert@deploy1003> |
Started scap sync-world: Deploy mediawiki chart 0.8.8 |
[production] |
| 13:52 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:52 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:51 |
<tgr_> |
UTC afternoon deploys done |
[production] |
| 13:51 |
<tgr@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1132020|Enable SUL3 everywhere (T384220)]], [[gerrit:1132615|OATHAuth: Mark centralnoticeadmin as requiring 2FA (T208113)]], [[gerrit:1131990|SpecialTranslationTargetLanguages: Use cxserver-supported language codes (T390300)]] (duration: 20m 25s) |
[production] |
| 13:50 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:49 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:45 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:45 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:44 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:44 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:44 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:44 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:43 |
<tgr@deploy1003> |
ngkountas, tgr: Continuing with sync |
[production] |
| 13:35 |
<tgr@deploy1003> |
ngkountas, tgr: Backport for [[gerrit:1132020|Enable SUL3 everywhere (T384220)]], [[gerrit:1132615|OATHAuth: Mark centralnoticeadmin as requiring 2FA (T208113)]], [[gerrit:1131990|SpecialTranslationTargetLanguages: Use cxserver-supported language codes (T390300)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:33 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply |
[production] |
| 13:33 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:32 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply |
[production] |
| 13:31 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:31 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:30 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:30 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:30 |
<tgr@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1132020|Enable SUL3 everywhere (T384220)]], [[gerrit:1132615|OATHAuth: Mark centralnoticeadmin as requiring 2FA (T208113)]], [[gerrit:1131990|SpecialTranslationTargetLanguages: Use cxserver-supported language codes (T390300)]] |
[production] |
| 13:30 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:30 |
<volans@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:05:00 on sretest1002.eqiad.wmnet with reason: Test |
[production] |
| 13:27 |
<tgr@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1131384|REST: Enable REST Sandbox on an initial set of production wikis (T389407)]] (duration: 18m 21s) |
[production] |
| 13:27 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:27 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:26 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:26 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:25 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:25 |
<dcausse@deploy1003> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:22 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: relforge1003* for ban relforge1003 prior to service restart - bking@cumin2002 - T390100 |
[production] |
| 13:22 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.ban Banning hosts: relforge1003* for ban relforge1003 prior to service restart - bking@cumin2002 - T390100 |
[production] |
| 13:20 |
<tgr@deploy1003> |
bpirkle, tgr: Continuing with sync |
[production] |
| 13:19 |
<bking@cumin2002> |
END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: apply updated master config - bking@cumin2002 - T390100 |
[production] |
| 13:19 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: apply updated master config - bking@cumin2002 - T390100 |
[production] |
| 13:17 |
<dcausse@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |