|
2026-06-12
§
|
| 11:36 |
<brouberol@deploy1003> |
helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 11:35 |
<brouberol@deploy1003> |
helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. |
[production] |
| 11:35 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 11:34 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 11:24 |
<jelto@cumin1003> |
END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1004.wikimedia.org with reason: Upgrade GitLab |
[production] |
| 11:14 |
<wmftkbot> |
Test Kitchen edge-unique experiments (poll 4) - adds: logged-out-retention-round12, share-highlight, we-1-8-mobile-account-menu, account-creation-reading-list-cta, logged-out-retention-round13, attribution-research-2026-06-run, logged-out-retention-round15, ncs-cors-test, logged-out-retention-round14; removes: none; fields: none - TK tips at https://w.wiki/FwuD |
[analytics] |
| 11:14 |
<wmftkbot> |
Test Kitchen edge-unique experiments (poll 3) - adds: logged-out-retention-round12, share-highlight, we-1-8-mobile-account-menu, account-creation-reading-list-cta, logged-out-retention-round13, attribution-research-2026-06-run, logged-out-retention-round15, ncs-cors-test, logged-out-retention-round14; removes: none; fields: none - TK tips at https://w.wiki/FwuD |
[analytics] |
| 11:14 |
<wmftkbot> |
Test Kitchen edge-unique experiments (poll 2) - adds: logged-out-retention-round12, share-highlight, we-1-8-mobile-account-menu, account-creation-reading-list-cta, logged-out-retention-round13, attribution-research-2026-06-run, logged-out-retention-round15, ncs-cors-test, logged-out-retention-round14; removes: none; fields: none - TK tips at https://w.wiki/FwuD |
[analytics] |
| 11:13 |
<wmftkbot> |
Test Kitchen edge-unique experiments (poll 1) - adds: logged-out-retention-round12, share-highlight, we-1-8-mobile-account-menu, account-creation-reading-list-cta, logged-out-retention-round13, attribution-research-2026-06-run, logged-out-retention-round15, ncs-cors-test, logged-out-retention-round14; removes: none; fields: none - TK tips at https://w.wiki/FwuD |
[analytics] |
| 11:07 |
<trueg@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply |
[production] |
| 10:56 |
<atsuko@deploy1003> |
helmfile [staging] DONE helmfile.d/services/toolhub: apply |
[production] |
| 10:56 |
<atsuko@deploy1003> |
helmfile [staging] START helmfile.d/services/toolhub: apply |
[production] |
| 10:55 |
<trueg@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply |
[production] |
| 10:49 |
<atsuko@deploy1003> |
helmfile [staging] DONE helmfile.d/services/toolhub: apply |
[production] |
| 10:49 |
<atsuko@deploy1003> |
helmfile [staging] START helmfile.d/services/toolhub: apply |
[production] |
| 10:40 |
<cgoubert@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 10:37 |
<cgoubert@deploy1003> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 10:36 |
<cgoubert@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 10:35 |
<cgoubert@deploy1003> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 10:35 |
<cgoubert@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 10:35 |
<cgoubert@deploy1003> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 10:21 |
<Krinkle> |
`krinkle@{doc1004,doc2003}:/srv/doc/mediawiki-core$ sudo -u doc-uploader rm -rf list/` - remove doc build for git-tag test. |
[releng] |
| 10:12 |
<atsuko@deploy1003> |
helmfile [staging] DONE helmfile.d/services/toolhub: apply |
[production] |
| 10:12 |
<atsuko@deploy1003> |
helmfile [staging] START helmfile.d/services/toolhub: apply |
[production] |
| 10:08 |
<jelto@cumin1003> |
START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1004.wikimedia.org with reason: Upgrade GitLab |
[production] |
| 09:59 |
<gkyziridis@deploy1003> |
helmfile [ml-serve-codfw] 'sync' command on namespace 'liftwing-openapi-server' for release 'main' . |
[production] |
| 09:58 |
<gkyziridis@deploy1003> |
helmfile [ml-serve-eqiad] 'sync' command on namespace 'liftwing-openapi-server' for release 'main' . |
[production] |
| 09:57 |
<gkyziridis@deploy1003> |
helmfile [ml-staging-codfw] 'sync' command on namespace 'liftwing-openapi-server' for release 'main' . |
[production] |
| 06:13 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.puppet.disable-merges (exit_code=0) |
[production] |
| 06:11 |
<jmm@cumin2002> |
START - Cookbook sre.puppet.disable-merges |
[production] |
| 03:42 |
<eileen> |
civicrm upgraded from 0b8db587 to 69a60dcb |
[fundraising] |
| 03:07 |
<ryankemper> |
T427951 sorry, `[eqiad,codfw].mediawiki.page_html_content_change.rc0` (accidentally a word) |
[production] |
| 03:06 |
<ryankemper> |
T427951 Deleted all 20 unused dev/test topics on kafka-jumbo (verified empty first); 2 (`[eqiad,codfw]page_html_content_change.rc0`) were immediately auto-recreated empty by a still-running `dse-k8s` enrichment consumer; awaiting owner confirmation before final re-delete |
[production] |
| 02:01 |
<mwpresync@deploy1003> |
Finished scap build-images: Publishing wmf/next image (duration: 01m 13s) |
[production] |
| 02:00 |
<mwpresync@deploy1003> |
Started scap build-images: Publishing wmf/next image |
[production] |
| 00:00 |
<bblack@cumin1003> |
END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on A:cp-upload and not P{cp7008.magru.wmnet} and A:cp - Upgrade wmfuniq to 0.3.0 () |
[production] |
|
2026-06-11
§
|
| 22:27 |
<atsuko@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 22:26 |
<atsuko@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 22:14 |
<atsuko@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-ttmserver-test: apply |
[production] |
| 22:13 |
<atsuko@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-ttmserver-test: apply |
[production] |
| 22:05 |
<egardner@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1300906|Restore MediaViewer toggle in Special:Preferences (T428742)]] (duration: 30m 51s) |
[production] |
| 21:58 |
<dzahn@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host releases2003.codfw.wmnet with OS trixie |
[production] |
| 21:52 |
<egardner@deploy1003> |
egardner: Continuing with deployment |
[production] |
| 21:51 |
<egardner@deploy1003> |
egardner: Backport for [[gerrit:1300906|Restore MediaViewer toggle in Special:Preferences (T428742)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 21:34 |
<egardner@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1300906|Restore MediaViewer toggle in Special:Preferences (T428742)]] |
[production] |
| 21:34 |
<dzahn@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on releases2003.codfw.wmnet with reason: host reimage |
[production] |
| 21:29 |
<arlolra@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1300913|Avoid the escaping from nowiki processing (T398967)]] (duration: 09m 09s) |
[production] |
| 21:28 |
<dzahn@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on releases2003.codfw.wmnet with reason: host reimage |
[production] |
| 21:25 |
<arlolra@deploy1003> |
arlolra: Continuing with deployment |
[production] |
| 21:22 |
<arlolra@deploy1003> |
arlolra: Backport for [[gerrit:1300913|Avoid the escaping from nowiki processing (T398967)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |