1-50 of 10000 results (1ms)
2026-09-30 ยง
22:52 <eileen> civicrm upgraded from b03afa89 to 3aadfcea [fundraising]
22:23 <brett@cumin1004> END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on A:cp-text_codfw - 7.1.1-2~bpo13+wmf3 () [production]
21:59 <larssandergreen> civicrm upgraded from da999791 to b03afa89 [fundraising]
21:22 <sbassett> Deployed updated mitigation for T435455 [production]
21:09 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply [production]
21:09 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply [production]
21:00 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply [production]
21:00 <arlolra@deploy1003> Finished scap sync-world: Backport for [[gerrit:1346763|Set a default for wgSplitParsoidParserCache in prod (T432760)]] (duration: 12m 28s) [production]
21:00 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply [production]
20:58 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply [production]
20:57 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply [production]
20:52 <arlolra@deploy1003> arlolra: Continuing with deployment [production]
20:52 <arlolra@deploy1003> arlolra: Backport for [[gerrit:1346763|Set a default for wgSplitParsoidParserCache in prod (T432760)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
20:47 <arlolra@deploy1003> Started scap sync-world: Backport for [[gerrit:1346763|Set a default for wgSplitParsoidParserCache in prod (T432760)]] [production]
20:44 <mstyles@deploy1003> Finished scap sync-world: Backport for [[gerrit:1346327|Email confirmation: implement delayed enforcement experiment (T439283)]], [[gerrit:1346328|Email confirmation: Add server-side enforcement (T439283)]], [[gerrit:1346329|Email confirmation: Prepare for next experiment (T439283)]] (duration: 32m 47s) [production]
20:44 <dancy> Done rebuilding gitlab-cloud-runners (staging) (T425441) [releng]
20:38 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply [production]
20:38 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply [production]
20:32 <mstyles@deploy1003> mstyles, catrope: Continuing with deployment [production]
20:31 <mstyles@deploy1003> mstyles, catrope: Backport for [[gerrit:1346327|Email confirmation: implement delayed enforcement experiment (T439283)]], [[gerrit:1346328|Email confirmation: Add server-side enforcement (T439283)]], [[gerrit:1346329|Email confirmation: Prepare for next experiment (T439283)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
20:20 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply [production]
20:20 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply [production]
20:20 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/matomo-next: apply [production]
20:11 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply [production]
20:11 <mstyles@deploy1003> Started scap sync-world: Backport for [[gerrit:1346327|Email confirmation: implement delayed enforcement experiment (T439283)]], [[gerrit:1346328|Email confirmation: Add server-side enforcement (T439283)]], [[gerrit:1346329|Email confirmation: Prepare for next experiment (T439283)]] [production]
20:06 <kemayo@deploy1003> helmfile [staging] DONE helmfile.d/services/editcheck-headless: apply [production]
20:05 <kemayo@deploy1003> helmfile [staging] START helmfile.d/services/editcheck-headless: apply [production]
19:50 <brett@cumin1004> END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on A:cp-upload_codfw - 7.1.1-2~bpo13+wmf3 () [production]
19:45 <jhathaway> reverting legacy facts disablement, discovered some remaining usage [production]
18:56 <dzahn@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
18:56 <dzahn@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
18:56 <dzahn@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
18:55 <dzahn@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
18:55 <dzahn@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
18:55 <dzahn@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
18:52 <brennen@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.22 refs T438218 [production]
18:49 <dancy> Rebuilding gitlab-cloud-runners (staging) (T425441) [releng]
18:47 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) [admin]
18:40 <brennen@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.22 refs T438218 [production]
18:37 <jhathaway> flipping flag to disable legacy facts in puppet prod [production]
18:30 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=0) [admin]
18:25 <larssandergreen> config revision changed from a79709ab to 7a5261b9 [fundraising]
18:24 <brennen@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.22 refs T438218 [production]
18:22 <brennen@deploy1003> Rolling back deployment [production]
18:22 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.bootstrap_and_add [admin]
18:21 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) [admin]
18:21 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) [admin]
18:21 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) (T429387) [admin]
18:21 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy (T429387) [admin]
18:16 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1346733 [releng]