1-50 of 10000 results (20ms)
2026-09-30 ยง
20:11 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/matomo-next: apply [production]
20:11 <mstyles@deploy1003> Started scap sync-world: Backport for [[gerrit:1346327|Email confirmation: implement delayed enforcement experiment (T439283)]], [[gerrit:1346328|Email confirmation: Add server-side enforcement (T439283)]], [[gerrit:1346329|Email confirmation: Prepare for next experiment (T439283)]] [production]
20:06 <kemayo@deploy1003> helmfile [staging] DONE helmfile.d/services/editcheck-headless: apply [production]
20:05 <kemayo@deploy1003> helmfile [staging] START helmfile.d/services/editcheck-headless: apply [production]
19:50 <brett@cumin1004> END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on A:cp-upload_codfw - 7.1.1-2~bpo13+wmf3 () [production]
19:45 <jhathaway> reverting legacy facts disablement, discovered some remaining usage [production]
18:56 <dzahn@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
18:56 <dzahn@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
18:56 <dzahn@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
18:55 <dzahn@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
18:55 <dzahn@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
18:55 <dzahn@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
18:52 <brennen@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.22 refs T438218 [production]
18:40 <brennen@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.22 refs T438218 [production]
18:37 <jhathaway> flipping flag to disable legacy facts in puppet prod [production]
18:24 <brennen@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.22 refs T438218 [production]
18:22 <brennen@deploy1003> Rolling back deployment [production]
18:10 <brennen> 1.47.0-wmf.22 train status (T438218): no current blockers, rolling to group1 [production]
17:47 <ryankemper@cumin2003> END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) [production]
17:34 <ebernhardson@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply [production]
17:34 <ebernhardson@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply [production]
17:33 <ebernhardson@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply [production]
17:33 <ebernhardson@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply [production]
17:28 <ryankemper@cumin2003> END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) [production]
17:21 <brett@cumin1004> START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-upload_codfw - 7.1.1-2~bpo13+wmf3 () [production]
17:21 <ebernhardson@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply [production]
17:21 <ebernhardson@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-ssd: apply [production]
17:21 <brett@cumin1004> START - Cookbook sre.cdn.roll-upgrade-varnish rolling upgrade of Varnish on A:cp-text_codfw - 7.1.1-2~bpo13+wmf3 () [production]
17:18 <ryankemper@cumin2003> START - Cookbook sre.wdqs.restart [production]
17:18 <ryankemper@cumin2003> START - Cookbook sre.wdqs.restart [production]
17:15 <bking@dns2004> END - running authdns-update [production]
17:13 <bking@dns2004> START - running authdns-update [production]
17:09 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
17:08 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
16:52 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
16:51 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
16:38 <andrew@cumin2003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcephosd1043.eqiad.wmnet with OS bookworm [production]
16:14 <aaron@deploy1003> helmfile [eqiad] DONE helmfile.d/services/developer-portal: apply [production]
16:14 <aaron@deploy1003> helmfile [eqiad] START helmfile.d/services/developer-portal: apply [production]
16:13 <aaron@deploy1003> helmfile [codfw] DONE helmfile.d/services/developer-portal: apply [production]
16:13 <aaron@deploy1003> helmfile [codfw] START helmfile.d/services/developer-portal: apply [production]
16:12 <andrew@cumin2003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1043.eqiad.wmnet with reason: host reimage [production]
16:11 <aaron@deploy1003> helmfile [staging] DONE helmfile.d/services/developer-portal: apply [production]
16:11 <aaron@deploy1003> helmfile [staging] START helmfile.d/services/developer-portal: apply [production]
16:08 <andrew@cumin2003> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1043.eqiad.wmnet with reason: host reimage [production]
15:49 <andrew@cumin2003> START - Cookbook sre.hosts.reimage for host cloudcephosd1043.eqiad.wmnet with OS bookworm [production]
15:47 <andrew@cumin2003> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcephosd1043.eqiad.wmnet with OS bookworm [production]
15:47 <andrew@cumin2003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1043.eqiad.wmnet with reason: host reimage [production]
15:42 <andrew@cumin2003> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1043.eqiad.wmnet with reason: host reimage [production]
15:41 <oblivian@deploy1003> helmfile [eqiad] DONE helmfile.d/services/shellbox-timeline: apply [production]