3701-3750 of 10000 results (124ms)
2025-10-08 §
01:54 <ryankemper@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1018.eqiad.wmnet with reason: host reimage [production]
01:50 <ryankemper@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1018.eqiad.wmnet with reason: host reimage [production]
01:14 <mwpresync@deploy2002> Finished scap build-images: Publishing wmf/next image (duration: 13m 13s) [production]
01:01 <mwpresync@deploy2002> Started scap build-images: Publishing wmf/next image [production]
00:27 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
00:17 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
00:09 <sbassett> Deployed security mitigation for T406664 to 1.45.0-wmf.22 [production]
00:00 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
2025-10-07 §
23:58 <sbassett> Deployed security mitigation for T406664 to 1.45.0-wmf.21 [production]
23:58 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:57 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:57 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:57 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:54 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:53 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:53 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:52 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:50 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:47 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:45 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:18 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
23:13 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
22:46 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
22:35 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
22:12 <bking@cumin2002> END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "running per cookbook error suggestion - bking@cumin2002 - T399778" [production]
22:11 <bking@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "running per cookbook error suggestion - bking@cumin2002 - T399778" [production]
22:04 <ryankemper@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1018.eqiad.wmnet with OS bullseye [production]
22:02 <bking@cumin2002> conftool action : set/weight=10:pooled=yes; selector: name=wdqs1020\.eqiad\.wmnet [production]
21:50 <bking@deploy2002> Finished deploy [wdqs/wdqs@fea7794]: T405978 (duration: 00m 45s) [production]
21:49 <bking@deploy2002> Started deploy [wdqs/wdqs@fea7794]: T405978 [production]
21:48 <bking@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on wdqs1020.eqiad.wmnet with reason: finish getting host ready for production [production]
21:41 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T405978, transfer main graph to newly-reimaged host) xfer wikidata_main from wdqs1011.eqiad.wmnet -> wdqs1020.eqiad.wmnet w/ force delete existing files, repooling source-only afterwards [production]
21:41 <tgr_> UTC late deploys done [production]
21:40 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1194261|session: Log actual class name in preventSessionsForUser exception (T406566)]], [[gerrit:1194262|session: Log actual class name in preventSessionsForUser exception (T406566)]], [[gerrit:1194281|session: Log cache write flags in `SessionStore::set()` (T405633 T405634)]], [[gerrit:1194282|session: Log cache write flags in `SessionStore::set()` (T405 [production]
21:36 <tgr@deploy2002> tgr: Continuing with sync [production]
21:34 <tgr@deploy2002> tgr: Backport for [[gerrit:1194261|session: Log actual class name in preventSessionsForUser exception (T406566)]], [[gerrit:1194262|session: Log actual class name in preventSessionsForUser exception (T406566)]], [[gerrit:1194281|session: Log cache write flags in `SessionStore::set()` (T405633 T405634)]], [[gerrit:1194282|session: Log cache write flags in `SessionStore::set()` (T405633 T405634)]] synced [production]
21:30 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1194261|session: Log actual class name in preventSessionsForUser exception (T406566)]], [[gerrit:1194262|session: Log actual class name in preventSessionsForUser exception (T406566)]], [[gerrit:1194281|session: Log cache write flags in `SessionStore::set()` (T405633 T405634)]], [[gerrit:1194282|session: Log cache write flags in `SessionStore::set()` (T4056 [production]
21:28 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
21:17 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
21:17 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
21:17 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
21:16 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
21:14 <amastilovic@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
20:58 <aaron@deploy2002> Finished scap sync-world: Backport for [[gerrit:1175942|Add restbase spec JSON files to which /rest_v1/?spec can be routed (T397203 T396805)]] (duration: 10m 13s) [production]
20:54 <aaron@deploy2002> aaron: Continuing with sync [production]
20:53 <aaron@deploy2002> aaron: Backport for [[gerrit:1175942|Add restbase spec JSON files to which /rest_v1/?spec can be routed (T397203 T396805)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
20:50 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dse-k8s-worker2003.codfw.wmnet with OS bookworm [production]
20:50 <bking@cumin2002> END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - bking@cumin2002" [production]
20:48 <aaron@deploy2002> Started scap sync-world: Backport for [[gerrit:1175942|Add restbase spec JSON files to which /rest_v1/?spec can be routed (T397203 T396805)]] [production]
20:48 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T405978, transfer main graph to newly-reimaged host) xfer wikidata_main from wdqs1011.eqiad.wmnet -> wdqs1020.eqiad.wmnet w/ force delete existing files, repooling source-only afterwards [production]