1-50 of 10000 results (50ms)
2026-09-09 ยง
18:23 <dduvall@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.19 refs T430838 [production]
18:06 <eevans@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1026.eqiad.wmnet with OS bookworm [production]
18:03 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wmde: apply [production]
18:03 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wmde: apply [production]
18:03 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wikidata: apply [production]
18:02 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wikidata: apply [production]
18:02 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
18:01 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
18:01 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-sre: apply [production]
18:00 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-sre: apply [production]
17:58 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-search: apply [production]
17:58 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-search: apply [production]
17:57 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-research: apply [production]
17:57 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-research: apply [production]
17:56 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-platform-eng: apply [production]
17:56 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-platform-eng: apply [production]
17:56 <ladsgroup@deploy1003> helmfile [eqiad] DONE helmfile.d/services/thumbor: apply [production]
17:56 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply [production]
17:55 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply [production]
17:54 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-main: apply [production]
17:54 <ladsgroup@deploy1003> helmfile [eqiad] START helmfile.d/services/thumbor: apply [production]
17:54 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-main: apply [production]
17:53 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-fr-tech: apply [production]
17:53 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-fr-tech: apply [production]
17:52 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-experiment-platform: apply [production]
17:52 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-experiment-platform: apply [production]
17:51 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply [production]
17:51 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply [production]
17:50 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
17:50 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
17:50 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-product: apply [production]
17:49 <rzl@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-cron: apply [production]
17:49 <eevans@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1026.eqiad.wmnet with reason: host reimage [production]
17:49 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-product: apply [production]
17:49 <rzl@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-cron: apply [production]
17:45 <eevans@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1026.eqiad.wmnet with reason: host reimage [production]
17:36 <eevans@cumin1003> START - Cookbook sre.hosts.reimage for host aqs1026.eqiad.wmnet with OS bookworm [production]
17:35 <eevans@cumin1003> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host aqs1026.eqiad.wmnet with OS bookworm [production]
17:31 <eevans@cumin1003> START - Cookbook sre.hosts.reimage for host aqs1026.eqiad.wmnet with OS bookworm [production]
17:29 <ladsgroup@deploy1003> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
17:27 <ladsgroup@deploy1003> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
17:23 <ladsgroup@deploy1003> helmfile [staging] DONE helmfile.d/services/thumbor: apply [production]
17:23 <ladsgroup@deploy1003> helmfile [staging] START helmfile.d/services/thumbor: apply [production]
17:12 <eevans@cumin1003> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs1026.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:04 <eevans@cumin1003> START - Cookbook sre.hosts.provision for host aqs1026.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
16:46 <urbanecm@deploy1003> Finished scap sync-world: Backport for [[gerrit:1338173|testwiki: allow testing new AccountSetup experiment (T436872)]] (duration: 09m 28s) [production]
16:41 <urbanecm@deploy1003> migr, urbanecm: Continuing with deployment [production]
16:41 <urbanecm@deploy1003> migr, urbanecm: Backport for [[gerrit:1338173|testwiki: allow testing new AccountSetup experiment (T436872)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
16:36 <urbanecm@deploy1003> Started scap sync-world: Backport for [[gerrit:1338173|testwiki: allow testing new AccountSetup experiment (T436872)]] [production]
16:28 <cgoubert@deploy1003> helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply [production]