251-300 of 10000 results (98ms)
2025-11-18 ยง
12:18 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host people1005.eqiad.wmnet [production]
12:15 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host people1005.eqiad.wmnet [production]
12:11 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host people2004.codfw.wmnet [production]
12:07 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host people2004.codfw.wmnet [production]
12:01 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host failoid1003.eqiad.wmnet [production]
11:59 <moritzm> installing rabbitmq-server security updates [production]
11:57 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host failoid1003.eqiad.wmnet [production]
11:53 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/wikifeeds: sync [production]
11:53 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/wikifeeds: sync [production]
11:34 <marostegui@cumin1003> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db1169.eqiad.wmnet with OS trixie [production]
11:29 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host failoid2003.codfw.wmnet [production]
11:27 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-platform-eng: apply [production]
11:27 <marostegui@cumin1003> START - Cookbook sre.hosts.reimage for host db1169.eqiad.wmnet with OS trixie [production]
11:26 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-platform-eng: apply [production]
11:26 <marostegui@cumin1003> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db1169.eqiad.wmnet'] [production]
11:26 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wikidata: apply [production]
11:25 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wikidata: apply [production]
11:25 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host failoid2003.codfw.wmnet [production]
11:25 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-search: apply [production]
11:24 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-search: apply [production]
11:23 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-research: apply [production]
11:22 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-research: apply [production]
11:20 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-main: apply [production]
11:19 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-main: apply [production]
11:18 <marostegui@cumin1003> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1169.eqiad.wmnet'] [production]
11:18 <marostegui@cumin1003> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db1169.eqiad.wmnet'] [production]
11:14 <moritzm> installing qemu security updates [production]
11:09 <marostegui@cumin1003> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1169.eqiad.wmnet'] [production]
11:09 <marostegui@cumin1003> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['db1169.eqiad.wmnet'] [production]
11:09 <marostegui@cumin1003> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1169.eqiad.wmnet'] [production]
11:09 <sfaci@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mpic-next: apply [production]
11:09 <marostegui@cumin1003> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['db1169.eqiad.wmnet'] [production]
11:09 <sfaci@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mpic-next: apply [production]
11:05 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-product: apply [production]
11:04 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-product: apply [production]
11:01 <marostegui@cumin1003> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1169.eqiad.wmnet'] [production]
10:53 <marostegui@cumin1003> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db1169.eqiad.wmnet with OS trixie [production]
10:41 <marostegui@cumin1003> START - Cookbook sre.hosts.reimage for host db1169.eqiad.wmnet with OS trixie [production]
10:40 <marostegui@cumin1003> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db1169.eqiad.wmnet with OS trixie [production]
10:30 <filippo@cumin1003> START - Cookbook sre.hosts.reimage for host cloudcontrol2010-dev.codfw.wmnet with OS trixie [production]
10:27 <moritzm> installing libxml2 security updates [production]
10:25 <jynus@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on backup[1006-1007].eqiad.wmnet,ms-backup[1001-1002].eqiad.wmnet with reason: Network maintenance [production]
10:24 <marostegui@cumin1003> START - Cookbook sre.hosts.reimage for host db1169.eqiad.wmnet with OS trixie [production]
10:24 <marostegui@cumin1003> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db1169.eqiad.wmnet with OS trixie [production]
10:15 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for ms-be1067.eqiad.wmnet [production]
10:15 <mvernon@cumin2002> START - Cookbook sre.hosts.remove-downtime for ms-be1067.eqiad.wmnet [production]
10:13 <hashar@deploy2002> Finished deploy [integration/docroot@a7f5910]: build: Updating npm dependencies (linting only) (duration: 00m 11s) [production]
10:13 <hashar@deploy2002> Started deploy [integration/docroot@a7f5910]: build: Updating npm dependencies (linting only) [production]
10:13 <marostegui@cumin1003> START - Cookbook sre.hosts.reimage for host db1169.eqiad.wmnet with OS trixie [production]
10:09 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1169.eqiad.wmnet with reason: Reimage to trixie [production]