151-200 of 10000 results (118ms)
2026-05-18 ยง
12:40 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2179', diff saved to https://phabricator.wikimedia.org/P92569 and previous config saved to /var/cache/conftool/dbconfig/20260518-124030-fceratto.json [production]
12:34 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host bast7002.wikimedia.org [production]
12:34 <blake@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2070-2072].codfw.wmnet [production]
12:32 <blake@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2070-2072].codfw.wmnet [production]
12:31 <blake@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2067-2069].codfw.wmnet [production]
12:31 <blake@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2067-2069].codfw.wmnet [production]
12:31 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host debmonitor1003.eqiad.wmnet [production]
12:30 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2179', diff saved to https://phabricator.wikimedia.org/P92568 and previous config saved to /var/cache/conftool/dbconfig/20260518-123022-fceratto.json [production]
12:27 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host debmonitor1003.eqiad.wmnet [production]
12:26 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host debmonitor2003.codfw.wmnet [production]
12:22 <blake@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2067-2069].codfw.wmnet [production]
12:22 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1069.eqiad.wmnet with OS bookworm [production]
12:22 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host debmonitor2003.codfw.wmnet [production]
12:21 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host debmonitor-dev2001.codfw.wmnet [production]
12:20 <blake@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2067-2069].codfw.wmnet [production]
12:20 <blake@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2062,2064-2065].codfw.wmnet [production]
12:20 <blake@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2062,2064-2065].codfw.wmnet [production]
12:20 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2179 (T419635)', diff saved to https://phabricator.wikimedia.org/P92567 and previous config saved to /var/cache/conftool/dbconfig/20260518-122014-fceratto.json [production]
12:18 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1071.eqiad.wmnet with OS bookworm [production]
12:17 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host debmonitor-dev2001.codfw.wmnet [production]
12:14 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1070.eqiad.wmnet with OS bookworm [production]
12:13 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netbox-dev2003.codfw.wmnet [production]
12:13 <blake@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2062,2064-2065].codfw.wmnet [production]
12:11 <blake@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2062,2064-2065].codfw.wmnet [production]
12:11 <blake@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2059-2061].codfw.wmnet [production]
12:11 <blake@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2059-2061].codfw.wmnet [production]
12:10 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1072.eqiad.wmnet with OS bookworm [production]
12:09 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host netbox-dev2003.codfw.wmnet [production]
12:07 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1069.eqiad.wmnet with reason: host reimage [production]
12:04 <blake@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2059-2061].codfw.wmnet [production]
12:02 <blake@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2059-2061].codfw.wmnet [production]
12:02 <blake@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2056-2058].codfw.wmnet [production]
12:02 <Dreamy_Jazz> Ran `scap remove-patch --message-body 'Dropping patch already made public' /srv/patches/next/extensions/ConfirmEdit/01-T423840.patch` [production]
12:02 <blake@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2056-2058].codfw.wmnet [production]
12:02 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1071.eqiad.wmnet with reason: host reimage [production]
12:00 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cumin2003.codfw.wmnet [production]
11:58 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1070.eqiad.wmnet with reason: host reimage [production]
11:55 <blake@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2056-2058].codfw.wmnet [production]
11:54 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1072.eqiad.wmnet with reason: host reimage [production]
11:54 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host cumin2003.codfw.wmnet [production]
11:53 <blake@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2056-2058].codfw.wmnet [production]
11:53 <blake@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2050-2051,2055].codfw.wmnet [production]
11:53 <blake@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2050-2051,2055].codfw.wmnet [production]
11:53 <jiji@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1069.eqiad.wmnet with reason: host reimage [production]
11:53 <fceratto@cumin1003> dbctl commit (dc=all): 'Promote db2213 to s5 primary T426600', diff saved to https://phabricator.wikimedia.org/P92566 and previous config saved to /var/cache/conftool/dbconfig/20260518-115304-fceratto.json [production]
11:52 <federico3> Starting s5 codfw failover from db2192 to db2213 - T426600 [production]
11:51 <jiji@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1070.eqiad.wmnet with reason: host reimage [production]
11:50 <jiji@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1071.eqiad.wmnet with reason: host reimage [production]
11:50 <jiji@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1072.eqiad.wmnet with reason: host reimage [production]
11:49 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host failoid1003.eqiad.wmnet [production]