201-250 of 10000 results (26ms)
2026-05-18 ยง
18:26 <swfrench@cumin1003> START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{wikikube-worker[2203-2331].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) [production]
18:22 <swfrench@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2191-2202].codfw.wmnet [production]
18:22 <swfrench@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2191-2202].codfw.wmnet [production]
18:20 <dzahn@cumin2002> START - Cookbook sre.hosts.reboot-single for host gitlab-runner2003.codfw.wmnet [production]
18:20 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gitlab-runner2002.codfw.wmnet [production]
18:19 <swfrench@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2189.codfw.wmnet [production]
18:19 <swfrench@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2189.codfw.wmnet [production]
18:17 <mutante> releases.wikimedia.org - rebooting backends [releng]
18:16 <mutante> releases.wikimedia.org - rebooting backends [production]
18:16 <dzahn@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on releases2003.codfw.wmnet with reason: T426563 [production]
18:14 <mutante> rebooting production gitlab-runners [releng]
18:13 <herron@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kafkamon1003.eqiad.wmnet [production]
18:13 <dzahn@cumin2002> START - Cookbook sre.hosts.reboot-single for host gitlab-runner2002.codfw.wmnet [production]
18:13 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gitlab-runner1004.eqiad.wmnet [production]
18:12 <dancy> gitlab-cloud-runners have been revived. [releng]
18:11 <herron@cumin1003> START - Cookbook sre.hosts.reboot-single for host kafkamon1003.eqiad.wmnet [production]
18:11 <James_F> Zuul: [design/codex] Switch CI to Node 24 [releng]
18:11 <herron@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kafkamon2003.codfw.wmnet [production]
18:07 <herron@cumin1003> START - Cookbook sre.hosts.reboot-single for host kafkamon2003.codfw.wmnet [production]
18:06 <dzahn@cumin2002> START - Cookbook sre.hosts.reboot-single for host gitlab-runner1004.eqiad.wmnet [production]
18:06 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gitlab-runner1003.eqiad.wmnet [production]
18:04 <blake@cumin1003> END (FAIL) - Cookbook sre.k8s.reboot-nodes (exit_code=1) rolling reboot on P{wikikube-worker[2155-2331].codfw.wmnet} and (A:wikikube-master-codfw or A:wikikube-worker-codfw) [production]
18:02 <Reedy> Deployed patch for T426631 [production]
17:59 <dzahn@cumin2002> START - Cookbook sre.hosts.reboot-single for host gitlab-runner1003.eqiad.wmnet [production]
17:56 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gitlab-runner1002.eqiad.wmnet [production]
17:56 <herron@cumin1003> END (PASS) - Cookbook sre.kafka.roll-restart-reboot-brokers (exit_code=0) rolling reboot on A:kafka-logging-eqiad [production]
17:50 <herron@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host centrallog2002.codfw.wmnet [production]
17:50 <dzahn@cumin2002> START - Cookbook sre.hosts.reboot-single for host gitlab-runner1002.eqiad.wmnet [production]
17:48 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1061.eqiad.wmnet with OS bookworm [production]
17:46 <dzahn@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on phab1005.eqiad.wmnet with reason: T426563 [production]
17:46 <herron> rebooting alert2002 [production]
17:45 <dzahn@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on phab2003.codfw.wmnet with reason: T426563 [production]
17:45 <herron@cumin1003> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host alert2002.wikimedia.org [production]
17:45 <herron@cumin1003> START - Cookbook sre.hosts.reboot-single for host alert2002.wikimedia.org [production]
17:44 <herron@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host grafana1002.eqiad.wmnet [production]
17:44 <herron@cumin1003> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host alert2002.wikimedia.org [production]
17:44 <herron@cumin1003> START - Cookbook sre.hosts.reboot-single for host alert2002.wikimedia.org [production]
17:44 <herron@cumin1003> START - Cookbook sre.hosts.reboot-single for host centrallog2002.codfw.wmnet [production]
17:43 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1062.eqiad.wmnet with OS bookworm [production]
17:40 <herron@cumin1003> START - Cookbook sre.hosts.reboot-single for host grafana1002.eqiad.wmnet [production]
17:38 <herron@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host graphite1005.eqiad.wmnet [production]
17:37 <mutante> stewards* - rebooting [production]
17:36 <herron@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host grafana2001.codfw.wmnet [production]
17:32 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1061.eqiad.wmnet with reason: host reimage [production]
17:32 <herron@cumin1003> START - Cookbook sre.hosts.reboot-single for host grafana2001.codfw.wmnet [production]
17:31 <herron@cumin1003> START - Cookbook sre.hosts.reboot-single for host graphite1005.eqiad.wmnet [production]
17:30 <herron@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host graphite2004.codfw.wmnet [production]
17:28 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1062.eqiad.wmnet with reason: host reimage [production]
17:25 <herron@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mwlog1003.eqiad.wmnet [production]
17:23 <jiji@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1061.eqiad.wmnet with reason: host reimage [production]