151-200 of 10000 results (1ms)
2026-09-10 ยง
14:21 <cdanis@cumin1004> END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: etcd fanout fix & details UX - cdanis@cumin1004 [production]
14:20 <cdanis@cumin1004> START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: etcd fanout fix & details UX - cdanis@cumin1004 [production]
14:20 <cdanis@cumin1004> START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "etcd fanout fix & details UX - cdanis@cumin1004" [production]
14:17 <bking@cumin2003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddumps1001.wikimedia.org with reason: host reimage [production]
14:13 <bking@cumin2003> START - Cookbook sre.hosts.downtime for 2:00:00 on clouddumps1001.wikimedia.org with reason: host reimage [production]
14:09 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-cli [tools]
14:07 <bking@cumin2003> START - Cookbook sre.presto.roll-restart-workers for Presto an-presto cluster: Roll restart of all Presto's jvm daemons. [production]
14:06 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component components-cli [tools]
14:06 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-cli [toolsbeta]
14:03 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component components-cli [toolsbeta]
14:02 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [tools]
13:57 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component components-api [tools]
13:57 <bking@cumin2003> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bookworm [production]
13:56 <jelto@cumin1003> END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-eqiad@eqiad [production]
13:56 <jelto@cumin1003> END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs [production]
13:55 <jelto@cumin1003> START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs [production]
13:52 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [toolsbeta]
13:51 <jelto@cumin1003> START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-eqiad@eqiad [production]
13:47 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component components-api [toolsbeta]
13:42 <jelto@cumin1003> END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-codfw@codfw [production]
13:41 <jelto@cumin1003> END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs [production]
13:41 <jelto@cumin1003> START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs [production]
13:37 <jelto@cumin1003> START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-codfw@codfw [production]
12:48 <klausman@dns1004> END - running authdns-update [production]
12:46 <klausman@dns1004> START - running authdns-update [production]
12:35 <dbrant@deploy1003> helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply [production]
12:35 <dbrant@deploy1003> helmfile [codfw] START helmfile.d/services/wikifeeds: apply [production]
12:34 <dbrant@deploy1003> helmfile [eqiad] DONE helmfile.d/services/wikifeeds: apply [production]
12:34 <dbrant@deploy1003> helmfile [eqiad] START helmfile.d/services/wikifeeds: apply [production]
12:33 <dbrant@deploy1003> helmfile [staging] DONE helmfile.d/services/wikifeeds: apply [production]
12:33 <dbrant@deploy1003> helmfile [staging] START helmfile.d/services/wikifeeds: apply [production]
12:05 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb1025.eqiad.wmnet with reason: Cloning x4 [production]
12:01 <btullis@cumin1004> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-worker2005.codfw.wmnet [production]
11:55 <btullis@cumin1004> START - Cookbook sre.hosts.reboot-single for host dse-k8s-worker2005.codfw.wmnet [production]
11:54 <btullis@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-worker1144.eqiad.wmnet with reason: Upgrading RAID firmware [production]
11:52 <cgoubert@dns1004> END - running authdns-update [production]
11:49 <cgoubert@dns1004> START - running authdns-update [production]
11:31 <btullis@cumin1004> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-worker2004.codfw.wmnet [production]
11:25 <btullis@cumin1004> START - Cookbook sre.hosts.reboot-single for host dse-k8s-worker2004.codfw.wmnet [production]
11:24 <btullis@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-worker1204.eqiad.wmnet with reason: Upgrading RAID firmware [production]
11:16 <btullis@cumin1004> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for an-worker1200.eqiad.wmnet [production]
11:16 <btullis@cumin1004> START - Cookbook sre.hosts.remove-downtime for an-worker1200.eqiad.wmnet [production]
11:04 <btullis@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-worker1200.eqiad.wmnet with reason: Upgrading RAID firmware [production]
11:03 <btullis@cumin1004> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for an-worker1199.eqiad.wmnet [production]
11:03 <btullis@cumin1004> START - Cookbook sre.hosts.remove-downtime for an-worker1199.eqiad.wmnet [production]
10:50 <sfaci@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen: apply [production]
10:50 <sfaci@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen: apply [production]
10:42 <btullis@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-worker1199.eqiad.wmnet with reason: Upgrading RAID firmware [production]
10:10 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb1024.eqiad.wmnet with reason: Cloning x4 [production]
10:00 <filippo@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1073.eqiad.wmnet with OS trixie [production]