951-1000 of 10000 results (23ms)
2026-03-17 ยง
19:25 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_etcd_node [toolsbeta]
19:23 <cdobbins@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp3068.esams.wmnet with reason: host reimage [production]
19:23 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0) [toolsbeta]
19:21 <cdobbins@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp3069.esams.wmnet with reason: host reimage [production]
19:20 <cdobbins@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp3068.esams.wmnet with reason: host reimage [production]
19:19 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on bast2003.wikimedia.org with reason: host reimage [production]
19:11 <jhancock@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on bast2003.wikimedia.org with reason: host reimage [production]
19:08 <dzahn@dns1004> END - running authdns-update [production]
19:07 <dzahn@dns1004> START - running authdns-update [production]
19:05 <brett@cumin2002> START - Cookbook sre.hosts.reimage for host cp3081.esams.wmnet with OS trixie [production]
19:05 <brett@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp3081.esams.wmnet with OS trixie [production]
19:04 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_etcd_node [toolsbeta]
19:02 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0) [toolsbeta]
19:00 <brett@puppetserver1001> conftool action : set/pooled=yes; selector: name=cp3080.* [production]
18:56 <cdobbins@cumin2002> START - Cookbook sre.hosts.reimage for host cp3069.esams.wmnet with OS trixie [production]
18:55 <cdobbins@cumin2002> conftool action : set/pooled=no; selector: name=cp3069.esams.wmnet [reason: trixie reimaging] [production]
18:55 <cdobbins@cumin2002> START - Cookbook sre.hosts.reimage for host cp3068.esams.wmnet with OS trixie [production]
18:55 <btullis@cumin1003> END (PASS) - Cookbook sre.druid.reboot-workers (exit_code=0) for Druid public cluster: Reboot Druid nodes [production]
18:54 <cdobbins@cumin2002> conftool action : set/pooled=no; selector: name=cp3068.esams.wmnet [reason: trixie reimaging] [production]
18:53 <cdobbins@cumin2002> conftool action : set/pooled=yes; selector: name=cp3066.esams.wmnet [reason: trixie reimaging] [production]
18:53 <cdobbins@cumin2002> conftool action : set/pooled=yes; selector: name=cp3067.esams.wmnet [reason: trixie reimaging] [production]
18:52 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_etcd_node [toolsbeta]
18:52 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99) [toolsbeta]
18:50 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host bast2003.wikimedia.org with OS trixie [production]
18:49 <swfrench-wmf> manually uncordoned wikikube-worker-exp1001.eqiad.wmnet after failed reboot [production]
18:47 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp3080.esams.wmnet with OS trixie [production]
18:44 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_etcd_node [toolsbeta]
18:43 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0) [toolsbeta]
18:43 <cdobbins@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp3067.esams.wmnet with OS trixie [production]
18:40 <cdobbins@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp3066.esams.wmnet with OS trixie [production]
18:40 <jgleeson> donorwiki upgraded from 4c09db39 to 7d1666f9 [fundraising]
18:32 <dwisehaupt@dns1005> END - running authdns-update [production]
18:31 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host bast2003.wikimedia.org with OS bookworm [production]
18:31 <dwisehaupt@dns1005> START - running authdns-update [production]
18:25 <btullis@cumin1003> START - Cookbook sre.hosts.reimage for host dse-k8s-worker1015.eqiad.wmnet with OS bookworm [production]
18:25 <btullis@cumin1003> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host dse-k8s-worker1015.eqiad.wmnet with OS bookworm [production]
18:23 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_etcd_node [toolsbeta]
18:21 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0) [toolsbeta]
18:20 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp3080.esams.wmnet with reason: host reimage [production]
18:19 <brett@cumin2002> END (PASS) - Cookbook sre.cdn.roll-reboot (exit_code=0) rolling reboot on P{cp700[5-8].magru.wmnet} and A:cp [production]
18:19 <brett@cumin2002> cookbooks.sre.cdn.roll-reboot finished rebooting cp7008.magru.wmnet [production]
18:17 <akhatun@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
18:16 <akhatun@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
18:16 <cdobbins@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp3067.esams.wmnet with reason: host reimage [production]
18:16 <kevinbazira@deploy2002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
18:13 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_etcd_node [toolsbeta]
18:13 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99) [toolsbeta]
18:13 <cdobbins@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp3066.esams.wmnet with reason: host reimage [production]
18:09 <brett@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp3080.esams.wmnet with reason: host reimage [production]
18:09 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on bast2003.wikimedia.org with reason: host reimage [production]