51-100 of 9447 results (18ms)
2025-09-24 §
17:07 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.k8s.reboot_stuck_workers for tools-k8s-worker-nfs-43 [tools]
16:57 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-73 (T400957) [tools]
16:50 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-73 (T400957) [tools]
13:49 <dcaro> patched all tools with new resource defaults, everything looks good [tools]
13:34 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api [tools]
13:21 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [tools]
13:09 <dcaro> depolyed jobs-api change to default resources, patching existing jobs [tools]
13:08 <dcaro@cloudcumin1001> END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component jobs-cli [tools]
13:07 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli [tools]
12:36 <dcaro@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-kubeusers [tools]
12:29 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers [tools]
12:28 <dcaro@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-kubeusers [tools]
12:14 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers [tools]
12:11 <dcaro@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-kubeusers [tools]
12:03 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers [tools]
03:54 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-12 [tools]
03:52 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-12 [tools]
2025-09-23 §
20:08 <andrewbogott> creating puppetdbpostgres and adding it to tools-puppetdb-2 to store postgres data; the root volume of that VM was filling up and causing widespread puppet issues [tools]
01:55 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-68 [tools]
01:49 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-68 [tools]
2025-09-22 §
16:37 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [tools]
16:32 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component components-api [tools]
12:45 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [tools]
12:40 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component components-api [tools]
2025-09-21 §
09:17 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-37, tools-k8s-worker-nfs-2 [tools]
09:02 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-37, tools-k8s-worker-nfs-2 [tools]
03:16 <dcaro> acking and silencing CPU capacity alerts to handle on Monday, they should not page [tools]
01:46 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [tools]
01:46 <andrew@cloudcumin1001> Added a new k8s worker tools-k8s-worker-113.tools.eqiad1.wikimedia.cloud to the cluster [tools]
01:36 <andrewbogott> adding additional worker node in response to repeated capacity alerts [tools]
01:35 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster [tools]
2025-09-19 §
13:09 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-11 [tools]
13:03 <filippo@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-11 [tools]
2025-09-18 §
13:46 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [tools]
13:42 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component components-api [tools]
11:56 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission [tools]
11:47 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission [tools]
11:45 <dcaro@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component ingress-admission [tools]
11:37 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission [tools]
11:35 <dcaro@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component ingress-admission [tools]
11:29 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.vps.instance.force_reboot (exit_code=0) vm tools-prometheus-9 (cluster eqiad1, project tools) [tools]
11:29 <wmbot~dcaro@acme> START - Cookbook wmcs.vps.instance.force_reboot vm tools-prometheus-9 (cluster eqiad1, project tools) [tools]
11:29 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0) [tools]
11:29 <wmbot~dcaro@acme> START - Cookbook wmcs.openstack.cloudvirt.vm_console [tools]
11:27 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission [tools]
09:42 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-55 [tools]
09:36 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-55 [tools]
09:34 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot_stuck_workers (exit_code=0) no stuck workers found [tools]
09:33 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.k8s.reboot_stuck_workers no stuck workers found [tools]
08:52 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-38, tools-k8s-worker-nfs-26, tools-k8s-worker-nfs-3 [tools]