1-50 of 8688 results (23ms)
2025-06-12 §
10:36 <dcaro> rebooting tools-prometheus-8 due to the VM having load issues (not responding to ssh) [tools]
10:34 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0) [tools]
10:28 <wmbot~dcaro@acme> START - Cookbook wmcs.openstack.cloudvirt.vm_console [tools]
2025-06-11 §
13:39 <chuckonwumelu@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld [tools]
13:33 <chuckonwumelu@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld [tools]
11:19 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.logging.copy_images_to_registry (exit_code=0) for Loki 3.5.0, Alloy 1.9.1 [tools]
11:18 <taavi@cloudcumin1001> Updating container image docker-registry.svc.toolforge.org/grafana/alloy:v1.9.1 [tools]
11:18 <taavi@cloudcumin1001> Updating container image docker-registry.svc.toolforge.org/grafana/loki:3.5.0 [tools]
11:18 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.logging.copy_images_to_registry for Loki 3.5.0, Alloy 1.9.1 [tools]
11:09 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.logging.copy_images_to_registry (exit_code=99) for Loki 3.5.0, Alloy 1.9.1 [tools]
11:09 <taavi@cloudcumin1001> Updating container image docker-registry.svc.toolforge.org/grafana/loki:3.5.0 [tools]
11:09 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.logging.copy_images_to_registry for Loki 3.5.0, Alloy 1.9.1 [tools]
2025-06-10 §
17:04 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-harbor [tools]
17:00 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor [tools]
16:41 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer [tools]
16:28 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer [tools]
16:26 <wmbot~dcaro@acme> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-emailer [tools]
16:21 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer [tools]
15:45 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api [tools]
15:33 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [tools]
15:21 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission [tools]
15:15 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission [tools]
14:59 <wmbot~dcaro@acme> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway [tools]
14:57 <wmbot~dcaro@acme> START - Cookbook wmcs.toolforge.component.deploy for component api-gateway [tools]
11:48 <taavi> add AAAA records to tools/toolsbeta-harbor proxies, previous monitoring issues resolved [tools]
2025-06-06 §
21:49 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-54, tools-k8s-worker-nfs-74 [tools]
21:40 <andrewbogott> restarting tools-prometheus-9 and tools-prometheus-8, lots of tools metrics just went dark [tools]
21:37 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-54, tools-k8s-worker-nfs-74 [tools]
18:33 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-cli [tools]
18:20 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli [tools]
15:20 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-5 [tools]
15:14 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-5 [tools]
2025-06-05 §
22:24 <andrewbogott> running /srv/tools/cleanup.sh on tools-nfs-2 in a screen session, trying to clear disk space alert [tools]
15:05 <chuckonwumelu@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api [tools]
14:53 <chuckonwumelu@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component builds-api [tools]
2025-05-30 §
16:27 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [tools]
16:26 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component components-api [tools]
15:48 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-46 [tools]
15:42 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-46 [tools]
15:40 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-38, tools-k8s-worker-nfs-11 [tools]
15:29 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [tools]
15:29 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component components-api [tools]
15:28 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-38, tools-k8s-worker-nfs-11 [tools]
15:28 <raymond-ndibe@cloudcumin1001> END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component components-api [tools]
15:26 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component components-api [tools]
07:38 <taavi> reboot tools-static-15 to unstuck NFS things [tools]
2025-05-24 §
12:57 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-65 [tools]
12:50 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-65 [tools]
2025-05-23 §
16:32 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-38, tools-k8s-worker-nfs-65 [tools]
16:23 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-38, tools-k8s-worker-nfs-65 [tools]