1-50 of 10000 results (26ms)
2026-07-30 §
15:18 <raymond-ndibe@cloudcumin1001> END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component jobs-cli [tools]
15:10 <godog> hard reboot tools-bastion-15 [tools]
15:09 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli [tools]
15:08 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-cli [tools]
15:06 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli [tools]
15:06 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-cli [tools]
15:05 <wmbot~dcaro@acme> END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255) [tools]
15:04 <dcaro> reboot tools-bastion-15, getting out of memory [tools]
15:03 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli [tools]
15:00 <wmbot~dcaro@acme> START - Cookbook wmcs.openstack.cloudvirt.vm_console [tools]
15:00 <wmbot~dcaro@acme> END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255) [tools]
15:00 <wmbot~dcaro@acme> START - Cookbook wmcs.openstack.cloudvirt.vm_console [tools]
14:43 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api [tools]
14:30 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [tools]
14:28 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api [tools]
14:25 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [tools]
14:01 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component webservice-cli [tools]
14:00 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component webservice-cli [tools]
13:59 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component webservice-cli [tools]
13:58 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component webservice-cli [tools]
10:14 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission [tools]
10:02 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission [tools]
09:29 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-36 [tools]
09:24 <filippo@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-36 [tools]
09:23 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-10 [tools]
09:18 <filippo@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-10 [tools]
09:16 <filippo@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-worker-nfs-10,tools-k8s-worker-nfs-36 [tools]
09:16 <filippo@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-10,tools-k8s-worker-nfs-36 [tools]
2026-07-29 §
09:54 <godog> test dumps-nfs failover - T432325 [tools]
09:47 <godog> enable 'dumps_use_nfs_lb: true' for nfs worker prefix - T432325 [tools]
09:45 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers (T433184) [tools]
09:45 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers (T433184) [tools]
09:40 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers (T433184) [tools]
09:40 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers (T433184) [tools]
08:56 <godog> enable 'dumps_use_nfs_lb: true' for bastion/static/mail prefixes - T432325 [tools]
2026-07-28 §
08:25 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-32, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-26, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-23, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-22, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-33, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-34, t [tools]
06:56 <godog> resume nfs workers roll restart - T432325 [tools]
06:55 <filippo@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-32, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-26, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-23, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-22, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-33, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-34, tools-k8s-worker-nfs [tools]
2026-07-27 §
17:18 <dcaro> increase object storage object number quota to 1M (T433281) [tools]
17:12 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api [tools]
16:59 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [tools]
12:06 <filippo@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for all NFS workers (T432325) [tools]
11:39 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission (T432325) [tools]
11:37 <filippo@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component volume-admission (T432325) [tools]
08:11 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0) [tools]
2026-07-23 §
17:11 <dcaro> resize haproxy-8 [tools]
16:56 <dcaro@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.vm_console [tools]
16:55 <dcaro> resize haproxy-7 [tools]
16:50 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) by 32 cores, 65536 ram [tools]
16:50 <dcaro@cloudcumin1001> START - Cookbook wmcs.openstack.quota_increase by 32 cores, 65536 ram [tools]