1-50 of 10000 results (34ms)
2026-07-08 §
19:25 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan+apply for main branch [admin]
19:25 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [admin]
19:23 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for main branch [admin]
19:23 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for main branch [admin]
17:21 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [admin]
17:21 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [admin]
17:20 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/326 [admin]
17:20 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/326 [admin]
17:05 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [admin]
17:04 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [admin]
17:03 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/323 [admin]
17:03 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/323 [admin]
16:58 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/322 [admin]
16:58 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/322 [admin]
16:57 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [admin]
16:56 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [admin]
16:55 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
16:55 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
16:53 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
16:53 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
16:49 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
16:49 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
13:52 <andrewbogott> removing some cloudvirts from the 'maintenance' aggregate. I imagine they are there in error after some automated reboots. cloudvirt1049, cloudvirt1053, cloudvirt1064, cloudvirt1078, cloudvirt1080, cloudvirtlocal1001 [admin]
2026-07-07 §
09:14 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [admin]
09:13 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [admin]
09:07 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/319 [admin]
09:06 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/319 [admin]
04:27 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) on host 'cloudvirt1071.eqiad.wmnet' (T431374) [admin]
04:23 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain on host 'cloudvirt1071.eqiad.wmnet' (T431374) [admin]
04:18 <andrewbogott> cycling power on cloudvirt1071 via mgmt/racadm; it seems unresponsive [admin]
2026-07-06 §
08:04 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'P{P:openstack::codfw1dev::nova::compute::service} AND NOT P{F:kernelversion = 6.12.95}' [admin]
07:08 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'P{P:openstack::codfw1dev::nova::compute::service} AND NOT P{F:kernelversion = 6.12.95}' [admin]
2026-06-29 §
13:17 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment codfw1dev for all services [admin]
13:14 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [admin]
2026-06-25 §
06:51 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T424802) [admin]
06:51 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T424802) [admin]
2026-06-22 §
18:57 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.nfs.migrate_service (exit_code=99) [admin]
18:57 <andrew@cloudcumin1001> START - Cookbook wmcs.nfs.migrate_service [admin]
2026-06-17 §
13:57 <dhinus> updated wikireplicas-utils from 0.1.0 to 0.2.0 on clouddb* [admin]
2026-06-16 §
17:51 <andrewbogott> ceph tell osd.126,127,129,131 compact [admin]
17:04 <andrewbogott> rebooting cloudcephosd1037 and cloudcephosd1038 because of missing volumes. The volumes reappeared after boot. [admin]
16:47 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.upgrade_osds (exit_code=99) (T428385) [admin]
16:47 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.upgrade_osds (T428385) [admin]
16:47 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol1006.eqiad.wmnet' (T429361) [admin]
16:36 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudcontrol1006.eqiad.wmnet' (T429361) [admin]
16:35 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol1007.eqiad.wmnet' (T429361) [admin]
16:23 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudcontrol1007.eqiad.wmnet' (T429361) [admin]
16:22 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol1011.eqiad.wmnet' (T429361) [admin]
16:12 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudcontrol1011.eqiad.wmnet' (T429361) [admin]
16:01 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol2010-dev.codfw.wmnet' (T429361) [admin]