1-50 of 10000 results (34ms)
2026-06-15 §
19:00 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.upgrade_osds (T428385) [admin]
18:40 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.upgrade_osds (exit_code=99) [admin]
18:27 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.upgrade_osds (T428385) [admin]
17:07 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.upgrade_osds (exit_code=99) [admin]
14:31 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.upgrade_osds (T428385) [admin]
14:10 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.upgrade_mons (exit_code=0) [admin]
13:50 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.upgrade_mons (T428385) [admin]
13:48 <andrew@cloudcumin1001> END (ERROR) - Cookbook wmcs.ceph.upgrade_osds (exit_code=97) [admin]
13:38 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.upgrade_osds (T428385) [admin]
13:08 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.upgrade_mons (exit_code=99) [admin]
12:46 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.upgrade_mons (T428385) [admin]
2026-06-12 §
14:00 <volans> clearing cloudback-original snapshots of cinder volumes created in 2026 to allow the backups to not fail for T428995 [admin]
2026-06-11 §
17:53 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol1011.eqiad.wmnet' (T428549) [admin]
17:39 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudcontrol1011.eqiad.wmnet' (T428549) [admin]
17:18 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol1007.eqiad.wmnet' (T428549) [admin]
17:05 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudcontrol1007.eqiad.wmnet' (T428549) [admin]
16:50 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol1006.eqiad.wmnet' (T428549) [admin]
16:37 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudcontrol1006.eqiad.wmnet' (T428549) [admin]
16:33 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=99) on host 'cloudcontrol2006.eqiad.wmnet' (T428549) [admin]
16:33 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudcontrol2006.eqiad.wmnet' (T428549) [admin]
15:11 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol2010-dev.codfw.wmnet' (T428549) [admin]
14:58 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudcontrol2010-dev.codfw.wmnet' (T428549) [admin]
14:41 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol2006-dev.codfw.wmnet' (T428549) [admin]
14:24 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudcontrol2006-dev.codfw.wmnet' (T428549) [admin]
14:22 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol2005-dev.codfw.wmnet' (T428549) [admin]
14:05 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudcontrol2005-dev.codfw.wmnet' (T428549) [admin]
2026-06-09 §
17:35 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.upgrade_osds (exit_code=0) [admin]
17:05 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.upgrade_osds (T428385) [admin]
16:59 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.upgrade_mons (exit_code=0) [admin]
16:35 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.upgrade_mons (T428385) [admin]
2026-06-07 §
17:55 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment eqiad1 for all services [admin]
17:40 <andrewbogott> wmcs.openstack.restart_openstack --cluster-name eqiad1 --all as step one in troubleshooting T428312 [admin]
17:39 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for all services [admin]
2026-06-02 §
20:31 <bd808> `kubectl sudo delete cm -n tool-arb-bot maintain-kubeusers-arb-bot` to trigger regeneration of .kube/config (IRC request) [admin]
2026-05-26 §
19:11 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.roll_reboot_osds (exit_code=0) [admin]
18:50 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.roll_reboot_osds [admin]
18:47 <andrew@cloudcumin1001> END (ERROR) - Cookbook wmcs.ceph.roll_reboot_mons (exit_code=97) [admin]
18:46 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.roll_reboot_mons [admin]
18:33 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.roll_reboot_mons (exit_code=0) [admin]
18:22 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.roll_reboot_mons [admin]
18:21 <andrew@cloudcumin1001> END (ERROR) - Cookbook wmcs.ceph.roll_reboot_mons (exit_code=97) [admin]
18:21 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.roll_reboot_mons [admin]
18:16 <andrew@cloudcumin1001> END (ERROR) - Cookbook wmcs.ceph.roll_reboot_mons (exit_code=97) [admin]
18:16 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.roll_reboot_mons [admin]
16:30 <andrew@cloudcumin1001> END (ERROR) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=97) on hosts matched by 'P{P:openstack::codfw1dev::nova::compute::service} AND NOT P{F:kernelversion = 6.12.88}' [admin]
16:30 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'P{P:openstack::codfw1dev::nova::compute::service} AND NOT P{F:kernelversion = 6.12.88}' [admin]
2026-05-25 §
09:04 <godog> move designate eqiad to zk backend [admin]
2026-05-21 §
18:38 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.roll_reboot_mons (exit_code=0) [admin]
18:27 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.roll_reboot_mons [admin]
18:17 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.roll_reboot_mons (exit_code=0) [admin]