3351-3400 of 10000 results (36ms)
2023-11-11 §
00:48 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
00:47 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) [admin]
00:46 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
00:46 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) [admin]
00:46 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
00:46 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) [admin]
00:46 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
00:44 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) [admin]
00:44 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
00:41 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) [admin]
00:41 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
00:40 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) on host 'cloudvirt1058' [admin]
00:39 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain on host 'cloudvirt1058' [admin]
00:38 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) [admin]
00:37 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot [admin]
00:14 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1061.eqiad.wmnet with OS bookworm [production]
00:13 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
00:13 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
2023-11-10 §
23:51 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1061.eqiad.wmnet with reason: host reimage [production]
23:48 <andrew@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1061.eqiad.wmnet with reason: host reimage [production]
23:34 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1061.eqiad.wmnet with OS bookworm [production]
21:40 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
21:40 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
21:00 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1064.eqiad.wmnet with OS bookworm [production]
20:51 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1064.eqiad.wmnet with OS bookworm [production]
20:27 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
20:27 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
20:26 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
20:26 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
20:26 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
20:25 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
20:25 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
20:25 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1064.eqiad.wmnet with reason: host reimage [production]
20:25 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
20:24 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=99) [cloudvirt-canary]
20:24 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
20:22 <andrew@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1064.eqiad.wmnet with reason: host reimage [production]
20:04 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1064.eqiad.wmnet with OS bookworm [production]
18:53 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=99) [cloudvirt-canary]
18:53 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
18:47 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1063.eqiad.wmnet with OS bookworm [production]
18:04 <bvibber> brion adding more vp9 backfill to the transcode runs on mwmaint2002 (requeueTranscodes -> job queue runners). Should increase load on transcode scaler job runners but not elsewhere [production]
17:59 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=99) [cloudvirt-canary]
17:59 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
17:57 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
17:57 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
17:54 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1066.eqiad.wmnet with OS bookworm [production]
17:54 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
17:54 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
17:53 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1064.eqiad.wmnet with reason: host reimage [production]