6901-6950 of 10000 results (48ms)
2023-11-14 §
03:33 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1042.eqiad.wmnet with OS bookworm [production]
03:26 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) (T345811) [admin]
03:25 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1041.eqiad.wmnet with reason: host reimage [production]
03:22 <andrew@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1041.eqiad.wmnet with reason: host reimage [production]
03:16 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1040.eqiad.wmnet with reason: host reimage [production]
03:13 <andrew@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1040.eqiad.wmnet with reason: host reimage [production]
03:09 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1041.eqiad.wmnet with OS bookworm [production]
03:08 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1039.eqiad.wmnet with OS bookworm [production]
03:08 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain (T345811) [admin]
03:07 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
03:06 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
03:04 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=99) [cloudvirt-canary]
03:04 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
02:59 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1040.eqiad.wmnet with OS bookworm [production]
02:59 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) (T345811) [admin]
02:59 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) [admin]
02:59 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
02:59 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain (T345811) [admin]
02:55 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) [admin]
02:52 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) (T345811) [admin]
02:46 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
02:46 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1039.eqiad.wmnet with reason: host reimage [production]
02:44 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) [admin]
02:43 <andrew@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1039.eqiad.wmnet with reason: host reimage [production]
02:34 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain (T345811) [admin]
02:32 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
02:27 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) [admin]
02:26 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
02:26 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1039.eqiad.wmnet with OS bookworm [production]
02:23 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) (T345811) [admin]
02:17 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) [admin]
02:00 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain [admin]
02:00 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain (T345811) [admin]
01:58 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
01:58 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
01:57 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) [cloudvirt-canary]
01:56 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary [cloudvirt-canary]
00:33 <dzahn@cumin1001> END (FAIL) - Cookbook sre.puppet.migrate-host (exit_code=99) for host stewards1001.eqiad.wmnet [production]
00:31 <dzahn@cumin1001> START - Cookbook sre.puppet.migrate-host for host stewards1001.eqiad.wmnet [production]
00:27 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host stewards1001.eqiad.wmnet with OS bookworm [production]
00:14 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on stewards1001.eqiad.wmnet with reason: host reimage [production]
00:11 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on stewards1001.eqiad.wmnet with reason: host reimage [production]
00:03 <dzahn@cumin1001> START - Cookbook sre.hosts.reimage for host stewards1001.eqiad.wmnet with OS bookworm [production]
2023-11-13 §
23:57 <dzahn@cumin1001> END (FAIL) - Cookbook sre.puppet.migrate-host (exit_code=99) for host stewards1001.eqiad.wmnet [production]
23:57 <dzahn@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host stewards1001.eqiad.wmnet with OS bookworm [production]
23:41 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host clouddb1021.eqiad.wmnet [production]
23:34 <btullis> rebooting clouddb1021 to pick up new kernel and puppet 7 CA. [analytics]
23:33 <btullis@cumin1001> START - Cookbook sre.hosts.reboot-single for host clouddb1021.eqiad.wmnet [production]
23:24 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on stewards1001.eqiad.wmnet with reason: host reimage [production]
23:21 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on stewards1001.eqiad.wmnet with reason: host reimage [production]