4851-4900 of 8386 results (11ms)
2024-03-05 §
21:47 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
21:30 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
21:27 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['wdqs1025'] [production]
21:20 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['elastic2107'] [production]
21:20 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2107'] [production]
21:17 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['wdqs1025'] [production]
21:17 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
21:03 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
21:02 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
19:47 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
19:46 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['wdqs1025'] [production]
19:46 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['wdqs1025'] [production]
19:44 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
18:28 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
18:28 <bking@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wdqs1025 [production]
18:28 <bking@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host wdqs1025 [production]
18:26 <bking@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:25 <bking@cumin2002> START - Cookbook sre.dns.netbox [production]
17:40 <inflatador> bking@prometheus1006 reload prometheus service as part of troubleshooting T358029 [production]
2024-03-04 §
22:19 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host elastic2108.codfw.wmnet with OS bullseye [production]
22:19 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host elastic2107.codfw.wmnet with OS bullseye [production]
21:14 <inflatador> bking@cumin2002 depool wdqs2007 for T355873 [production]
20:58 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2108.codfw.wmnet with OS bullseye [production]
20:58 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2107.codfw.wmnet with OS bullseye [production]
20:41 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host elastic2105.codfw.wmnet with OS bullseye [production]
20:38 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host elastic2104.codfw.wmnet with OS bullseye [production]
20:18 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on elastic2105.codfw.wmnet with reason: host reimage [production]
20:15 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on elastic2104.codfw.wmnet with reason: host reimage [production]
20:14 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on elastic2105.codfw.wmnet with reason: host reimage [production]
20:12 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on elastic2104.codfw.wmnet with reason: host reimage [production]
19:58 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2105.codfw.wmnet with OS bullseye [production]
19:56 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2104.codfw.wmnet with OS bullseye [production]
2024-03-01 §
22:48 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host elastic2109.codfw.wmnet with OS bullseye [production]
22:32 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on elastic2109.codfw.wmnet with reason: host reimage [production]
22:27 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on elastic2109.codfw.wmnet with reason: host reimage [production]
22:11 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2109.codfw.wmnet with OS bullseye [production]
20:45 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2109.codfw.wmnet with OS bullseye [production]
2024-02-29 §
19:58 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in cloudelastic [production]
19:58 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Unbanning all hosts in cloudelastic [production]
16:12 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in search_codfw [production]
16:12 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Unbanning all hosts in search_codfw [production]
2024-02-28 §
22:14 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic2043*,2044*,2079*,2080* for switch maintenance - bking@cumin2002 - T355872 [production]
22:13 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: elastic2043*,2044*,2079*,2080* for switch maintenance - bking@cumin2002 - T355872 [production]
14:27 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on wdqs2008.codfw.wmnet with reason: T355617 [production]
14:27 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 6:00:00 on wdqs2008.codfw.wmnet with reason: T355617 [production]
00:06 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 15:00:00 on wdqs1011.eqiad.wmnet with reason: T355617 [production]
00:06 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 15:00:00 on wdqs1011.eqiad.wmnet with reason: T355617 [production]
2024-02-22 §
18:22 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in search_codfw [production]
18:22 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Unbanning all hosts in search_codfw [production]
2024-02-21 §
22:02 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic2041*,elastic2042*,elastic2057*,elastic2063*,elastic2064*,elastic2077*,elastic2078*,elastic2092*,elastic2093*,elastic2094* for switch maintenance - bking@cumin2002 - T355860 [production]