5351-5400 of 8392 results (32ms)
2024-01-18 §
16:22 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2097.codfw.wmnet with OS bullseye [production]
16:18 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on elastic2090.codfw.wmnet with reason: host reimage [production]
16:18 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2096.codfw.wmnet with OS bullseye [production]
16:15 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on elastic2090.codfw.wmnet with reason: host reimage [production]
16:15 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2095.codfw.wmnet with OS bullseye [production]
16:12 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2094.codfw.wmnet with OS bullseye [production]
16:09 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2093.codfw.wmnet with OS bullseye [production]
16:06 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2092.codfw.wmnet with OS bullseye [production]
16:04 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2091.codfw.wmnet with OS bullseye [production]
15:57 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2090.codfw.wmnet with OS bullseye [production]
15:50 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2089.codfw.wmnet with OS bullseye [production]
15:45 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2088.codfw.wmnet with OS bullseye [production]
2024-01-17 §
22:01 <inflatador> bking@kafka-main2001 `kafka topics --alter --topic eqiad.cirrussearch.update_pipeline.fetch_error.rc0 --partitions 5` T354595 [production]
21:50 <inflatador> bking@kafka-main2001 `kafka topics --alter --topic codfw.cirrussearch.update_pipeline.fetch_error.rc0 --partitions 5` T354595 [production]
21:47 <inflatador> bking@kafka-main2001 `kafka topics --alter --topic eqiad.cirrussearch.update_pipeline.update.rc0 --partitions 5` T354595 [production]
21:16 <inflatador> bking@kafka-main1001 `kafka topics --alter --topic codfw.cirrussearch.update_pipeline.fetch_error.rc0 --partitions 5 [production]
21:15 <inflatador> bking@kafka-main1001 `kafka topics --alter --topic eqiad.cirrussearch.update_pipeline.update.rc0 --partitions 5` T354595 [production]
21:13 <inflatador> bking@kafka-main1001 `kafka topics --alter --topic codfw.cirrussearch.update_pipeline.update.rc0 --partitions 5` [production]
2024-01-09 §
14:26 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.restart (exit_code=99) [production]
14:26 <bking@cumin2002> START - Cookbook sre.wdqs.restart [production]
2024-01-03 §
23:33 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) [production]
23:01 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) [production]
22:40 <bking@cumin2002> START - Cookbook sre.wdqs.restart [production]
22:36 <bking@cumin2002> START - Cookbook sre.wdqs.restart [production]
22:20 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host elastic2087.codfw.wmnet with OS bullseye [production]
22:03 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on elastic2087.codfw.wmnet with reason: host reimage [production]
21:59 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on elastic2087.codfw.wmnet with reason: host reimage [production]
21:43 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2087.codfw.wmnet with OS bullseye [production]
2024-01-02 §
22:29 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host elastic2087.codfw.wmnet with OS bullseye [production]
21:08 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2087.codfw.wmnet with OS bullseye [production]
2023-12-21 §
14:09 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 18 days, 0:00:00 on 10 hosts with reason: T352878 [production]
14:09 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 18 days, 0:00:00 on 10 hosts with reason: T352878 [production]
14:08 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 18 days, 0:00:00 on 13 hosts with reason: T352878 [production]
14:07 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 18 days, 0:00:00 on 13 hosts with reason: T352878 [production]
2023-12-20 §
22:59 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for wdqs[1020-1021].eqiad.wmnet [production]
22:59 <bking@cumin2002> START - Cookbook sre.hosts.remove-downtime for wdqs[1020-1021].eqiad.wmnet [production]
22:58 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 18 days, 0:00:00 on wdqs[1020-1024].eqiad.wmnet with reason: T352878 [production]
22:58 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 18 days, 0:00:00 on wdqs[1020-1024].eqiad.wmnet with reason: T352878 [production]
19:27 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reboot-single (exit_code=97) for host wdqs1022.eqiad.wmnet [production]
17:26 <bking@cumin2002> START - Cookbook sre.hosts.reboot-single for host wdqs1022.eqiad.wmnet [production]
17:25 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
17:25 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
15:14 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
15:09 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts wdqs1024.eqiad.wmnet [production]
15:09 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host wdqs1024.eqiad.wmnet [production]
15:06 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
15:05 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
15:05 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
15:05 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts wdqs1023.eqiad.wmnet [production]
15:05 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs1023.eqiad.wmnet [production]