5201-5250 of 8776 results (23ms)
2024-03-21 §
21:00 <bking@cumin2002> conftool action : set/weight=10:pooled=yes; selector: name=elastic20[89-99]\.codfw\.wmnet [production]
20:37 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_codfw: introduce new masters - bking@cumin2002 - T353878 [production]
20:35 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_codfw: introduce new masters - bking@cumin2002 - T353878 [production]
2024-03-20 §
18:54 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host elastic2108.codfw.wmnet with OS bullseye [production]
18:37 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on elastic2108.codfw.wmnet with reason: host reimage [production]
18:35 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on elastic2108.codfw.wmnet with reason: host reimage [production]
18:19 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2108.codfw.wmnet with OS bullseye [production]
17:24 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host elastic2107.codfw.wmnet with OS bullseye [production]
17:07 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on elastic2107.codfw.wmnet with reason: host reimage [production]
17:04 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on elastic2107.codfw.wmnet with reason: host reimage [production]
16:48 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2107.codfw.wmnet with OS bullseye [production]
2024-03-07 §
22:47 <inflatador> bking@pcc-worker1006 deleted all dirs older than 22 Jan to free up space [production]
18:22 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 60 days, 0:00:00 on wdqs[1022-1025].eqiad.wmnet with reason: T337013 [production]
18:22 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 60 days, 0:00:00 on wdqs[1022-1025].eqiad.wmnet with reason: T337013 [production]
14:14 <bking@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "sync hiera as instructed by failed reimage cookbook - bking@cumin2002 - T358727" [production]
14:13 <bking@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "sync hiera as instructed by failed reimage cookbook - bking@cumin2002 - T358727" [production]
14:11 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
14:11 <bking@cumin2002> END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - bking@cumin2002" [production]
01:50 <bking@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - bking@cumin2002" [production]
01:31 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1025.eqiad.wmnet with reason: host reimage [production]
01:29 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1025.eqiad.wmnet with reason: host reimage [production]
00:46 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
00:37 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
2024-03-06 §
23:16 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
21:04 <bking@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wdqs1025 [production]
21:04 <bking@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host wdqs1025 [production]
20:20 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
19:00 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
18:59 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['wdqs1025'] [production]
18:59 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['wdqs1025'] [production]
18:59 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
17:53 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
15:31 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
14:18 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
2024-03-05 §
23:53 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
22:37 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 60 days, 0:00:00 on wdqs[1022-1024].eqiad.wmnet with reason: T337013 [production]
22:37 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 60 days, 0:00:00 on wdqs[1022-1024].eqiad.wmnet with reason: T337013 [production]
22:33 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
22:09 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['wdqs1025'] [production]
21:47 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['wdqs1025'] [production]
21:47 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
21:30 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
21:27 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['wdqs1025'] [production]
21:20 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['elastic2107'] [production]
21:20 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2107'] [production]
21:17 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['wdqs1025'] [production]
21:17 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
21:03 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]
21:02 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1025.eqiad.wmnet with OS bullseye [production]
19:47 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye [production]