5751-5800 of 8776 results (25ms)
2024-01-17 §
21:15 <inflatador> bking@kafka-main1001 `kafka topics --alter --topic eqiad.cirrussearch.update_pipeline.update.rc0 --partitions 5` T354595 [production]
21:13 <inflatador> bking@kafka-main1001 `kafka topics --alter --topic codfw.cirrussearch.update_pipeline.update.rc0 --partitions 5` [production]
2024-01-09 §
14:26 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.restart (exit_code=99) [production]
14:26 <bking@cumin2002> START - Cookbook sre.wdqs.restart [production]
2024-01-03 §
23:33 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) [production]
23:01 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) [production]
22:40 <bking@cumin2002> START - Cookbook sre.wdqs.restart [production]
22:36 <bking@cumin2002> START - Cookbook sre.wdqs.restart [production]
22:20 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host elastic2087.codfw.wmnet with OS bullseye [production]
22:03 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on elastic2087.codfw.wmnet with reason: host reimage [production]
21:59 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on elastic2087.codfw.wmnet with reason: host reimage [production]
21:43 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2087.codfw.wmnet with OS bullseye [production]
2024-01-02 §
22:29 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host elastic2087.codfw.wmnet with OS bullseye [production]
21:08 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host elastic2087.codfw.wmnet with OS bullseye [production]
2023-12-21 §
14:09 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 18 days, 0:00:00 on 10 hosts with reason: T352878 [production]
14:09 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 18 days, 0:00:00 on 10 hosts with reason: T352878 [production]
14:08 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 18 days, 0:00:00 on 13 hosts with reason: T352878 [production]
14:07 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 18 days, 0:00:00 on 13 hosts with reason: T352878 [production]
2023-12-20 §
22:59 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for wdqs[1020-1021].eqiad.wmnet [production]
22:59 <bking@cumin2002> START - Cookbook sre.hosts.remove-downtime for wdqs[1020-1021].eqiad.wmnet [production]
22:58 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 18 days, 0:00:00 on wdqs[1020-1024].eqiad.wmnet with reason: T352878 [production]
22:58 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 18 days, 0:00:00 on wdqs[1020-1024].eqiad.wmnet with reason: T352878 [production]
19:27 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reboot-single (exit_code=97) for host wdqs1022.eqiad.wmnet [production]
17:26 <bking@cumin2002> START - Cookbook sre.hosts.reboot-single for host wdqs1022.eqiad.wmnet [production]
17:25 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
17:25 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
15:14 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
15:09 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts wdqs1024.eqiad.wmnet [production]
15:09 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host wdqs1024.eqiad.wmnet [production]
15:06 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
15:05 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
15:05 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs1022.eqiad.wmnet [production]
15:05 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts wdqs1023.eqiad.wmnet [production]
15:05 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs1023.eqiad.wmnet [production]
15:02 <bking@cumin2002> START - Cookbook sre.hosts.reboot-single for host wdqs1024.eqiad.wmnet [production]
15:02 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs1024.eqiad.wmnet [production]
15:01 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts wdqs1024.eqiad.wmnet [production]
15:01 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts wdqs1024.eqiad.wmnet [production]
14:58 <inflatador> bking@cumin2002 disable/mask wmf_auto_restart_prometheus-blazegraph-exporter-wdqs-categories on wdqs102[24] T352878 [production]
2023-12-18 §
17:14 <inflatador> bking@kafka-jumbo1007 kafka topics --alter --topic codfw.mediawiki.cirrussearch.page_rerender.v1 --partitions 5 T351503 [production]
17:12 <inflatador> bking@kafka-jumbo1007 kafka topics --alter --topic eqiad.mediawiki.cirrussearch.page_rerender.v1 --partitions 5 T351503 [production]
2023-12-13 §
23:17 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_codfw: apply new extra plugins - bking@cumin2002 - T353270 [production]
21:59 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_codfw: apply new extra plugins - bking@cumin2002 - T353270 [production]
19:03 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: apply new extra plugins - bking@cumin2002 - T353270 [production]
17:44 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: apply new extra plugins - bking@cumin2002 - T353270 [production]
16:39 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: apply new extra plugins - bking@cumin2002 - T353270 [production]
16:14 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: apply new extra plugins - bking@cumin2002 - T353270 [production]
2023-12-12 §
19:59 <bking@cumin2002> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: enable new wmf-elasticsearch-search-plugins - bking@cumin2002 - T353270 [production]
19:18 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: enable new wmf-elasticsearch-search-plugins - bking@cumin2002 - T353270 [production]
19:03 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: enable new wmf-elasticsearch-search-plugins - bking@cumin2002 - T353270 [production]