1-50 of 7766 results (17ms)
2025-08-21 §
21:49 <bking@cumin1002> conftool action : set/pooled=true; selector: dnsdisc=search,name=eqiad [production]
21:26 <bking@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 55 hosts with reason: T400160 [production]
21:25 <bking@cumin1002> DONE (ERROR) - Cookbook sre.hosts.downtime (exit_code=97) for 2:00:00 on 55 hosts with reason: T395571 [production]
21:25 <bking@cumin1002> conftool action : set/pooled=false; selector: dnsdisc=search,name=eqiad [production]
19:04 <bking@cumin1002> conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2113\.codfw\.wmnet [production]
19:04 <bking@cumin1002> conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2091\.codfw\.wmnet [production]
18:56 <bking@cumin1002> conftool action : set/weight=10; selector: name=cirrussearch2091. [production]
18:55 <bking@cumin1002> conftool action : set/weight=10; selector: name=cirrussearch2113. [production]
18:55 <bking@cumin1002> conftool action : set/weight=10; selector: name=cirrussearch2113. [production]
14:33 <bking@cumin1002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in search_eqiad [production]
14:33 <bking@cumin1002> START - Cookbook sre.elasticsearch.ban Unbanning all hosts in search_eqiad [production]
2025-08-19 §
19:47 <bking@cumin1002> conftool action : set/pooled=true; selector: dnsdisc=search,name=eqiad [production]
19:44 <bking@cumin1002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: apply logging config change - bking@cumin1002 - T395571 [production]
19:38 <bking@cumin1002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: apply logging config change - bking@cumin1002 - T395571 [production]
19:32 <bking@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 55 hosts with reason: T395571 [production]
19:30 <bking@cumin1002> conftool action : set/pooled=false; selector: dnsdisc=search,name=eqiad [production]
2025-08-18 §
21:58 <bking@cumin1002> conftool action : set/pooled=true; selector: dnsdisc=search,name=eqiad [production]
21:35 <bking@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 55 hosts with reason: T395571 [production]
21:33 <bking@cumin1002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: apply logging config change - bking@cumin1002 - T395571 [production]
19:24 <bking@cumin1002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: apply logging config change - bking@cumin1002 - T395571 [production]
19:22 <bking@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on 55 hosts with reason: T395571 [production]
19:15 <bking@cumin1002> conftool action : set/pooled=false; selector: dnsdisc=search,name=eqiad [production]
18:33 <bking@cumin1002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_codfw: apply logging config change - bking@cumin1002 - T395571 [production]
16:50 <bking@cumin1002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_codfw: apply logging config change - bking@cumin1002 - T395571 [production]
16:49 <bking@cumin1002> conftool action : set/weight=10; selector: name=cirrussearch2091. [production]
16:42 <bking@cumin1002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_codfw: apply logging config change - bking@cumin1002 - T395571 [production]
16:40 <bking@cumin1002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_codfw: apply logging config change - bking@cumin1002 - T395571 [production]
16:29 <bking@cumin1002> END (PASS) - Cookbook sre.opensearch.roll-restart-reboot (exit_code=0) rolling restart_daemons on A:datahubsearch [production]
16:21 <bking@cumin1002> START - Cookbook sre.opensearch.roll-restart-reboot rolling restart_daemons on A:datahubsearch [production]
16:01 <bking@cumin1002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: apply logging config change - bking@cumin1002 - T395571 [production]
15:32 <bking@cumin1002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: apply logging config change - bking@cumin1002 - T395571 [production]
15:31 <bking@cumin1002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: apply logging config change - bking@cumin1002 - T395571 [production]
15:24 <bking@cumin1002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: apply logging config change - bking@cumin1002 - T395571 [production]
2025-08-12 §
21:54 <bking@cumin2002> conftool action : set/pooled=true; selector: dnsdisc=search,name=eqiad [production]
21:14 <bking@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 55 hosts with reason: investigate cluster quorum failure [production]
20:57 <bking@cumin2002> conftool action : set/pooled=false; selector: dnsdisc=search,name=eqiad [production]
2025-08-07 §
22:15 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs1016.eqiad.wmnet w/ force delete existing files, repooling both afterwards [production]
22:09 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs2007.codfw.wmnet -> wdqs2018.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
21:15 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs2007.codfw.wmnet -> wdqs2018.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
21:15 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs1016.eqiad.wmnet w/ force delete existing files, repooling both afterwards [production]
18:24 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs1015.eqiad.wmnet w/ force delete existing files, repooling both afterwards [production]
18:23 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs2007.codfw.wmnet -> wdqs2022.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
17:27 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs2007.codfw.wmnet -> wdqs2022.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
17:26 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs1015.eqiad.wmnet w/ force delete existing files, repooling both afterwards [production]
2025-08-06 §
21:57 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs1014.eqiad.wmnet w/ force delete existing files, repooling both afterwards [production]
21:50 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs2007.codfw.wmnet -> wdqs2015.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
20:59 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs1014.eqiad.wmnet w/ force delete existing files, repooling both afterwards [production]
20:53 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs2007.codfw.wmnet -> wdqs2015.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
18:17 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs1013.eqiad.wmnet w/ force delete existing files, repooling both afterwards [production]
18:13 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs2007.codfw.wmnet -> wdqs2014.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]