7701-7750 of 7789 results (13ms)
2022-03-29 §
15:19 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.reboot (exit_code=99) [production]
15:19 <bking@cumin1001> START - Cookbook sre.wdqs.reboot [production]
2022-03-28 §
14:46 <inflatador> 'bking@cumin1001 repooling wdqs services in IAD ref T302494' [production]
14:46 <bking@cumin1001> conftool action : set/pooled=true; selector: dnsdisc=wdqs-internal,name=eqiad [production]
14:45 <bking@cumin1001> conftool action : set/pooled=true; selector: dnsdisc=wdqs*,name=eqiad [production]
2022-03-24 §
21:11 <inflatador> bking@cumin1001 restarting blazegraph on wdqs[1003-1013].eqiad.wmnet for T293862 [production]
18:05 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.restart (exit_code=99) [production]
17:44 <bking@cumin1001> START - Cookbook sre.wdqs.restart [production]
17:36 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.restart (exit_code=99) [production]
17:36 <bking@cumin1001> START - Cookbook sre.wdqs.restart [production]
2022-03-23 §
21:42 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic ES 6.8 upgrade - bking@cumin1001 - T301956 [production]
20:18 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic ES 6.8 upgrade - bking@cumin1001 - T301956 [production]
20:01 <bking@cumin1001> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic ES 6.8 upgrade - bking@cumin1001 - T301956 [production]
19:53 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic ES 6.8 upgrade - bking@cumin1001 - T301956 [production]
19:23 <bking@cumin1001> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic ES 6.8 upgrade - bking@cumin1001 - T301956 [production]
19:10 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic ES 6.8 upgrade - bking@cumin1001 - T301956 [production]
14:50 <bking@cumin1001> END (PASS) - Cookbook sre.wdqs.reboot (exit_code=0) [production]
14:28 <bking@cumin1001> START - Cookbook sre.wdqs.reboot [production]
14:27 <bking@cumin1001> END (PASS) - Cookbook sre.wdqs.reboot (exit_code=0) [production]
14:20 <bking@cumin1001> START - Cookbook sre.wdqs.reboot [production]
14:19 <bking@cumin1001> conftool action : set/pooled=yes; selector: name=wcqs1002.eqiad.wmnet [production]
14:11 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.reboot (exit_code=99) [production]
14:04 <bking@cumin1001> START - Cookbook sre.wdqs.reboot [production]
14:04 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.reboot (exit_code=99) [production]
14:04 <bking@cumin1001> START - Cookbook sre.wdqs.reboot [production]
14:04 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.reboot (exit_code=99) [production]
14:04 <bking@cumin1001> START - Cookbook sre.wdqs.reboot [production]
14:00 <bking@cumin1001> END (PASS) - Cookbook sre.wdqs.reboot (exit_code=0) [production]
13:57 <bking@cumin1001> START - Cookbook sre.wdqs.reboot [production]
2022-03-21 §
22:26 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
21:45 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
2022-03-15 §
21:56 <bking@cumin1001> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
21:55 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
14:22 <inflatador> T303256 bking@cumin1001 restarting wdqs services `sudo -E cumin -b 4 'A:wdqs-all' 'systemctl restart wdqs-blazegraph` [production]
2022-03-14 §
22:03 <bking@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=wdqs-internal,name=eqiad [production]
22:03 <bking@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=wdqs,name=eqiad [production]
22:03 <inflatador> T302494 bking@puppetmaster1001 depooling eqiad in DNS-discovery for wdqs and wdqs-internal services [production]
21:38 <inflatador> T302494 bking@puppetmaster1001 conftool action : set/pooled=true; selector: dnsdisc=wdqs-internal,name=codfw [production]
21:37 <bking@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=wdqs,name=codfw [production]
21:36 <inflatador> bking@cumin pooling codfw in DNS-discovery for wdqs and wdqs-internal services [production]
2022-03-10 §
22:08 <bking@cumin1001> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
22:05 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
22:04 <bking@cumin1001> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
22:04 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
22:02 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
22:02 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
2022-03-08 §
16:53 <inflatador> bking@deneb manually installed tox for T293862 . moritzm will add puppet patch for this [production]
16:02 <inflatador> bking@deneb manually installed openjdk-11-jdk for T293862 . moritzm will add puppet patch for this [production]
2022-02-09 §
23:18 <bking@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts elastic[1032-1038,1040-1042,1044-1047].eqiad.wmnet [production]
22:50 <bking@cumin1001> START - Cookbook sre.hosts.decommission for hosts elastic[1032-1038,1040-1042,1044-1047].eqiad.wmnet [production]