4101-4150 of 7785 results (27ms)
2024-06-11 §
17:56 <bking@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
17:56 <bking@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
17:45 <bking@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
17:45 <bking@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
17:04 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in search_eqiad [production]
17:04 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Unbanning all hosts in search_eqiad [production]
2024-06-10 §
14:43 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on elastic1107.eqiad.wmnet with reason: T365982 [production]
14:43 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on elastic1107.eqiad.wmnet with reason: T365982 [production]
13:57 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic1107.eqiad.wmnet for T348977 - bking@cumin2002 [production]
13:57 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1107.eqiad.wmnet for T348977 - bking@cumin2002 [production]
13:57 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.ban (exit_code=99) Banning hosts: elastic1107 for T348977 - bking@cumin2002 [production]
13:57 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1107 for T348977 - bking@cumin2002 [production]
2024-06-05 §
22:50 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.REBOOT (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin2002 - T366555 [production]
21:42 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.REBOOT (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin2002 - T366555 [production]
21:41 <bking@cumin2002> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin2002 - T366555 [production]
21:36 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin2002 - T366555 [production]
13:48 <inflatador> bking@an-db1001 install python3-psycopg2 pkg T363001 [production]
13:45 <inflatador> bking@an-db1001 install acl pkg T363001 [production]
2024-05-28 §
19:45 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in search_eqiad [production]
19:45 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Unbanning all hosts in search_eqiad [production]
19:01 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host elastic1056.eqiad.wmnet [production]
18:54 <bking@cumin2002> START - Cookbook sre.hosts.reboot-single for host elastic1056.eqiad.wmnet [production]
18:53 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on elastic1056.eqiad.wmnet with reason: rebooting after abnormally high load [production]
18:53 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 1:00:00 on elastic1056.eqiad.wmnet with reason: rebooting after abnormally high load [production]
17:25 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic1056.eqiad.wmnet for ban highly-loaded node - bking@cumin2002 [production]
17:25 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1056.eqiad.wmnet for ban highly-loaded node - bking@cumin2002 [production]
17:25 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.ban (exit_code=99) Banning hosts: elastic1056 for ban highly-loaded node - bking@cumin2002 [production]
17:25 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1056 for ban highly-loaded node - bking@cumin2002 [production]
2024-05-20 §
21:38 <bking@cumin2002> END (PASS) - Cookbook sre.zookeeper.roll-restart-zookeeper (exit_code=0) for Zookeeper A:zookeeper-flink-eqiad cluster: Roll restart of jvm daemons. [production]
21:32 <bking@cumin2002> START - Cookbook sre.zookeeper.roll-restart-zookeeper for Zookeeper A:zookeeper-flink-eqiad cluster: Roll restart of jvm daemons. [production]
21:29 <bking@cumin2002> END (PASS) - Cookbook sre.zookeeper.roll-restart-zookeeper (exit_code=0) for Zookeeper A:zookeeper-flink-codfw cluster: Roll restart of jvm daemons. [production]
21:22 <bking@cumin2002> START - Cookbook sre.zookeeper.roll-restart-zookeeper for Zookeeper A:zookeeper-flink-codfw cluster: Roll restart of jvm daemons. [production]
2024-05-13 §
22:55 <bking@cumin2002> conftool action : set/weight=10:pooled=yes; selector: name=elastic110[5|7]\.eqiad\.wmnet [production]
22:27 <bking@cumin2002> conftool action : set/weight=10:pooled=no; selector: name=elastic110[5|7]\.eqiad\.wmnet [production]
2024-05-01 §
14:12 <bking@deploy1002> helmfile [eqiad] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
14:11 <bking@deploy1002> helmfile [eqiad] START helmfile.d/services/rdf-streaming-updater: apply [production]
14:03 <bking@deploy1002> helmfile [codfw] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
14:03 <bking@deploy1002> helmfile [codfw] START helmfile.d/services/rdf-streaming-updater: apply [production]
2024-04-30 §
18:59 <bking@deploy1002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
18:55 <bking@deploy1002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
2024-04-26 §
17:27 <bking@cumin2002> conftool action : set/weight=10:pooled=yes; selector: name=elastic110[3-7]\.eqiad\.wmnet [production]
16:57 <bking@cumin2002> conftool action : set/pooled=no; selector: name=elastic1107\.eqiad\.wmnet [production]
16:36 <bking@cumin2002> conftool action : set/weight=20:pooled=yes; selector: name=elastic1107\.eqiad\.wmnet [production]
16:35 <bking@cumin2002> conftool action : set/pooled=no; selector: name=elastic1106\.eqiad\.wmnet [production]
16:33 <bking@cumin2002> conftool action : set/weight=20:pooled=yes; selector: name=elastic1106\.eqiad\.wmnet [production]
16:32 <bking@cumin2002> conftool action : set/pooled=no; selector: name=elastic1105\.eqiad\.wmnet [production]
15:56 <bking@cumin2002> conftool action : set/pooled=no; selector: name=elastic1104\.eqiad\.wmnet [production]
15:56 <bking@cumin2002> conftool action : set/pooled=no; selector: name=elastic1103\.eqiad\.wmnet [production]
15:43 <bking@cumin2002> conftool action : set/weight=20:pooled=yes; selector: name=elastic1105\.eqiad\.wmnet [production]
15:31 <bking@cumin2002> conftool action : set/weight=20:pooled=yes; selector: name=elastic1104\.eqiad\.wmnet [production]