1351-1400 of 10000 results (3ms)
2026-05-14 §
13:53 <bking@deploy1003> helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. [production]
2026-05-13 §
20:43 <bking@deploy1003> helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search: apply [production]
20:43 <bking@deploy1003> helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-semantic-search: apply [production]
20:43 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search: apply [production]
20:43 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search: apply [production]
20:43 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search: apply [production]
20:42 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search: apply [production]
20:41 <bking@deploy1003> helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. [production]
20:41 <bking@deploy1003> helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. [production]
20:25 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search: apply [production]
20:25 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search: apply [production]
20:21 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search: apply [production]
20:21 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search: apply [production]
20:17 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
20:17 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
2026-05-12 §
17:36 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-toolhub: apply [production]
17:35 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-toolhub: apply [production]
14:45 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-toolhub-test: apply [production]
14:44 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-toolhub-test: apply [production]
2026-05-11 §
19:39 <inflatador> [bking@cumin2002] ~$ sudo cumin 'A:wdqs-main and A:codfw' 'systemctl restart wdqs-blazegraph' <- restart after banning scraper [production]
19:11 <inflatador> bking@archiva1002 `sudo rm -rfv /var/cache/archiva/temp* && sudo systemctl restart archiva`. to free up disk space [production]
2026-05-06 §
21:27 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudelastic1012.eqiad.wmnet with OS trixie [production]
20:29 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudelastic1012.eqiad.wmnet with reason: host reimage [production]
20:25 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudelastic1012.eqiad.wmnet with reason: host reimage [production]
20:14 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1012.eqiad.wmnet with OS trixie [production]
20:00 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1012.eqiad.wmnet with OS trixie [production]
19:57 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1012.eqiad.wmnet with OS trixie [production]
19:24 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudelastic1012.eqiad.wmnet with OS trixie [production]
19:05 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudelastic1012.eqiad.wmnet with reason: host reimage [production]
19:01 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudelastic1012.eqiad.wmnet with reason: host reimage [production]
18:49 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1012.eqiad.wmnet with OS trixie [production]
17:59 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: restart to test fixes from T425301 - bking@cumin2002 [production]
13:53 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: restart to test fixes from T425301 - bking@cumin2002 [production]
2026-05-04 §
19:40 <bking@cumin2002> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: remove privatemounts to see if it helps - bking@cumin2002 - T424852 [production]
19:37 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: remove privatemounts to see if it helps - bking@cumin2002 - T424852 [production]
19:28 <bking@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on 6 hosts with reason: ongoing troubleshooting [production]
19:23 <bking@cumin2002> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: remove privatemounts to see if it helps - bking@cumin2002 - T424852 [production]
19:23 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: remove privatemounts to see if it helps - bking@cumin2002 - T424852 [production]
2026-04-29 §
14:41 <bking@cumin2002> conftool action : set/pooled=yes; selector: name=cloudelastic1007.eqiad.wmnet [production]
14:40 <bking@cumin2002> conftool action : set/pooled/yes; selector: dc=eqiad,cluster=cloudelastic,name=cloudelastic1007. [production]
14:40 <bking@cumin2002> conftool action : set/weight=10; selector: name=cloudelastic1007. [production]
14:37 <inflatador> bking@cloudelastic1010 run smartctl against all physical disks T424852 [production]
14:23 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudelastic1007.eqiad.wmnet with OS trixie [production]
13:59 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudelastic1007.eqiad.wmnet with reason: host reimage [production]
13:52 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudelastic1007.eqiad.wmnet with reason: host reimage [production]
13:32 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1007.eqiad.wmnet with OS trixie [production]
2026-04-28 §
21:51 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudelastic1009.eqiad.wmnet with OS trixie [production]
21:20 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudelastic1009.eqiad.wmnet with reason: host reimage [production]
21:16 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudelastic1009.eqiad.wmnet with reason: host reimage [production]
20:56 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1009.eqiad.wmnet with OS trixie [production]