1851-1900 of 10000 results (18ms)
2025-12-04 §
15:06 <bking@deploy2002> helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply [production]
15:06 <bking@deploy2002> helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply [production]
2025-12-02 §
22:33 <bking@deploy2002> helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. [production]
22:32 <bking@deploy2002> helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. [production]
21:43 <bking@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply [production]
21:43 <bking@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply [production]
21:34 <bking@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply [production]
21:34 <bking@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply [production]
21:33 <bking@dns1004> END - running authdns-update [production]
21:32 <bking@dns1004> START - running authdns-update [production]
21:31 <bking@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply [production]
16:43 <inflatador> bking@wmf3062 restart WDQS codfw to resolve lag/possible deadlocks [production]
2025-12-01 §
22:50 <bking@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply [production]
15:50 <inflatador> bking@wmf3062 restart wdqs codfw for high lag https://docs.google.com/spreadsheets/d/1UaabYlqj37EEaLAkrRArn4yNuNviGObgsGTfquIIHAQ/edit?gid=0#gid=0 [production]
15:43 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1032.eqiad.wmnet with OS bookworm [production]
15:24 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1032.eqiad.wmnet with reason: host reimage [production]
15:19 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1032.eqiad.wmnet with reason: host reimage [production]
14:59 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1032.eqiad.wmnet with OS bookworm [production]
2025-11-24 §
23:30 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1029.eqiad.wmnet with OS trixie [production]
23:29 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1028.eqiad.wmnet with OS trixie [production]
22:34 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1032.eqiad.wmnet with OS trixie [production]
22:10 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1029.eqiad.wmnet with OS trixie [production]
22:09 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1028.eqiad.wmnet with OS trixie [production]
21:56 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1031.eqiad.wmnet with OS trixie [production]
21:38 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1031.eqiad.wmnet with reason: host reimage [production]
21:34 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1031.eqiad.wmnet with reason: host reimage [production]
21:16 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1031.eqiad.wmnet with OS trixie [production]
21:14 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1032.eqiad.wmnet with OS trixie [production]
20:55 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1030.eqiad.wmnet with OS trixie [production]
20:38 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1030.eqiad.wmnet with reason: host reimage [production]
20:32 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1030.eqiad.wmnet with reason: host reimage [production]
20:15 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1030.eqiad.wmnet with OS trixie [production]
20:14 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1030.eqiad.wmnet with OS trixie [production]
19:27 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1030.eqiad.wmnet with OS trixie [production]
18:21 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1032.eqiad.wmnet with OS trixie [production]
17:11 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1032.eqiad.wmnet with OS trixie [production]
17:10 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1030.eqiad.wmnet with OS trixie [production]
17:10 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1029.eqiad.wmnet with OS trixie [production]
17:09 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1028.eqiad.wmnet with OS trixie [production]
17:01 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1032.eqiad.wmnet with OS trixie [production]
17:00 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1031.eqiad.wmnet with OS trixie [production]
15:51 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1032.eqiad.wmnet with OS trixie [production]
15:51 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1031.eqiad.wmnet with OS trixie [production]
15:50 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1030.eqiad.wmnet with OS trixie [production]
15:50 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1029.eqiad.wmnet with OS trixie [production]
15:49 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wdqs1028.eqiad.wmnet with OS trixie [production]
15:46 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wdqs1032.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
15:40 <bking@cumin2002> START - Cookbook sre.hosts.provision for host wdqs1032.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
15:39 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wdqs1031.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
15:38 <bking@cumin2002> START - Cookbook sre.hosts.provision for host wdqs1031.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]