2251-2300 of 10000 results (25ms)
2023-12-04 §
11:40 <elukey@deploy2002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
11:39 <elukey@deploy2002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
09:36 <elukey> upgrade istio (buster -> bullseye) on ml-serve-codfw - T351933 [production]
08:58 <elukey> upgrade istio (buster -> bullseye) on ml-serve-eqiad - T351933 [production]
08:48 <elukey> upgrade istio (buster -> bullseye) on aux-k8s-eqiad - T351933 [production]
08:43 <elukey> upgrade istio (buster -> bullseye) on dse-k8s-eqiad - T351933 [production]
2023-11-29 §
16:13 <elukey> restart pyrra-filesystem on titan* [production]
16:13 <elukey> reload all thanos-rule daemons on titan* to pick up new Pyrra Lift Wing rules [production]
14:15 <elukey> reload thanos-rule on titan[12]001 to pick up new pyrra rec rules [production]
2023-11-27 §
10:58 <elukey> powercycle ml-serve2007 (OEM/DIMM error registered in getsel) [production]
2023-11-22 §
10:43 <elukey@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:ml-cache-codfw: Roll restart after change in the CA bundle - elukey@cumin1001 [production]
10:25 <elukey@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching A:ml-cache-codfw: Roll restart after change in the CA bundle - elukey@cumin1001 [production]
10:25 <elukey@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:ml-cache-eqiad: Roll restart after change in the CA bundle - elukey@cumin1001 [production]
10:07 <elukey@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching A:ml-cache-eqiad: Roll restart after change in the CA bundle - elukey@cumin1001 [production]
09:47 <elukey> Update of the profile::base::certificate's CA bundle fleet wide (https://gerrit.wikimedia.org/r/c/operations/puppet/+/976659) [production]
09:40 <elukey@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: kafka::main [production]
09:34 <elukey@cumin1001> START - Cookbook sre.puppet.migrate-role for role: kafka::main [production]
2023-11-21 §
16:11 <elukey@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host kafka-main1001.eqiad.wmnet [production]
16:07 <elukey@cumin1001> START - Cookbook sre.puppet.migrate-host for host kafka-main1001.eqiad.wmnet [production]
2023-11-20 §
08:10 <elukey@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: sync [production]
08:09 <elukey@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: sync [production]
08:09 <elukey@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop: sync [production]
08:09 <elukey@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop: sync [production]
2023-11-17 §
14:20 <elukey@cumin1001> END (FAIL) - Cookbook sre.puppet.migrate-host (exit_code=99) for host ml-serve1001.eqiad.wmnet [production]
14:20 <elukey@cumin1001> START - Cookbook sre.puppet.migrate-host for host ml-serve1001.eqiad.wmnet [production]
2023-11-16 §
17:07 <elukey@deploy2002> helmfile [codfw] DONE helmfile.d/services/changeprop: sync [production]
17:07 <elukey@deploy2002> helmfile [codfw] START helmfile.d/services/changeprop: sync [production]
16:52 <elukey@deploy2002> helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: sync [production]
16:51 <elukey@deploy2002> helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: sync [production]
16:50 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/changeprop-jobqueue: sync [production]
16:50 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/changeprop-jobqueue: sync [production]
2023-11-15 §
16:25 <elukey> reload thanos-rule on titan[12]001 to pick up new pyrra generated configs [production]
2023-11-14 §
17:03 <elukey@deploy2002> helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: sync [production]
17:02 <elukey@deploy2002> helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: sync [production]
16:47 <elukey@deploy2002> helmfile [codfw] DONE helmfile.d/services/changeprop: sync [production]
16:47 <elukey@deploy2002> helmfile [codfw] START helmfile.d/services/changeprop: sync [production]
15:23 <elukey@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: sync [production]
15:22 <elukey@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: sync [production]
14:46 <elukey@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop: sync [production]
14:46 <elukey@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop: sync [production]
13:43 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/changeprop-jobqueue: sync [production]
13:43 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/changeprop-jobqueue: sync [production]
13:42 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/changeprop: sync [production]
13:42 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/changeprop: sync [production]
09:26 <elukey@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: sync [production]
09:25 <elukey@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: sync [production]
2023-10-31 §
16:49 <elukey@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop: sync [production]
16:49 <elukey@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop: sync [production]
16:35 <elukey@deploy2002> helmfile [codfw] DONE helmfile.d/services/changeprop: sync [production]
16:34 <elukey@deploy2002> helmfile [codfw] START helmfile.d/services/changeprop: sync [production]