101-150 of 10000 results (3ms)
2026-08-31 ยง
10:24 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1008.eqiad.wmnet [production]
10:24 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1008.eqiad.wmnet [production]
10:19 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1008.eqiad.wmnet [production]
10:18 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1008.eqiad.wmnet [production]
10:18 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1007.eqiad.wmnet [production]
10:18 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1007.eqiad.wmnet [production]
10:15 <ozge@deploy1003> helmfile [codfw] DONE helmfile.d/services/linkrecommendation: apply [production]
10:14 <ozge@deploy1003> helmfile [codfw] START helmfile.d/services/linkrecommendation: apply [production]
10:13 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1007.eqiad.wmnet [production]
10:12 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1007.eqiad.wmnet [production]
10:12 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-worker1006.eqiad.wmnet [production]
10:12 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-worker1006.eqiad.wmnet [production]
10:12 <ozge@deploy1003> helmfile [eqiad] DONE helmfile.d/services/linkrecommendation: apply [production]
10:10 <ozge@deploy1003> helmfile [eqiad] START helmfile.d/services/linkrecommendation: apply [production]
10:07 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-worker1006.eqiad.wmnet [production]
10:07 <fabfur> enable puppet on A:cp to apply https://gerrit.wikimedia.org/r/c/operations/puppet/+/1330314 (T434766) [production]
10:07 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-worker1006.eqiad.wmnet [production]
10:07 <elukey@cumin1003> START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{aux-k8s-worker[1006-1009].eqiad.wmnet} and (A:aux-master-eqiad or A:aux-worker-eqiad) [production]
10:02 <ozge@deploy1003> helmfile [staging] DONE helmfile.d/services/linkrecommendation: apply [production]
10:01 <ozge@deploy1003> helmfile [staging] START helmfile.d/services/linkrecommendation: apply [production]
09:59 <fabfur> disabled puppet on A:cp to gradually apply https://gerrit.wikimedia.org/r/c/operations/puppet/+/1330314, starting from cp6016 [production]
09:47 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on A:aux-master-codfw [production]
09:47 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-ctrl2003.codfw.wmnet [production]
09:47 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-ctrl2003.codfw.wmnet [production]
09:42 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-ctrl2003.codfw.wmnet [production]
09:42 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-ctrl2003.codfw.wmnet [production]
09:42 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-ctrl2002.codfw.wmnet [production]
09:42 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-ctrl2002.codfw.wmnet [production]
09:37 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-ctrl2002.codfw.wmnet [production]
09:37 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-ctrl2002.codfw.wmnet [production]
09:37 <elukey@cumin1003> START - Cookbook sre.k8s.reboot-nodes rolling reboot on A:aux-master-codfw [production]
09:34 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on A:aux-master-eqiad [production]
09:34 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-ctrl1003.eqiad.wmnet [production]
09:34 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-ctrl1003.eqiad.wmnet [production]
09:30 <elukey@puppetserver1001> conftool action : set/pooled=true; selector: dnsdisc=pki,name=eqiad [production]
09:29 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-ctrl1003.eqiad.wmnet [production]
09:29 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-ctrl1003.eqiad.wmnet [production]
09:29 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host aux-k8s-ctrl1002.eqiad.wmnet [production]
09:29 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host aux-k8s-ctrl1002.eqiad.wmnet [production]
09:29 <elukey@cumin1003> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host pki1002.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL [production]
09:24 <elukey@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host aux-k8s-ctrl1002.eqiad.wmnet [production]
09:24 <elukey@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host aux-k8s-ctrl1002.eqiad.wmnet [production]
09:24 <elukey@cumin1003> START - Cookbook sre.k8s.reboot-nodes rolling reboot on A:aux-master-eqiad [production]
09:21 <elukey@cumin1003> START - Cookbook sre.hosts.provision for host pki1002.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL [production]
09:16 <elukey@cumin1003> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts pki1002.eqiad.wmnet [production]
09:16 <elukey@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host pki1002.eqiad.wmnet [production]
09:16 <elukey> move cx-server and citoid-beta endpoints in deployment-prep to two new Trixie VMs, update their configs and Docker images and delete the old images. [releng]
09:06 <elukey@cumin1003> START - Cookbook sre.hosts.reboot-single for host pki1002.eqiad.wmnet [production]
08:58 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on 22 hosts with reason: Cloning [production]
08:58 <marostegui> Stop mariadb on sanitarium s2,s4,s6,s7 there will be lag on wikireplicas for those sections T407942 [production]