6801-6850 of 10000 results (120ms)
2024-11-14 ยง
12:29 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7002.magru.wmnet [production]
12:25 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7002.magru.wmnet [production]
12:22 <jayme@cumin2002> START - Cookbook sre.hosts.reimage for host wikikube-ctrl2003.codfw.wmnet with OS bookworm [production]
12:19 <jmm@cumin2002> END (PASS) - Cookbook sre.misc-clusters.roll-restart-reboot-eventschemas (exit_code=0) rolling restart_daemons on A:schema-codfw [production]
12:18 <jmm@cumin2002> START - Cookbook sre.misc-clusters.roll-restart-reboot-eventschemas rolling restart_daemons on A:schema-codfw [production]
12:17 <jayme@cumin2002> START - Cookbook sre.k8s.reimage-stacked-control-plane Reimaging k8s control planes of cluster wikikube-codfw: containerd migration [production]
12:10 <jmm@cumin2002> END (PASS) - Cookbook sre.cdn.roll-restart-reboot-ncredir (exit_code=0) rolling restart_daemons on A:ncredir [production]
12:00 <jmm@cumin2002> START - Cookbook sre.cdn.roll-restart-reboot-ncredir rolling restart_daemons on A:ncredir [production]
11:57 <moritzm> restarting postfix on inbound/outbound servers to pick up openssl updates [production]
11:17 <moritzm> installing openssl security updates [production]
11:08 <jayme@cumin2002> END (PASS) - Cookbook sre.k8s.reimage-stacked-control-plane (exit_code=0) Reimaging k8s control planes of cluster wikikube-codfw: containerd migration [production]
11:08 <jayme@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-ctrl2001.codfw.wmnet with OS bookworm [production]
10:47 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s_services/services/datahub: sync on production [production]
10:45 <jayme@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-ctrl2001.codfw.wmnet with reason: host reimage [production]
10:44 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s_services/services/datahub: apply on production [production]
10:42 <jayme@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-ctrl2001.codfw.wmnet with reason: host reimage [production]
10:16 <moritzm> remove ganeti2017 from active ganeti nodes T376594 [production]
10:15 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2017.codfw.wmnet [production]
10:11 <jayme@cumin2002> START - Cookbook sre.hosts.reimage for host wikikube-ctrl2001.codfw.wmnet with OS bookworm [production]
10:07 <jnuche@deploy2002> Finished deploy [releng/jenkins-deploy@34b35a5] (releasing): (no justification provided) (duration: 00m 47s) [production]
10:06 <jayme@cumin2002> START - Cookbook sre.k8s.reimage-stacked-control-plane Reimaging k8s control planes of cluster wikikube-codfw: containerd migration [production]
10:06 <jnuche@deploy2002> Started deploy [releng/jenkins-deploy@34b35a5] (releasing): (no justification provided) [production]
10:03 <jnuche@deploy2002> Finished deploy [releng/jenkins-deploy@34b35a5] (releasing): (no justification provided) (duration: 00m 21s) [production]
10:03 <jnuche@deploy2002> Started deploy [releng/jenkins-deploy@34b35a5] (releasing): (no justification provided) [production]
09:43 <kart_> Done: UTC morning backport window [production]
09:37 <kartik@deploy2002> Finished scap sync-world: Backport for [[gerrit:1090988|Correction to virtual-globaljsonlinks mapping (T374746)]] (duration: 10m 03s) [production]
09:37 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
09:36 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
09:35 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
09:34 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
09:32 <kartik@deploy2002> bvibber, kartik: Continuing with sync [production]
09:31 <kartik@deploy2002> bvibber, kartik: Backport for [[gerrit:1090988|Correction to virtual-globaljsonlinks mapping (T374746)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
09:27 <kartik@deploy2002> Started scap sync-world: Backport for [[gerrit:1090988|Correction to virtual-globaljsonlinks mapping (T374746)]] [production]
09:25 <kartik@deploy2002> Finished scap sync-world: Backport for [[gerrit:1091007|CX3 Build 0.2.0+20241113 (T368718 T374567)]] (duration: 29m 40s) [production]
09:21 <kartik@deploy2002> kartik: Continuing with sync [production]
09:17 <volans> installed spicerack v8.16.0 on cumin2002 [production]
09:08 <vgutierrez@cumin1002> END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on P{cp4044.ulsfo.wmnet,cp4052.ulsfo.wmnet} and A:cp [production]
09:04 <vgutierrez@cumin1002> START - Cookbook sre.cdn.roll-upgrade-haproxy rolling upgrade of HAProxy on P{cp4044.ulsfo.wmnet,cp4052.ulsfo.wmnet} and A:cp [production]
09:00 <kartik@deploy2002> kartik: Backport for [[gerrit:1091007|CX3 Build 0.2.0+20241113 (T368718 T374567)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
08:56 <kartik@deploy2002> Started scap sync-world: Backport for [[gerrit:1091007|CX3 Build 0.2.0+20241113 (T368718 T374567)]] [production]
08:55 <vgutierrez> import haproxy 2.8.12 to thirtdparty/haproxy28 component for bullseye-wikimedia (apt.wm.o) - T379891 [production]
08:54 <kartik@deploy2002> Finished scap sync-world: Backport for [[gerrit:1090937|Allow Wikidata bureaucrats to remove admin rights (T379635)]] (duration: 11m 49s) [production]
08:49 <kartik@deploy2002> dreamrimmer, kartik: Continuing with sync [production]
08:47 <kartik@deploy2002> dreamrimmer, kartik: Backport for [[gerrit:1090937|Allow Wikidata bureaucrats to remove admin rights (T379635)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
08:42 <kartik@deploy2002> Started scap sync-world: Backport for [[gerrit:1090937|Allow Wikidata bureaucrats to remove admin rights (T379635)]] [production]
08:38 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 26744 [production]
08:37 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 26744 [production]
08:35 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 141082 [production]
08:35 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 141082 [production]
08:34 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 9299 [production]