1601-1650 of 10000 results (18ms)
2024-12-13 §
07:59 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
07:58 <jelto@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2125.codfw.wmnet [production]
07:58 <jelto@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2126.codfw.wmnet [production]
07:57 <jelto@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2126.codfw.wmnet [production]
07:49 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]
07:41 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
07:30 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]
2024-12-12 §
22:36 <tgr|away> UTC late deploys done [production]
22:35 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1102959|change metric types back to counters (T374050)]] (duration: 19m 10s) [production]
22:30 <tgr@deploy2002> tgr, cwhite: Continuing with sync [production]
22:29 <eileen> config revision changed from ca701cba to 404bbbd5 [production]
22:20 <tgr@deploy2002> tgr, cwhite: Backport for [[gerrit:1102959|change metric types back to counters (T374050)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
22:16 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1102959|change metric types back to counters (T374050)]] [production]
22:14 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1102885|EditCheck: move checks to a sidebar (T341308 T379443)]] (duration: 29m 12s) [production]
22:13 <wmbot~lucaswerkmeister@tools-bastion-13> deployed 5ffdfb2c55 (l10n updates: he, nl) [tools.lexeme-forms]
22:09 <kamila@cumin1002> END (PASS) - Cookbook sre.k8s.roll-reimage-nodes (exit_code=0) rolling reimage on P{wikikube-worker127[6-7].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) [production]
22:09 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1277.eqiad.wmnet with OS bookworm [production]
22:08 <inflatador> bking@cumin2002 sudo cumin A:gitlab-runner 'systemctl restart ferm.service' T371994 [production]
22:03 <tgr@deploy2002> tgr, kemayo: Continuing with sync [production]
22:02 <tgr@deploy2002> tgr, kemayo: Backport for [[gerrit:1102885|EditCheck: move checks to a sidebar (T341308 T379443)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:49 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1277.eqiad.wmnet with reason: host reimage [production]
21:46 <kamila@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1277.eqiad.wmnet with reason: host reimage [production]
21:45 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1102885|EditCheck: move checks to a sidebar (T341308 T379443)]] [production]
21:38 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1102905|Reader Survey: Deploy on eswiki, dewiki and frwiki. (T378660)]] (duration: 12m 42s) [production]
21:35 <inflatador> bking@gitlab-runner2004 restart ferm to troubleshoot missing iptables rules T371994 [production]
21:32 <tgr@deploy2002> dani, tgr: Continuing with sync [production]
21:32 <inflatador> bking@gitlab-runner2004 restart docker to troubleshoot missing iptables rules T371994 [production]
21:32 <tgr@deploy2002> dani, tgr: Backport for [[gerrit:1102905|Reader Survey: Deploy on eswiki, dewiki and frwiki. (T378660)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:25 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1277 [production]
21:25 <kamila@cumin1002> START - Cookbook sre.hosts.move-vlan for host wikikube-worker1277 [production]
21:25 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1102905|Reader Survey: Deploy on eswiki, dewiki and frwiki. (T378660)]] [production]
21:25 <kamila@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1277.eqiad.wmnet with OS bookworm [production]
21:25 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1276.eqiad.wmnet with OS bookworm [production]
21:20 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1275.eqiad.wmnet with OS bookworm [production]
21:05 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1276.eqiad.wmnet with reason: host reimage [production]
21:02 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1275.eqiad.wmnet with reason: host reimage [production]
21:00 <kamila@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1276.eqiad.wmnet with reason: host reimage [production]
20:56 <kamila@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1275.eqiad.wmnet with reason: host reimage [production]
20:40 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1276 [production]
20:40 <kamila@cumin1002> START - Cookbook sre.hosts.move-vlan for host wikikube-worker1276 [production]
20:40 <kamila@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1276.eqiad.wmnet with OS bookworm [production]
20:38 <kamila@cumin1002> START - Cookbook sre.k8s.roll-reimage-nodes rolling reimage on P{wikikube-worker127[6-7].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) [production]
20:37 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1275 [production]
20:37 <kamila@cumin1002> START - Cookbook sre.hosts.move-vlan for host wikikube-worker1275 [production]
20:37 <kamila@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1275.eqiad.wmnet with OS bookworm [production]
20:32 <kamila@cumin1002> END (FAIL) - Cookbook sre.k8s.roll-reimage-nodes (exit_code=1) rolling reimage on P{wikikube-worker[1270-1275].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) [production]
20:32 <kamila@cumin1002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wikikube-worker1275.eqiad.wmnet with OS bookworm [production]
19:48 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1275 [production]
19:48 <kamila@cumin1002> START - Cookbook sre.hosts.move-vlan for host wikikube-worker1275 [production]
19:48 <kamila@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1275.eqiad.wmnet with OS bookworm [production]