101-150 of 10000 results (27ms)
2026-03-09 ยง
18:24 <wmftkbot> Test Kitchen edge-unique experiments (poll 232088) - adds: mobile-toc-abc, logged-out-retention-round3; removes: message; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:24 <wmftkbot> Test Kitchen edge-unique experiments (poll 232087) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:23 <swfrench@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
18:23 <dzahn@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host contint1003.wikimedia.org with OS trixie [production]
18:23 <swfrench@deploy2002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
18:20 <wmftkbot> Test Kitchen edge-unique experiments (poll 232084) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:19 <wmftkbot> Test Kitchen edge-unique experiments (poll 232083) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:16 <swfrench@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
18:16 <swfrench@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
18:13 <wmftkbot> Test Kitchen edge-unique experiments (poll 232078) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:09 <wmftkbot> Test Kitchen edge-unique experiments (poll 232075) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:07 <wmftkbot> Test Kitchen edge-unique experiments (poll 232074) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:06 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dse-k8s-ctrl1001.eqiad.wmnet [production]
18:05 <herron@deploy2002> Sync cancelled. [production]
18:04 <wmftkbot> Test Kitchen edge-unique experiments (poll 232072) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:04 <herron@deploy2002> herron: Backport for [[gerrit:1249361|Revert "udp2log: switch to new hosts"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
18:03 <wmftkbot> Test Kitchen edge-unique experiments (poll 232071) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:02 <wmftkbot> Test Kitchen edge-unique experiments (poll 232070) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:02 <herron@deploy2002> Started scap sync-world: Backport for [[gerrit:1249361|Revert "udp2log: switch to new hosts"]] [production]
18:01 <bking@cumin2002> START - Cookbook sre.hosts.reboot-single for host dse-k8s-ctrl1001.eqiad.wmnet [production]
18:01 <wmftkbot> Test Kitchen edge-unique experiments (poll 232069) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
17:59 <wmftkbot> Test Kitchen edge-unique experiments (poll 232068) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
17:55 <wmftkbot> Test Kitchen edge-unique experiments (poll 232065) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
17:54 <fceratto@deploy2002> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
17:51 <wmftkbot> Test Kitchen edge-unique experiments (poll 232062) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
17:47 <fceratto@deploy2002> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
17:46 <wmftkbot> Test Kitchen edge-unique experiments (poll 232060) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
17:45 <wmftkbot> Test Kitchen edge-unique experiments (poll 232059) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
17:42 <fceratto@deploy2002> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
17:42 <wmftkbot> Test Kitchen edge-unique experiments (poll 232057) - adds: message; removes: mobile-toc-abc, logged-out-retention-round3; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
17:42 <herron@deploy2002> Sync cancelled. [production]
17:40 <fceratto@deploy2002> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
17:39 <fceratto@deploy2002> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
17:38 <mutante> contint1003 - unable to get uptime Caused by: Cumin execution failed (exit_code=2) [101/240] - attempted manual powercycle - Initializing Firmware Interfaces... blank screen T418544 [production]
17:37 <damilare> payments-wiki upgraded from 5b747b97 to f40a1153 [fundraising]
17:34 <mutante> contint1003.mgmt - racadm serveraction powercycle T418544 - not reacting [production]
17:25 <fceratto@deploy2002> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
17:25 <herron@deploy2002> herron: Backport for [[gerrit:1249332|udp2log: switch to new hosts (T417002)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
17:23 <herron@deploy2002> Started scap sync-world: Backport for [[gerrit:1249332|udp2log: switch to new hosts (T417002)]] [production]
17:19 <fceratto@deploy2002> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
17:15 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host netflow4003.ulsfo.wmnet [production]
17:15 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host netflow4003.ulsfo.wmnet with OS bookworm [production]
17:13 <fceratto@deploy2002> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
17:13 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway [tools]
17:08 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component api-gateway [tools]
17:08 <fceratto@deploy2002> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
17:07 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet [production]
17:04 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway [toolsbeta]
17:03 <dzahn@cumin2002> START - Cookbook sre.hosts.reimage for host contint1003.wikimedia.org with OS trixie [production]
17:01 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet [production]