301-350 of 10000 results (111ms)
2026-03-30 §
07:45 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host bast4006.wikimedia.org with OS trixie [production]
07:45 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host bast4006.wikimedia.org with OS trixie [production]
07:42 <javiermonton@deploy1003> Started scap sync-world: Backport for [[gerrit:1261377|stream: mediawiki.page_html_content_change (T421341)]] [production]
07:38 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host bast4006.wikimedia.org with OS trixie [production]
07:24 <tappof> prometheus7002: switch to nftables and reboot (T419960) [production]
07:18 <tappof> prometheus6002: switch to nftables and reboot (T419960) [production]
07:11 <tappof> prometheus5002: switch to nftables and reboot (T419960) [production]
07:08 <tappof> prometheus4003: reboot (T419960) [production]
07:05 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host bast4006.wikimedia.org with OS trixie [production]
07:04 <tappof> prometheus3004: switch to nftables and reboot (T419960) [production]
05:16 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on clouddb1022.eqiad.wmnet with reason: Downgrade clouddb1022 to 10.11.13 [production]
05:14 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on clouddb1022.eqiad.wmnet with reason: Downgrade clouddb1022 to 10.11.13 [production]
02:07 <mwpresync@deploy1003> Finished scap build-images: Publishing wmf/next image (duration: 06m 50s) [production]
02:00 <mwpresync@deploy1003> Started scap build-images: Publishing wmf/next image [production]
2026-03-29 §
02:07 <mwpresync@deploy1003> Finished scap build-images: Publishing wmf/next image (duration: 06m 13s) [production]
02:00 <mwpresync@deploy1003> Started scap build-images: Publishing wmf/next image [production]
2026-03-28 §
14:48 <dzahn@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on zuul2002.codfw.wmnet with reason: T421398 [production]
14:48 <dzahn@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on zuul1002.eqiad.wmnet with reason: T421398 [production]
14:16 <mutante> releases1003 - re-enabled puppet which was disabled due to T418109 but should not have been disabled during switch of the deployment server; leading to T421532 [production]
2026-03-27 §
18:11 <jhathaway@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:00 <jhathaway@cumin2002> START - Cookbook sre.hosts.provision for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:50 <jhathaway@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:40 <jhathaway@cumin2002> START - Cookbook sre.hosts.provision for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:39 <tchin@deploy1003> helmfile [codfw] DONE helmfile.d/services/eventgate-analytics-external: apply [production]
17:39 <jhathaway@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:39 <tchin@deploy1003> helmfile [codfw] START helmfile.d/services/eventgate-analytics-external: apply [production]
17:38 <tchin@deploy1003> helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics-external: apply [production]
17:37 <tchin@deploy1003> helmfile [eqiad] START helmfile.d/services/eventgate-analytics-external: apply [production]
17:37 <tchin@deploy1003> helmfile [staging] DONE helmfile.d/services/eventgate-analytics-external: apply [production]
17:35 <kevinbazira@deploy1003> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
17:34 <tchin@deploy1003> helmfile [staging] START helmfile.d/services/eventgate-analytics-external: apply [production]
17:34 <jhathaway@cumin2002> START - Cookbook sre.hosts.provision for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:30 <jhathaway@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:30 <jhathaway@cumin2002> START - Cookbook sre.hosts.provision for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:24 <jhathaway@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:19 <jhathaway@cumin2002> START - Cookbook sre.hosts.provision for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:15 <jhathaway@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
17:04 <jhathaway@cumin2002> START - Cookbook sre.hosts.provision for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
16:55 <jhathaway@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
16:50 <jhathaway@cumin2002> START - Cookbook sre.hosts.provision for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
16:47 <jhathaway@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
16:42 <dancy@deploy1003> Finished deploy [releng/jenkins-deploy@31ace7e] (releasing): (no justification provided) (duration: 01m 18s) [production]
16:41 <dancy@deploy1003> Started deploy [releng/jenkins-deploy@31ace7e] (releasing): (no justification provided) [production]
16:37 <jhathaway@cumin2002> START - Cookbook sre.hosts.provision for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
16:36 <jhathaway@cumin2002> END (ERROR) - Cookbook sre.hosts.provision (exit_code=97) for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
16:27 <jhathaway@cumin2002> START - Cookbook sre.hosts.provision for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
16:22 <jhathaway@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
16:13 <brouberol@deploy1003> helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. [production]
16:12 <brouberol@deploy1003> helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. [production]
16:12 <jhathaway@cumin2002> START - Cookbook sre.hosts.provision for host wikikube-ctrl2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]