1-50 of 10000 results (17ms)
2026-03-30 ยง
11:37 <hashar> Reloaded Zuul to to add 3 persons to the allow list [releng]
11:20 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment eqiad1 for all services [admin]
11:06 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for all services [admin]
11:06 <btullis@deploy1003> helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. [production]
11:05 <btullis@deploy1003> helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. [production]
11:05 <btullis@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
11:04 <btullis@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
11:01 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.rabbitmq.rebuild_rabbit_cluster (exit_code=0) on deployment eqiad1 [admin]
10:57 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.rabbitmq.rebuild_rabbit_cluster on deployment eqiad1 [admin]
10:43 <James_F> Docker: Re-pushing to try to create quibble-coverage 1.16.0-s2 [releng]
10:37 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host bast4006.wikimedia.org with OS bookworm [production]
10:36 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment eqiad1 for all services [admin]
10:21 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for all services [admin]
10:15 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on bast4006.wikimedia.org with reason: host reimage [production]
10:09 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on bast4006.wikimedia.org with reason: host reimage [production]
09:58 <filippo@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.rabbitmq.rebuild_rabbit_cluster (exit_code=99) on deployment eqiad1 [admin]
09:57 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.rabbitmq.rebuild_rabbit_cluster on deployment eqiad1 [admin]
09:53 <wmftkbot> Test Kitchen edge-unique experiments (poll 45906) - adds: synth-aa-test-traffic-impact-2, synth-aa-test-traffic-impact-1, synth-aa-test-traffic-impact-3; removes: none; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
09:46 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host bast4006.wikimedia.org with OS bookworm [production]
09:42 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment eqiad1 for service: project,designate [admin]
09:42 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host bast4006.wikimedia.org with OS trixie [production]
09:41 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for service: project,designate [admin]
09:30 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment eqiad1 for service: project,nova,neutron,designate [admin]
09:30 <filippo@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) on deployment eqiad1 for service: project,designate [admin]
09:28 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for service: project,designate [admin]
09:19 <ayounsi@cumin1003> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 42 [production]
09:18 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for service: project,nova,neutron,designate [admin]
09:18 <filippo@cloudcumin1001> END (ERROR) - Cookbook wmcs.openstack.restart_openstack (exit_code=97) on deployment eqiad1 for service: project,neutron,designate [admin]
09:17 <ayounsi@cumin1003> START - Cookbook sre.network.peering with action 'email' for AS: 42 [production]
09:15 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment codfw1dev for all services [admin]
09:15 <ayounsi@cumin1003> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 12200 [production]
09:14 <ayounsi@cumin1003> START - Cookbook sre.network.peering with action 'email' for AS: 12200 [production]
09:14 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for service: project,neutron,designate [admin]
09:14 <filippo@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment eqiad1 for service: project,heat [admin]
09:13 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for service: project,heat [admin]
09:12 <filippo@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) on deployment eqiad1 for all services [admin]
09:11 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for all services [admin]
09:11 <tappof> prometheus[12]008: reboot (T419960) [production]
09:11 <filippo@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [admin]
09:10 <tappof> prometheus[12]006: reboot (T419960) [production]
08:56 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host bast4006.wikimedia.org with OS trixie [production]
08:52 <XioNoX> push pfw policy - T421556 [production]
08:51 <tappof> prometheus[12]007: reboot (T419960) [production]
08:38 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
08:38 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
08:37 <tappof> prometheus[12]005: reboot (T419960) [production]
08:34 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host bast4006.wikimedia.org with OS trixie [production]
08:17 <javiermonton@deploy1003> Finished scap sync-world: Backport for [[gerrit:1261377|stream: mediawiki.page_html_content_change (T421341)]] (duration: 35m 10s) [production]
08:03 <javiermonton@deploy1003> javiermonton: Continuing with sync [production]
08:00 <javiermonton@deploy1003> javiermonton: Backport for [[gerrit:1261377|stream: mediawiki.page_html_content_change (T421341)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]