1-50 of 10000 results (30ms)
2026-07-08 ยง
18:00 <bking@cumin2003> START - Cookbook sre.hosts.reimage for host cirrussearch1102.eqiad.wmnet with OS trixie [production]
17:55 <bking@cumin2003> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cirrussearch1102.eqiad.wmnet with OS trixie [production]
17:48 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'cloudinfra-db' [cloudinfra]
17:46 <bking@cumin2003> START - Cookbook sre.hosts.reimage for host cirrussearch1102.eqiad.wmnet with OS trixie [production]
17:43 <taavi@cloudcumin1001> START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'cloudinfra-db' [cloudinfra]
17:43 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance cloudinfra-db-3 [cloudinfra]
17:42 <kamila@deploy2003> Started scap sync-world: Test deployment to validate deployment server switchover - T423714 [production]
17:42 <kamila@deploy2003> Finished scap sync-world: Test deployment to validate deployment server switchover - T423714 (duration: 19m 50s) [production]
17:42 <taavi@cloudcumin1001> START - Cookbook wmcs.vps.remove_instance for instance cloudinfra-db-3 [cloudinfra]
17:42 <kamila@deploy2003> Rolling back deployment [production]
17:41 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'cloudinfra-db' [cloudinfra]
17:36 <taavi@cloudcumin1001> START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'cloudinfra-db' [cloudinfra]
17:35 <bking@cumin2003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch1102.eqiad.wmnet with OS trixie [production]
17:31 <kamila@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host deploy2003.codfw.wmnet [production]
17:21 <taavi> move syslogaudit2.svc to syslog-server-audit-4 T402007 [cloudinfra]
17:21 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [admin]
17:21 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [admin]
17:20 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/326 [admin]
17:20 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/326 [admin]
17:18 <kamila@cumin1003> START - Cookbook sre.hosts.reboot-single for host deploy2003.codfw.wmnet [production]
17:16 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'syslog-server-audit' [cloudinfra]
17:16 <kamila@deploy1003> Unlocked for deployment [MediaWiki]: switching deployment server (duration: 22m 07s) [production]
17:15 <bking@cumin2003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch1102.eqiad.wmnet with reason: host reimage [production]
17:11 <kamila@dns1005> END - running authdns-update [production]
17:10 <taavi@cloudcumin1001> START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'syslog-server-audit' [cloudinfra]
17:09 <kamila@dns1005> START - running authdns-update [production]
17:07 <bking@cumin2003> START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch1102.eqiad.wmnet with reason: host reimage [production]
17:06 <taavi> move syslogaudit1.svc to syslog-server-audit-3 T402007 [cloudinfra]
17:05 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [admin]
17:04 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [admin]
17:04 <jasmine@cumin2002> END (FAIL) - Cookbook sre.k8s.renumber-node (exit_code=1) Renumbering for host wikikube-worker1164.eqiad.wmnet [production]
17:04 <jasmine@cumin2002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1164.eqiad.wmnet [production]
17:04 <jasmine@cumin2002> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1164.eqiad.wmnet [production]
17:03 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/323 [admin]
17:03 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/323 [admin]
16:58 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/322 [admin]
16:58 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/322 [admin]
16:57 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [admin]
16:56 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [admin]
16:56 <taavi> converting syslogaudit svc names to CNAMEs for easier manageability [cloudinfra]
16:56 <kamila@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on releases2003.codfw.wmnet,releases1003.eqiad.wmnet with reason: Deployment server switchover [production]
16:55 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
16:55 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
16:54 <kamila@deploy1003> Locking from deployment [MediaWiki]: switching deployment server [production]
16:53 <kamila@deploy1003> Unlocked for deployment [MediaWiki]: switching deployment server (duration: 04m 02s) [production]
16:53 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
16:53 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
16:52 <bking@cumin2003> START - Cookbook sre.hosts.reimage for host cirrussearch1102.eqiad.wmnet with OS trixie [production]
16:49 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/321 [admin]
16:49 <kamila@deploy1003> Locking from deployment [MediaWiki]: switching deployment server [production]