1-50 of 10000 results (27ms)
2026-03-09 ยง
12:43 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component istio-gateway [toolsbeta]
12:43 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component istio-gateway [toolsbeta]
12:42 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a gateway role in the toolsbeta cluster [toolsbeta]
12:42 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a gateway role in the toolsbeta cluster [toolsbeta]
12:42 <taavi@cloudcumin1001> Added a new k8s gateway toolsbeta-test-k8s-gateway-2.toolsbeta.eqiad1.wikimedia.cloud to the cluster [toolsbeta]
12:32 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a gateway role in the toolsbeta cluster [toolsbeta]
12:32 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) by 65536 ram [toolsbeta]
12:32 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.quota_increase by 65536 ram [toolsbeta]
12:29 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a gateway role in the toolsbeta cluster [toolsbeta]
12:29 <moritzm> installing python3.9 security updates [production]
12:29 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a gateway role in the toolsbeta cluster [toolsbeta]
12:22 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a gateway role in the toolsbeta cluster [toolsbeta]
12:22 <taavi@cloudcumin1001> Added a new k8s gateway toolsbeta-test-k8s-gateway-1.toolsbeta.eqiad1.wikimedia.cloud to the cluster [toolsbeta]
12:12 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a gateway role in the toolsbeta cluster [toolsbeta]
12:11 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-test-k8s-gateway-1 [toolsbeta]
12:11 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host prometheus4003.ulsfo.wmnet with OS bookworm [production]
12:11 <taavi@cloudcumin1001> START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-test-k8s-gateway-1 [toolsbeta]
12:00 <reedy@deploy2002> Finished scap sync-world: Backport for [[gerrit:1239026|Revert "CommonSettings: Temporarily set $wgOATHUserHandlesTable = true" (T416544)]], [[gerrit:1249253|CommonSettings: Remove orphaned $wgWebAuthnNewCredsDisabled]] (duration: 06m 13s) [production]
11:56 <reedy@deploy2002> reedy: Continuing with sync [production]
11:56 <reedy@deploy2002> reedy: Backport for [[gerrit:1239026|Revert "CommonSettings: Temporarily set $wgOATHUserHandlesTable = true" (T416544)]], [[gerrit:1249253|CommonSettings: Remove orphaned $wgWebAuthnNewCredsDisabled]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
11:54 <reedy@deploy2002> Started scap sync-world: Backport for [[gerrit:1239026|Revert "CommonSettings: Temporarily set $wgOATHUserHandlesTable = true" (T416544)]], [[gerrit:1249253|CommonSettings: Remove orphaned $wgWebAuthnNewCredsDisabled]] [production]
11:44 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a gateway role in the toolsbeta cluster [toolsbeta]
11:44 <phuedx@deploy2002> Finished scap sync-world: Backport for [[gerrit:1249245|Hooks: Really only add global logging context for pageviews]] (duration: 12m 02s) [production]
11:38 <phuedx@deploy2002> phuedx: Continuing with sync [production]
11:35 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a gateway role in the toolsbeta cluster [toolsbeta]
11:34 <phuedx@deploy2002> phuedx: Backport for [[gerrit:1249245|Hooks: Really only add global logging context for pageviews]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
11:32 <phuedx@deploy2002> Started scap sync-world: Backport for [[gerrit:1249245|Hooks: Really only add global logging context for pageviews]] [production]
11:29 <btullis@deploy2002> helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-test: apply [production]
11:29 <btullis@deploy2002> helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-test: apply [production]
11:03 <brouberol@deploy2002> helmfile [aux-k8s-codfw] DONE helmfile.d/aux-k8s-services/kafka-mirrormaker: apply [production]
11:03 <wmftkbot> Test Kitchen edge-unique experiments (poll 230884) - adds: none; removes: none; fields: logged-out-retention-round3, logged-out-retention-round3 - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
11:03 <brouberol@deploy2002> helmfile [aux-k8s-codfw] START helmfile.d/aux-k8s-services/kafka-mirrormaker: apply [production]
10:59 <wmftkbot> Test Kitchen edge-unique experiments (poll 230871) - adds: logged-out-retention-round3; removes: none; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
10:57 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host prometheus4003.ulsfo.wmnet with OS bookworm [production]
10:56 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host prometheus4003.ulsfo.wmnet with OS bookworm [production]
10:50 <brouberol@deploy2002> helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/kafka-mirrormaker: apply [production]
10:49 <brouberol@deploy2002> helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/kafka-mirrormaker: apply [production]
10:45 <jmm@cumin2002> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host prometheus4003.ulsfo.wmnet [production]
10:45 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host prometheus4003.ulsfo.wmnet with OS bookworm [production]
10:45 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host prometheus4003.ulsfo.wmnet with OS bookworm [production]
10:44 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM prometheus4003.ulsfo.wmnet - jmm@cumin2002" [production]
10:44 <jmm@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM prometheus4003.ulsfo.wmnet - jmm@cumin2002" [production]
10:43 <jmm@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) prometheus4003.ulsfo.wmnet on all recursors [production]
10:43 <jmm@cumin2002> START - Cookbook sre.dns.wipe-cache prometheus4003.ulsfo.wmnet on all recursors [production]
10:43 <jmm@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
10:43 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM prometheus4003.ulsfo.wmnet - jmm@cumin2002" [production]
10:43 <jmm@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM prometheus4003.ulsfo.wmnet - jmm@cumin2002" [production]
10:40 <brouberol@deploy2002> helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/kafka-mirrormaker: apply [production]
10:40 <hashar> gerrit: mediawiki/vendor: converted `es6` and `es710` branches to tags # T417804 [releng]
10:39 <brouberol@deploy2002> helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/kafka-mirrormaker: apply [production]