101-150 of 10000 results (22ms)
2026-04-01 ยง
17:21 <swfrench@deploy1003> helmfile [codfw] START helmfile.d/services/mw-cron: apply [production]
17:18 <swfrench@deploy1003> Finished scap sync-world: helmfile-only deployment to remove unused image-suggestion listener - T368096 (duration: 07m 25s) [production]
17:15 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-cli [toolsbeta]
17:15 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli [toolsbeta]
17:14 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-cli [toolsbeta]
17:14 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli [toolsbeta]
17:14 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-cli [toolsbeta]
17:14 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli [toolsbeta]
17:12 <swfrench@deploy1003> Started scap sync-world: helmfile-only deployment to remove unused image-suggestion listener - T368096 [production]
17:09 <fabfur@cumin1003> END (PASS) - Cookbook sre.cdn.roll-restart-haproxy (exit_code=0) rolling restart of HAProxy on A:cp-drmrs - New configuration/test (T421402) [production]
16:53 <dreamyjazz@deploy1003> Finished scap sync-world: Backport for [[gerrit:1266317|hCaptcha: Add log and counter when all SiteVerify attempts fail (T421678)]], [[gerrit:1266316|hCaptcha: Add log and counter when all SiteVerify attempts fail (T421678)]] (duration: 11m 30s) [production]
16:49 <dreamyjazz@deploy1003> dreamyjazz: Continuing with sync [production]
16:44 <dreamyjazz@deploy1003> dreamyjazz: Backport for [[gerrit:1266317|hCaptcha: Add log and counter when all SiteVerify attempts fail (T421678)]], [[gerrit:1266316|hCaptcha: Add log and counter when all SiteVerify attempts fail (T421678)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
16:42 <dreamyjazz@deploy1003> Started scap sync-world: Backport for [[gerrit:1266317|hCaptcha: Add log and counter when all SiteVerify attempts fail (T421678)]], [[gerrit:1266316|hCaptcha: Add log and counter when all SiteVerify attempts fail (T421678)]] [production]
16:36 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T421714, prepare newly-reimaged host) xfer commons from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
16:33 <fabfur@cumin1003> START - Cookbook sre.cdn.roll-restart-haproxy rolling restart of HAProxy on A:cp-drmrs - New configuration/test (T421402) [production]
16:31 <dancy> Upgrade buildkit to 0.29.0 in staging gitlab-cloud-runners (T415284) [releng]
16:29 <urbanecm@deploy1003> Finished scap sync-world: Backport for [[gerrit:1266309|Set the default for UserEmailConfirmationUseHTML to true (T411147)]], [[gerrit:1260011|cleanup: Remove UserEmailConfirmationUseHTML (defaults to true) (T411147)]] (duration: 09m 31s) [production]
16:26 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
16:26 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
16:24 <urbanecm@deploy1003> urbanecm: Continuing with sync [production]
16:21 <urbanecm@deploy1003> urbanecm: Backport for [[gerrit:1266309|Set the default for UserEmailConfirmationUseHTML to true (T411147)]], [[gerrit:1260011|cleanup: Remove UserEmailConfirmationUseHTML (defaults to true) (T411147)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
16:19 <urbanecm@deploy1003> Started scap sync-world: Backport for [[gerrit:1266309|Set the default for UserEmailConfirmationUseHTML to true (T411147)]], [[gerrit:1260011|cleanup: Remove UserEmailConfirmationUseHTML (defaults to true) (T411147)]] [production]
16:15 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host conf2007.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
16:14 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host conf2007.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
16:14 <jhancock@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host conf2007 [production]
16:14 <jhancock@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host conf2007 [production]
16:13 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:13 <jhancock@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding conf2007 to codfw - jhancock@cumin2002" [production]
16:13 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding conf2007 to codfw - jhancock@cumin2002" [production]
16:09 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
15:32 <fabfur@cumin1003> END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on A:cp-text_ulsfo - 3.2 upgrade (T421402) [production]
15:27 <wmftkbot> Test Kitchen mw-user experiment (poll 55433) - adds: email_confirmation_banner_ab_test; removes: none; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
15:26 <fabfur@cumin1003> END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on A:cp-upload_ulsfo - 3.2 upgrade (T421402) [production]
15:26 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
15:26 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
15:24 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
15:24 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
15:24 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T421714, prepare newly-reimaged host) xfer commons from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
15:23 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T421714, prepare newly-reimaged host) xfer commons from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
15:23 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T421714, prepare newly-reimaged host) xfer commons from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
15:22 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T421714, prepare newly-reimaged host) xfer commons from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
15:22 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T421714, prepare newly-reimaged host) xfer commons from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
15:22 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T421714, prepare newly-reimaged host) xfer wikidata from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
15:20 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T421714, prepare newly-reimaged host) xfer wikidata from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
15:20 <wmftkbot> Test Kitchen mw-user experiment (poll 55412) - adds: none; removes: email_confirmation_banner_ab_test; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
15:13 <jforrester@deploy1003> Finished scap sync-world: Backport for [[gerrit:1266290|Wikifunctions: Switch cache from mcrouter-wikifunctions to special access (T411807)]] (duration: 12m 53s) [production]
15:12 <lucaswerkmeister-wmde@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-experimental: apply [production]
15:11 <lucaswerkmeister-wmde@deploy1003> helmfile [codfw] START helmfile.d/services/mw-experimental: apply [production]
15:11 <lucaswerkmeister-wmde@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply [production]