1-50 of 10000 results (53ms)
2026-09-21 ยง
16:01 <cklimas@deploy1003> helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply [production]
16:00 <cklimas@deploy1003> helmfile [codfw] START helmfile.d/services/wikifeeds: apply [production]
16:00 <cklimas@deploy1003> helmfile [eqiad] DONE helmfile.d/services/wikifeeds: apply [production]
16:00 <cklimas@deploy1003> helmfile [eqiad] START helmfile.d/services/wikifeeds: apply [production]
16:00 <elukey@cumin1004> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host registry2004.codfw.wmnet with OS trixie [production]
15:55 <cklimas@deploy1003> helmfile [staging] DONE helmfile.d/services/wikifeeds: apply [production]
15:54 <cklimas@deploy1003> helmfile [staging] START helmfile.d/services/wikifeeds: apply [production]
15:49 <fceratto@deploy1003> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
15:44 <jdlrobson@deploy1003> Finished scap sync-world: Backport for [[gerrit:1343579|Fixes: '.action_context' should be string (T437122)]] (duration: 12m 40s) [production]
15:42 <elukey@cumin1004> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on registry2004.codfw.wmnet with reason: host reimage [production]
15:39 <jdlrobson@deploy1003> jdlrobson: Continuing with deployment [production]
15:39 <cdobbins@cumin1004> START - Cookbook sre.hosts.reimage for host ncredir6002.drmrs.wmnet with OS trixie [production]
15:38 <elukey@cumin1004> START - Cookbook sre.hosts.downtime for 2:00:00 on registry2004.codfw.wmnet with reason: host reimage [production]
15:36 <jdlrobson@deploy1003> jdlrobson: Backport for [[gerrit:1343579|Fixes: '.action_context' should be string (T437122)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
15:33 <slyngshede@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-api-ext: apply [production]
15:32 <slyngshede@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-api-ext: apply [production]
15:32 <jdlrobson@deploy1003> Started scap sync-world: Backport for [[gerrit:1343579|Fixes: '.action_context' should be string (T437122)]] [production]
15:19 <elukey@puppetserver1001> conftool action : set/pooled=false; selector: name=registry2004.* [production]
15:18 <elukey@cumin1004> START - Cookbook sre.hosts.reimage for host registry2004.codfw.wmnet with OS trixie [production]
15:16 <cdobbins@cumin1004> conftool action : set/pooled=yes; selector: name=ncredir3006.* [production]
15:11 <fceratto@deploy1003> helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . [production]
15:07 <slyngshede@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-web: apply [production]
15:07 <slyngshede@deploy1003> helmfile [codfw] START helmfile.d/services/mw-web: apply [production]
15:03 <cdobbins@cumin1004> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ncredir3006.esams.wmnet with OS trixie [production]
15:01 <slyngshede@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-api-ext: apply [production]
15:01 <slyngshede@deploy1003> helmfile [codfw] START helmfile.d/services/mw-api-ext: apply [production]
14:47 <elukey@cumin1004> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host ml-serve1016.eqiad.wmnet with OS trixie [production]
14:39 <cdobbins@cumin1004> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ncredir3006.esams.wmnet with reason: host reimage [production]
14:36 <elukey@cumin1004> START - Cookbook sre.hosts.reimage for host ml-serve1016.eqiad.wmnet with OS trixie [production]
14:34 <cdobbins@cumin1004> START - Cookbook sre.hosts.downtime for 2:00:00 on ncredir3006.esams.wmnet with reason: host reimage [production]
14:26 <elukey@cumin1004> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
14:20 <elukey@cumin1004> START - Cookbook sre.hosts.provision for host ml-serve1016.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
14:13 <zabe@deploy1003> Finished scap sync-world: Backport for [[gerrit:1343542|Move wbc_entity_usage to x1 for mediawikiwiki (T438716)]], [[gerrit:1343556|Set db explicitly to false for virtual-wikibase-entityusage]] (duration: 08m 09s) [production]
14:08 <zabe@deploy1003> zabe: Continuing with deployment [production]
14:08 <zabe@deploy1003> zabe: Backport for [[gerrit:1343542|Move wbc_entity_usage to x1 for mediawikiwiki (T438716)]], [[gerrit:1343556|Set db explicitly to false for virtual-wikibase-entityusage]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
14:07 <cdobbins@cumin1004> START - Cookbook sre.hosts.reimage for host ncredir3006.esams.wmnet with OS trixie [production]
14:05 <zabe@deploy1003> Started scap sync-world: Backport for [[gerrit:1343542|Move wbc_entity_usage to x1 for mediawikiwiki (T438716)]], [[gerrit:1343556|Set db explicitly to false for virtual-wikibase-entityusage]] [production]
14:01 <zabe@deploy1003> Started scap sync-world: Backport for [[gerrit:1343542|Move wbc_entity_usage to x1 for mediawikiwiki (T438716)]], [[gerrit:1343556|Set db explicitly to false for virtual-wikibase-entityusage]] [production]
13:55 <dreamyjazz@deploy1003> Finished scap sync-world: Backport for [[gerrit:1337580|nlwiki: enable SecurePoll local elections (T434045)]] (duration: 12m 30s) [production]
13:51 <dreamyjazz@deploy1003> dreamyjazz, novemlinguae: Continuing with deployment [production]
13:47 <dreamyjazz@deploy1003> dreamyjazz, novemlinguae: Backport for [[gerrit:1337580|nlwiki: enable SecurePoll local elections (T434045)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
13:45 <cmooney@cumin1004> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:45 <cmooney@cumin1004> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add entries for new eqiad links - cmooney@cumin1004" [production]
13:45 <cmooney@cumin1004> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add entries for new eqiad links - cmooney@cumin1004" [production]
13:43 <zabe> reconcile wbc_entity_usage from local cluster to x1 for mediawikiwiki # T438716 [production]
13:43 <dreamyjazz@deploy1003> Started scap sync-world: Backport for [[gerrit:1337580|nlwiki: enable SecurePoll local elections (T434045)]] [production]
13:41 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/webrequest-pageview: apply [production]
13:41 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/webrequest-pageview: apply [production]
13:41 <cmooney@cumin1004> START - Cookbook sre.dns.netbox [production]
13:40 <samtar@deploy1003> Finished scap sync-world: Backport for [[gerrit:1343319|arywiki: Create patroller and autopatrolled user groups (T438421)]] (duration: 11m 40s) [production]