1-50 of 10000 results (98ms)
2026-05-01 §
13:45 <zabe@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply [production]
13:44 <zabe@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-experimental: apply [production]
13:24 <_Gerges> WikiMonitor setup [production]
13:09 <jclark@cumin1003> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080 [production]
13:09 <jclark@cumin1003> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078 [production]
13:09 <jclark@cumin1003> END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079 [production]
13:09 <jclark@cumin1003> END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077 [production]
13:09 <jclark@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080 [production]
13:09 <jclark@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079 [production]
13:09 <jclark@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078 [production]
13:08 <jclark@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077 [production]
13:08 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
13:08 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
13:07 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
13:07 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
13:06 <jclark@cumin1003> START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
13:06 <jclark@cumin1003> START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
13:06 <jclark@cumin1003> START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
13:06 <jclark@cumin1003> START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
13:04 <jclark@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:04 <jclark@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003" [production]
13:04 <jclark@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003" [production]
13:00 <jclark@cumin1003> START - Cookbook sre.dns.netbox [production]
12:34 <jgiannelos@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply [production]
12:34 <jgiannelos@deploy1003> helmfile [codfw] START helmfile.d/services/mw-parsoid: apply [production]
12:33 <jgiannelos@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply [production]
12:33 <jgiannelos@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply [production]
09:57 <samtar@deploy1003> Finished scap sync-world: Backport for [[gerrit:1281423|Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s) [production]
09:53 <samtar@deploy1003> samtar: Continuing with deployment [production]
09:52 <samtar@deploy1003> samtar: Backport for [[gerrit:1281423|Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
09:50 <samtar@deploy1003> Started scap sync-world: Backport for [[gerrit:1281423|Switch watchstar from Popover to Dialog (T417847)]] [production]
09:38 <urbanecm@deploy1003> Finished scap sync-world: Backport for [[gerrit:1281426|Update the interwiki cache (T239173)]] (duration: 06m 05s) [production]
09:32 <urbanecm@deploy1003> Started scap sync-world: Backport for [[gerrit:1281426|Update the interwiki cache (T239173)]] [production]
08:13 <cgoubert@deploy1003> helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply [production]
08:13 <cgoubert@deploy1003> helmfile [eqiad] START helmfile.d/services/rest-gateway: apply [production]
08:13 <cgoubert@deploy1003> helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply [production]
08:13 <cgoubert@deploy1003> helmfile [codfw] START helmfile.d/services/rest-gateway: apply [production]
08:13 <cgoubert@deploy1003> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
08:12 <cgoubert@deploy1003> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
03:26 <akhatun@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply [production]
03:26 <akhatun@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply [production]
02:07 <mwpresync@deploy1003> Finished scap build-images: Publishing wmf/next image (duration: 06m 41s) [production]
02:00 <mwpresync@deploy1003> Started scap build-images: Publishing wmf/next image [production]
00:16 <zabe@deploy1003> Finished scap sync-world: Backport for [[gerrit:1280417|Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s) [production]
00:13 <zabe@deploy1003> zabe: Continuing with deployment [production]
00:11 <zabe@deploy1003> zabe: Backport for [[gerrit:1280417|Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
00:09 <zabe@deploy1003> Started scap sync-world: Backport for [[gerrit:1280417|Add script to fix fr_deleted drifts (T424553)]] [production]
2026-04-30 §
23:51 <dzahn@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on zuul1001.eqiad.wmnet with reason: T421398 [production]
23:51 <dzahn@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on zuul2001.codfw.wmnet with reason: T421398 [production]
21:31 <cdobbins@cumin2002> conftool action : get/pooled; selector: name=cp4040.ulsfo.wmnet [production]