|
2026-09-24
ยง
|
| 13:19 |
<filippo@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.set_maintenance (T437407) |
[admin] |
| 13:18 |
<mlitn@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1344617|Instrument five-arm image carousel retest (T431362)]], [[gerrit:1344619|Wire image carousel retest instrumentation (T431362)]], [[gerrit:1344627|ThumbExtractor: trim nbsp and dangling colons from caption text (T435672)]], [[gerrit:1344630|ThumbExtractor: exclude lead infobox images from the carousel (T438907)]] (duration: 12m 25s) |
[production] |
| 13:13 |
<mlitn@deploy1003> |
mfossati, mlitn: Continuing with deployment |
[production] |
| 13:10 |
<mlitn@deploy1003> |
mfossati, mlitn: Backport for [[gerrit:1344617|Instrument five-arm image carousel retest (T431362)]], [[gerrit:1344619|Wire image carousel retest instrumentation (T431362)]], [[gerrit:1344627|ThumbExtractor: trim nbsp and dangling colons from caption text (T435672)]], [[gerrit:1344630|ThumbExtractor: exclude lead infobox images from the carousel (T438907)]] synced to the testservers (see https://wiki |
[production] |
| 13:08 |
<cdobbins@cumin1004> |
START - Cookbook sre.hosts.reimage for host ncredir4004.ulsfo.wmnet with OS trixie |
[production] |
| 13:07 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 13:07 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 13:06 |
<mlitn@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1344617|Instrument five-arm image carousel retest (T431362)]], [[gerrit:1344619|Wire image carousel retest instrumentation (T431362)]], [[gerrit:1344627|ThumbExtractor: trim nbsp and dangling colons from caption text (T435672)]], [[gerrit:1344630|ThumbExtractor: exclude lead infobox images from the carousel (T438907)]] |
[production] |
| 13:06 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 13:06 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 13:02 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 13:02 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 13:02 |
<vgutierrez@cumin1004> |
START - Cookbook sre.cdn.roll-upgrade-haproxy rolling upgrade of HAProxy on A:cp-text_eqsin and A:cp - 3.2.23 upgrade (T438828) |
[production] |
| 13:01 |
<cdobbins@cumin1004> |
conftool action : set/pooled=yes; selector: name=cp2059.* |
[production] |
| 12:59 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 12:59 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 12:58 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 12:58 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 12:52 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 12:52 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 12:45 |
<filippo@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T435537) |
[admin] |
| 12:45 |
<filippo@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T435537) |
[admin] |
| 12:43 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 12:43 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply |
[production] |
| 12:34 |
<urbanecm@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply |
[production] |
| 12:34 |
<urbanecm@deploy1003> |
helmfile [eqiad] START helmfile.d/services/mw-experimental: apply |
[production] |
| 12:04 |
<btullis@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 12:03 |
<btullis@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 11:21 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on P{cp[5031,5032].*} and A:cp - 3.2.23 upgrade (T438828) |
[production] |
| 11:13 |
<hnowlan> |
restarted restbase on restbase2029 |
[production] |
| 11:04 |
<vgutierrez@cumin1004> |
START - Cookbook sre.cdn.roll-upgrade-haproxy rolling upgrade of HAProxy on P{cp[5031,5032].*} and A:cp - 3.2.23 upgrade (T438828) |
[production] |
| 11:01 |
<filippo@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) on host 'cloudvirt1080' (T435537) |
[admin] |
| 10:50 |
<hnowlan> |
deleting stuck mw-web pods in eqiad |
[production] |
| 10:45 |
<dreamyjazz@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1344621|AbuseReview: Let specific users and suppressors see vandalism tag (T438860)]] (duration: 10m 09s) |
[production] |
| 10:44 |
<sfaci@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/growthboo-next: apply |
[production] |
| 10:42 |
<vgutierrez@cumin1004> |
END (FAIL) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=1) rolling upgrade of HAProxy on A:cp-upload_eqsin and A:cp - 3.2.23 upgrade (T438828) |
[production] |
| 10:40 |
<dreamyjazz@deploy1003> |
dreamyjazz: Continuing with deployment |
[production] |
| 10:39 |
<dreamyjazz@deploy1003> |
dreamyjazz: Backport for [[gerrit:1344621|AbuseReview: Let specific users and suppressors see vandalism tag (T438860)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 10:36 |
<filippo@cumin1004> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on cloudvirt1080.eqiad.wmnet with reason: provision |
[production] |
| 10:35 |
<dreamyjazz@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1344621|AbuseReview: Let specific users and suppressors see vandalism tag (T438860)]] |
[production] |
| 10:34 |
<sfaci@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/growthbook-next: apply |
[production] |
| 10:32 |
<dreamyjazz@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1344281|WikimediaAntiAbuse: Enable likely vandalism classifier on testwiki (T438860)]] (duration: 10m 34s) |
[production] |
| 10:30 |
<filippo@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.drain on host 'cloudvirt1080' (T435537) |
[admin] |
| 10:30 |
<filippo@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T435537) |
[admin] |
| 10:30 |
<filippo@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T435537) |
[admin] |
| 10:29 |
<trueg@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs-next: apply |
[production] |
| 10:26 |
<dreamyjazz@deploy1003> |
dreamyjazz: Continuing with deployment |
[production] |
| 10:26 |
<trueg@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs-next: apply |
[production] |
| 10:25 |
<dreamyjazz@deploy1003> |
dreamyjazz: Backport for [[gerrit:1344281|WikimediaAntiAbuse: Enable likely vandalism classifier on testwiki (T438860)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 10:23 |
<filippo@cumin1004> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on cloudvirt1079.eqiad.wmnet with reason: provision |
[production] |