|
2026-08-25
§
|
| 07:13 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-research: apply |
[production] |
| 07:13 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-platform-eng: apply |
[production] |
| 07:13 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-platform-eng: apply |
[production] |
| 07:13 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply |
[production] |
| 07:12 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply |
[production] |
| 07:12 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-main: apply |
[production] |
| 07:12 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-main: apply |
[production] |
| 07:10 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-fr-tech: apply |
[production] |
| 07:10 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-fr-tech: apply |
[production] |
| 07:09 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-dumps: apply |
[production] |
| 07:09 |
<brouberol@deploy1003> |
helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 07:09 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-dumps: apply |
[production] |
| 07:09 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply |
[production] |
| 07:08 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply |
[production] |
| 07:08 |
<brouberol@deploy1003> |
helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. |
[production] |
| 07:07 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 07:07 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 07:05 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-product: apply |
[production] |
| 07:04 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-product: apply |
[production] |
| 04:56 |
<marostegui> |
Failover m5 from db1228 to db1243 - T435895 |
[production] |
| 04:47 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2160,2235].codfw.wmnet,db[1217,1228,1243].eqiad.wmnet with reason: m5 master switch T432967 |
[production] |
| 04:02 |
<mwpresync@deploy1003> |
Pruned MediaWiki: 1.47.0-wmf.14 (duration: 02m 24s) |
[production] |
| 03:40 |
<mwpresync@deploy1003> |
Finished scap sync-world: testwikis to 1.47.0-wmf.17 refs T430836 (duration: 37m 04s) |
[production] |
| 03:29 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Remove db1243 from dbctl T435892', diff saved to https://phabricator.wikimedia.org/P96240 and previous config saved to /var/cache/conftool/dbconfig/20260825-032952-marostegui.json |
[production] |
| 03:29 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db[1217,1243].eqiad.wmnet with reason: Cloning |
[production] |
| 03:03 |
<mwpresync@deploy1003> |
Started scap sync-world: testwikis to 1.47.0-wmf.17 refs T430836 |
[production] |
| 02:08 |
<mwpresync@deploy1003> |
Finished scap build-images: Publishing wmf/next image (duration: 07m 28s) |
[production] |
| 02:00 |
<mwpresync@deploy1003> |
Started scap build-images: Publishing wmf/next image |
[production] |
|
2026-08-24
§
|
| 23:57 |
<TimStarling> |
on db1220 x1 wikishared created Produnto tables T421436 |
[production] |
| 23:18 |
<jdlrobson@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1328722|[beta] Set $wgCentralNoticeProhibitedExperiments]], [[gerrit:1328721|Deploy ReaderExperiments in fa,cs,bn]], [[gerrit:1328376|Support change tags for anonymous users on English Wikivoyage]] (duration: 08m 58s) |
[production] |
| 23:14 |
<jdlrobson@deploy1003> |
ksarabia, jdlrobson: Continuing with deployment |
[production] |
| 23:11 |
<jdlrobson@deploy1003> |
ksarabia, jdlrobson: Backport for [[gerrit:1328722|[beta] Set $wgCentralNoticeProhibitedExperiments]], [[gerrit:1328721|Deploy ReaderExperiments in fa,cs,bn]], [[gerrit:1328376|Support change tags for anonymous users on English Wikivoyage]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 23:09 |
<jdlrobson@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1328722|[beta] Set $wgCentralNoticeProhibitedExperiments]], [[gerrit:1328721|Deploy ReaderExperiments in fa,cs,bn]], [[gerrit:1328376|Support change tags for anonymous users on English Wikivoyage]] |
[production] |
| 23:04 |
<dreamyjazz@deploy1003> |
Synchronized private/PrivateSettings.php: WikimediaAntiAbusePrivate change for T432848 (duration: 08m 59s) |
[production] |
| 23:00 |
<dreamyjazz@deploy1003> |
dreamyjazz: Continuing with deployment |
[production] |
| 22:57 |
<dreamyjazz@deploy1003> |
dreamyjazz: WikimediaAntiAbusePrivate change for T432848 synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 22:47 |
<maryum> |
Deployed security fix for T435457 |
[production] |
| 22:28 |
<RScout-WMF> |
Deployed security fix for T435822 |
[production] |
| 22:17 |
<ryankemper@cumin2003> |
END (PASS) - Cookbook sre.presto.roll-restart-workers (exit_code=0) for Presto an-presto cluster: Roll restart of all Presto's jvm daemons. |
[production] |
| 21:49 |
<ryankemper> |
T435862 [presto] Rolling restart in progress to activate `experimental.spill-enabled=false `from https://gerrit.wikimedia.org/r/c/operations/puppet/+/1328713 |
[production] |
| 21:45 |
<ryankemper> |
T435861 [presto] Kicking off rolling restart to pick up `experimental.spill-enabled=false` changes |
[production] |
| 21:44 |
<ryankemper@cumin2003> |
START - Cookbook sre.presto.roll-restart-workers for Presto an-presto cluster: Roll restart of all Presto's jvm daemons. |
[production] |
| 21:41 |
<sbassett> |
Deployed security fix for T435624 |
[production] |
| 21:21 |
<sbassett> |
Deployed security mitigation for T435455 |
[production] |
| 21:02 |
<arlolra@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1328707|Set a default for wgUseParsoidLinksUpdate in prod (T432760)]] (duration: 07m 07s) |
[production] |
| 20:58 |
<arlolra@deploy1003> |
arlolra: Continuing with deployment |
[production] |
| 20:57 |
<arlolra@deploy1003> |
arlolra: Backport for [[gerrit:1328707|Set a default for wgUseParsoidLinksUpdate in prod (T432760)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 20:55 |
<arlolra@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1328707|Set a default for wgUseParsoidLinksUpdate in prod (T432760)]] |
[production] |
| 20:48 |
<aaron@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1326410|Mark all Mathoid-based endpoints as deprecated (T431372)]] (duration: 12m 33s) |
[production] |
| 20:44 |
<aaron@deploy1003> |
aaron: Continuing with deployment |
[production] |