|
2025-12-03
ยง
|
| 14:44 |
<esanders@deploy2002> |
esanders: Backport for [[gerrit:1212161|DiscussionTools: cleanup unused config]], [[gerrit:1164178|Remove wgVisualEditorEditCheckSingleCheckMode]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 14:42 |
<esanders@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1212161|DiscussionTools: cleanup unused config]], [[gerrit:1164178|Remove wgVisualEditorEditCheckSingleCheckMode]] |
[production] |
| 14:38 |
<esanders@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1213501|Set Flow to read-only everywhere (T402552)]] (duration: 09m 44s) |
[production] |
| 14:33 |
<esanders@deploy2002> |
esanders: Continuing with sync |
[production] |
| 14:31 |
<esanders@deploy2002> |
esanders: Backport for [[gerrit:1213501|Set Flow to read-only everywhere (T402552)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 14:29 |
<esanders@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1213501|Set Flow to read-only everywhere (T402552)]] |
[production] |
| 14:27 |
<XioNoX> |
push pfw policies - T411566 |
[production] |
| 14:27 |
<sbisson@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1214036|CX3 Build 1.0.0+20251201 (T408842 T408844)]] (duration: 12m 01s) |
[production] |
| 14:21 |
<sbisson@deploy2002> |
sbisson: Continuing with sync |
[production] |
| 14:17 |
<sbisson@deploy2002> |
sbisson: Backport for [[gerrit:1214036|CX3 Build 1.0.0+20251201 (T408842 T408844)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 14:15 |
<sbisson@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1214036|CX3 Build 1.0.0+20251201 (T408842 T408844)]] |
[production] |
| 13:50 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2239.codfw.wmnet with reason: Maintenance |
[production] |
| 13:50 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2227 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P86380 and previous config saved to /var/cache/conftool/dbconfig/20251203-135000-marostegui.json |
[production] |
| 13:34 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2227', diff saved to https://phabricator.wikimedia.org/P86379 and previous config saved to /var/cache/conftool/dbconfig/20251203-133452-marostegui.json |
[production] |
| 13:32 |
<kart_> |
Updated Recommendation API to 2025-12-02-200719-production (T408845, T408844, T384485) |
[production] |
| 13:30 |
<kartik@deploy2002> |
helmfile [ml-serve-codfw] 'sync' command on namespace 'recommendation-api-ng' for release 'main' . |
[production] |
| 13:25 |
<kartik@deploy2002> |
helmfile [ml-serve-eqiad] 'sync' command on namespace 'recommendation-api-ng' for release 'main' . |
[production] |
| 13:22 |
<kartik@deploy2002> |
helmfile [ml-staging-codfw] 'sync' command on namespace 'recommendation-api-ng' for release 'main' . |
[production] |
| 13:19 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2227', diff saved to https://phabricator.wikimedia.org/P86378 and previous config saved to /var/cache/conftool/dbconfig/20251203-131945-marostegui.json |
[production] |
| 13:14 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Depooling db2229 (T410589)', diff saved to https://phabricator.wikimedia.org/P86377 and previous config saved to /var/cache/conftool/dbconfig/20251203-131448-ladsgroup.json |
[production] |
| 13:14 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db2229.codfw.wmnet with reason: Maintenance |
[production] |
| 13:14 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2224 (T410589)', diff saved to https://phabricator.wikimedia.org/P86376 and previous config saved to /var/cache/conftool/dbconfig/20251203-131435-ladsgroup.json |
[production] |
| 13:04 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2227 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P86375 and previous config saved to /var/cache/conftool/dbconfig/20251203-130437-marostegui.json |
[production] |
| 13:01 |
<daniel@deploy2002> |
helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply |
[production] |
| 13:00 |
<daniel@deploy2002> |
helmfile [codfw] START helmfile.d/services/rest-gateway: apply |
[production] |
| 13:00 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db2227 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P86374 and previous config saved to /var/cache/conftool/dbconfig/20251203-130002-marostegui.json |
[production] |
| 12:59 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2227.codfw.wmnet with reason: Maintenance |
[production] |
| 12:59 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2209 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P86373 and previous config saved to /var/cache/conftool/dbconfig/20251203-125938-marostegui.json |
[production] |
| 12:59 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2224', diff saved to https://phabricator.wikimedia.org/P86372 and previous config saved to /var/cache/conftool/dbconfig/20251203-125927-ladsgroup.json |
[production] |
| 12:57 |
<daniel@deploy2002> |
helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply |
[production] |
| 12:56 |
<daniel@deploy2002> |
helmfile [eqiad] START helmfile.d/services/rest-gateway: apply |
[production] |
| 12:56 |
<cgoubert@deploy2002> |
helmfile [codfw] DONE helmfile.d/services/api-gateway: apply |
[production] |
| 12:55 |
<cgoubert@deploy2002> |
helmfile [codfw] START helmfile.d/services/api-gateway: apply |
[production] |
| 12:54 |
<cgoubert@deploy2002> |
helmfile [eqiad] DONE helmfile.d/services/api-gateway: apply |
[production] |
| 12:53 |
<cgoubert@deploy2002> |
helmfile [eqiad] START helmfile.d/services/api-gateway: apply |
[production] |
| 12:52 |
<daniel@deploy2002> |
helmfile [staging] DONE helmfile.d/services/rest-gateway: apply |
[production] |
| 12:52 |
<cgoubert@deploy2002> |
helmfile [staging] DONE helmfile.d/services/api-gateway: apply |
[production] |
| 12:51 |
<cgoubert@deploy2002> |
helmfile [staging] START helmfile.d/services/api-gateway: apply |
[production] |
| 12:51 |
<daniel@deploy2002> |
helmfile [staging] START helmfile.d/services/rest-gateway: apply |
[production] |
| 12:50 |
<daniel@deploy2002> |
helmfile [staging] DONE helmfile.d/services/rest-gateway: apply |
[production] |
| 12:50 |
<daniel@deploy2002> |
helmfile [staging] START helmfile.d/services/rest-gateway: apply |
[production] |
| 12:50 |
<daniel@deploy2002> |
helmfile [staging] DONE helmfile.d/services/rest-gateway: apply |
[production] |
| 12:49 |
<daniel@deploy2002> |
helmfile [staging] START helmfile.d/services/rest-gateway: apply |
[production] |
| 12:44 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2209', diff saved to https://phabricator.wikimedia.org/P86371 and previous config saved to /var/cache/conftool/dbconfig/20251203-124430-marostegui.json |
[production] |
| 12:44 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2224', diff saved to https://phabricator.wikimedia.org/P86370 and previous config saved to /var/cache/conftool/dbconfig/20251203-124419-ladsgroup.json |
[production] |
| 12:32 |
<claime> |
Restarting failed timer dump_cloud_ip_ranges on puppetservers |
[production] |
| 12:30 |
<daniel@deploy2002> |
helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply |
[production] |
| 12:30 |
<daniel@deploy2002> |
helmfile [codfw] START helmfile.d/services/rest-gateway: apply |
[production] |
| 12:29 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2209', diff saved to https://phabricator.wikimedia.org/P86369 and previous config saved to /var/cache/conftool/dbconfig/20251203-122923-marostegui.json |
[production] |
| 12:29 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2224 (T410589)', diff saved to https://phabricator.wikimedia.org/P86368 and previous config saved to /var/cache/conftool/dbconfig/20251203-122912-ladsgroup.json |
[production] |