|
2026-01-27
ยง
|
| 14:49 |
<lucaswerkmeister-wmde@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1137428|[u4cwiki] Add signature button to edit toolbar in Case namespace (T392286)]], [[gerrit:1233711|[enwikibooks] Enable VisualEditor on Project and Transwiki namespaces (T415595)]] |
[production] |
| 14:44 |
<lucaswerkmeister-wmde@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1233721|Revert "metrics(ReviseTone): add missing instrumentation parameters" (T415580)]] (duration: 09m 37s) |
[production] |
| 14:40 |
<lucaswerkmeister-wmde@deploy2002> |
lucaswerkmeister-wmde: Continuing with sync |
[production] |
| 14:37 |
<lucaswerkmeister-wmde@deploy2002> |
lucaswerkmeister-wmde: Backport for [[gerrit:1233721|Revert "metrics(ReviseTone): add missing instrumentation parameters" (T415580)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 14:35 |
<lucaswerkmeister-wmde@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1233721|Revert "metrics(ReviseTone): add missing instrumentation parameters" (T415580)]] |
[production] |
| 14:16 |
<lucaswerkmeister-wmde@deploy2002> |
lucaswerkmeister-wmde, migr: Continuing with sync |
[production] |
| 14:15 |
<lucaswerkmeister-wmde@deploy2002> |
lucaswerkmeister-wmde, migr: Backport for [[gerrit:1233709|metrics(ReviseTone): add missing instrumentation parameters (T415580)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 14:13 |
<lucaswerkmeister-wmde@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1233709|metrics(ReviseTone): add missing instrumentation parameters (T415580)]] |
[production] |
| 14:08 |
<dreamyjazz@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1233706|CheckUser: Enable read new for user agent table migration on group1 (T361199)]] (duration: 11m 16s) |
[production] |
| 14:04 |
<dreamyjazz@deploy2002> |
dreamyjazz: Continuing with sync |
[production] |
| 13:59 |
<dreamyjazz@deploy2002> |
dreamyjazz: Backport for [[gerrit:1233706|CheckUser: Enable read new for user agent table migration on group1 (T361199)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 13:57 |
<dreamyjazz@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1233706|CheckUser: Enable read new for user agent table migration on group1 (T361199)]] |
[production] |
| 13:53 |
<samtar@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1233688|Watchlist: do not double-escape labels, and always use `<bdi>` (T415489)]] (duration: 07m 05s) |
[production] |
| 13:49 |
<samtar@deploy2002> |
samtar: Continuing with sync |
[production] |
| 13:49 |
<fnegri@cumin1003> |
END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) for database pplwiki (T415050) |
[production] |
| 13:49 |
<fnegri@cumin1003> |
START - Cookbook sre.wikireplicas.add-wiki for database pplwiki (T415050) |
[production] |
| 13:48 |
<samtar@deploy2002> |
samtar: Backport for [[gerrit:1233688|Watchlist: do not double-escape labels, and always use `<bdi>` (T415489)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 13:48 |
<fnegri@cumin1003> |
END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) for database kaiwiki (T414240) |
[production] |
| 13:48 |
<fnegri@cumin1003> |
START - Cookbook sre.wikireplicas.add-wiki for database kaiwiki (T414240) |
[production] |
| 13:46 |
<samtar@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1233688|Watchlist: do not double-escape labels, and always use `<bdi>` (T415489)]] |
[production] |
| 13:36 |
<samtar@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1214584|EditWatchlistPaginate feature flag has been removed from MW code, so remove it from config (T410908)]] (duration: 07m 31s) |
[production] |
| 13:32 |
<samtar@deploy2002> |
samtar, cparle: Continuing with sync |
[production] |
| 13:31 |
<samtar@deploy2002> |
samtar, cparle: Backport for [[gerrit:1214584|EditWatchlistPaginate feature flag has been removed from MW code, so remove it from config (T410908)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 13:29 |
<samtar@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1214584|EditWatchlistPaginate feature flag has been removed from MW code, so remove it from config (T410908)]] |
[production] |
| 12:55 |
<fnegri@cumin1003> |
END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) for database kajwiki (T415041) |
[production] |
| 12:53 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db2195 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87960 and previous config saved to /var/cache/conftool/dbconfig/20260127-125338-marostegui.json |
[production] |
| 12:53 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db2195.codfw.wmnet with reason: Maintenance |
[production] |
| 12:53 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2181 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87959 and previous config saved to /var/cache/conftool/dbconfig/20260127-125314-marostegui.json |
[production] |
| 12:43 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P87958 and previous config saved to /var/cache/conftool/dbconfig/20260127-124305-marostegui.json |
[production] |
| 12:32 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P87957 and previous config saved to /var/cache/conftool/dbconfig/20260127-123257-marostegui.json |
[production] |
| 12:22 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2181 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87956 and previous config saved to /var/cache/conftool/dbconfig/20260127-122248-marostegui.json |
[production] |
| 11:58 |
<fnegri@cumin1003> |
START - Cookbook sre.wikireplicas.add-wiki for database kajwiki (T415041) |
[production] |
| 11:22 |
<brouberol@deploy2002> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-airflow-main: apply |
[production] |
| 11:22 |
<brouberol@deploy2002> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-airflow-main: apply |
[production] |
| 09:22 |
<urbanecm@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1233648|Update interwiki cache (T413283)]] (duration: 10m 55s) |
[production] |
| 09:13 |
<cwhite@deploy2002> |
Finished deploy [performance/arc-lamp@03c538c]: T391517 (duration: 00m 08s) |
[production] |
| 09:13 |
<cwhite@deploy2002> |
Started deploy [performance/arc-lamp@03c538c]: T391517 |
[production] |
| 09:11 |
<urbanecm@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1233648|Update interwiki cache (T413283)]] |
[production] |
| 09:05 |
<marostegui@cumin1003> |
END (PASS) - Cookbook sre.mysql.newpool (exit_code=0) pool db2248: After reimage |
[production] |
| 08:28 |
<marostegui> |
Deploy schema change on old s1 master db1163 T411163 T411164 |
[production] |
| 08:27 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db1163.eqiad.wmnet with reason: schema change |
[production] |
| 08:25 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depool db1163 T415238', diff saved to https://phabricator.wikimedia.org/P87952 and previous config saved to /var/cache/conftool/dbconfig/20260127-082542-marostegui.json |
[production] |
| 08:25 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Promote db1184 to s1 primary T415238', diff saved to https://phabricator.wikimedia.org/P87951 and previous config saved to /var/cache/conftool/dbconfig/20260127-082502-marostegui.json |
[production] |
| 08:24 |
<marostegui> |
Starting s1 eqiad failover from db1163 to db1184 - T415238 |
[production] |
| 08:20 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 32 hosts with reason: Primary switchover s1 T415238 |
[production] |
| 08:20 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Set db1184 with weight 0 T415238', diff saved to https://phabricator.wikimedia.org/P87950 and previous config saved to /var/cache/conftool/dbconfig/20260127-082020-marostegui.json |
[production] |
| 08:19 |
<marostegui@cumin1003> |
START - Cookbook sre.mysql.newpool pool db2248: After reimage |
[production] |
| 08:01 |
<marostegui@cumin1003> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2248.codfw.wmnet with OS trixie |
[production] |
| 07:39 |
<marostegui@cumin1003> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2248.codfw.wmnet with reason: host reimage |
[production] |
| 07:36 |
<marostegui@cumin1003> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db2248.codfw.wmnet with reason: host reimage |
[production] |