|
2024-05-08
ยง
|
| 13:53 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1214.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:52 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1214.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:52 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1211 (T361627)', diff saved to https://phabricator.wikimedia.org/P62083 and previous config saved to /var/cache/conftool/dbconfig/20240508-135250-marostegui.json |
[production] |
| 13:52 |
<moritzm> |
installing Java 17 security updates |
[production] |
| 13:47 |
<jiji@deploy1002> |
helmfile [staging] DONE helmfile.d/services/ipoid: apply |
[production] |
| 13:45 |
<jiji@deploy1002> |
helmfile [staging] START helmfile.d/services/ipoid: apply |
[production] |
| 13:45 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host db1236.eqiad.wmnet |
[production] |
| 13:43 |
<vgutierrez> |
update to tcp-mss-clamper 0.5 on ncredir6001 |
[production] |
| 13:41 |
<zabe@deploy1002> |
zabe and dreamrimmer: Continuing with sync |
[production] |
| 13:41 |
<vgutierrez> |
uploaded tcp-mss-clamper 0.5 (bullseye|bookworm)-wikimedia (apt.wm.o) |
[production] |
| 13:39 |
<jiji@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 13:38 |
<jiji@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 13:38 |
<jmm@cumin2002> |
START - Cookbook sre.puppet.migrate-host for host db1236.eqiad.wmnet |
[production] |
| 13:37 |
<zabe@deploy1002> |
zabe and dreamrimmer: Backport for [[gerrit:1019822|Enable 'flood' user group at en.wikiquote (T351250)]], [[gerrit:1019423|Remove wmgCollectionArticleNamespaces config for enWS (T361422)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:37 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host db1227.eqiad.wmnet |
[production] |
| 13:37 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1211', diff saved to https://phabricator.wikimedia.org/P62082 and previous config saved to /var/cache/conftool/dbconfig/20240508-133742-marostegui.json |
[production] |
| 13:35 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=ms-fe1014.eqiad.wmnet |
[production] |
| 13:35 |
<zabe@deploy1002> |
Started scap: Backport for [[gerrit:1019822|Enable 'flood' user group at en.wikiquote (T351250)]], [[gerrit:1019423|Remove wmgCollectionArticleNamespaces config for enWS (T361422)]] |
[production] |
| 13:32 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=ms-fe1014.eqiad.wmnet |
[production] |
| 13:31 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=ms-fe1013.eqiad.wmnet |
[production] |
| 13:28 |
<zabe@deploy1002> |
Finished scap: Backport for [[gerrit:1026604|Add tm: as alias to template: on English Wikipedia (T363757)]], [[gerrit:1019390|[ruwiki] Limit the use of the ContentTranslation tool (T362440)]] (duration: 21m 36s) |
[production] |
| 13:28 |
<jiji@deploy1002> |
helmfile [codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 13:27 |
<jiji@deploy1002> |
helmfile [codfw] START helmfile.d/admin 'apply'. |
[production] |
| 13:27 |
<jiji@deploy1002> |
helmfile [eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 13:27 |
<jiji@deploy1002> |
helmfile [eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 13:27 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=ms-fe1013.eqiad.wmnet |
[production] |
| 13:27 |
<jiji@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 13:26 |
<jmm@cumin2002> |
START - Cookbook sre.puppet.migrate-host for host db1227.eqiad.wmnet |
[production] |
| 13:25 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host db1202.eqiad.wmnet |
[production] |
| 13:23 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=ms-fe1012.eqiad.wmnet |
[production] |
| 13:22 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1211', diff saved to https://phabricator.wikimedia.org/P62081 and previous config saved to /var/cache/conftool/dbconfig/20240508-132235-marostegui.json |
[production] |
| 13:21 |
<jiji@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 13:17 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=ms-fe1012.eqiad.wmnet |
[production] |
| 13:16 |
<jmm@cumin2002> |
START - Cookbook sre.puppet.migrate-host for host db1202.eqiad.wmnet |
[production] |
| 13:15 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=ms-fe1011.eqiad.wmnet |
[production] |
| 13:14 |
<zabe@deploy1002> |
zabe and dreamrimmer: Continuing with sync |
[production] |
| 13:12 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host db1191.eqiad.wmnet |
[production] |
| 13:11 |
<zabe@deploy1002> |
zabe and dreamrimmer: Backport for [[gerrit:1026604|Add tm: as alias to template: on English Wikipedia (T363757)]], [[gerrit:1019390|[ruwiki] Limit the use of the ContentTranslation tool (T362440)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:10 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=ms-fe1011.eqiad.wmnet |
[production] |
| 13:07 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1211 (T361627)', diff saved to https://phabricator.wikimedia.org/P62080 and previous config saved to /var/cache/conftool/dbconfig/20240508-130727-marostegui.json |
[production] |
| 13:06 |
<zabe@deploy1002> |
Started scap: Backport for [[gerrit:1026604|Add tm: as alias to template: on English Wikipedia (T363757)]], [[gerrit:1019390|[ruwiki] Limit the use of the ContentTranslation tool (T362440)]] |
[production] |
| 13:05 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=ms-fe1010.eqiad.wmnet |
[production] |
| 12:58 |
<elukey> |
depool/deploy/repool every node in the range ms-fe10[10-14] to upgrade envoy to PKI TLS certs |
[production] |
| 12:57 |
<klausman@deploy1002> |
helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 12:57 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=ms-fe1010.eqiad.wmnet |
[production] |
| 12:56 |
<klausman@deploy1002> |
helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. |
[production] |
| 12:53 |
<jmm@cumin2002> |
START - Cookbook sre.puppet.migrate-host for host db1191.eqiad.wmnet |
[production] |
| 12:49 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host db1181.eqiad.wmnet |
[production] |
| 12:26 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P62076 and previous config saved to /var/cache/conftool/dbconfig/20240508-122631-marostegui.json |
[production] |
| 12:22 |
<jmm@cumin2002> |
START - Cookbook sre.puppet.migrate-host for host db1174.eqiad.wmnet |
[production] |