|
2021-03-22
ยง
|
| 22:44 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission for hosts mw2249.codfw.wmnet |
[production] |
| 21:18 |
<wm-bot> |
<lucaswerkmeister> deployed 2e4e3dca5a (improved Malayalam nouns [not verbs as it says in the commit message, oops] + i18n updates) |
[tools.lexeme-forms] |
| 21:08 |
<sbassett> |
Deployed security patch for T272244 |
[production] |
| 20:02 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2279.codfw.wmnet,service=canary |
[production] |
| 20:02 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2278.codfw.wmnet,service=canary |
[production] |
| 20:02 |
<dzahn@cumin1001> |
conftool action : set/weight=1; selector: name=mw2279.codfw.wmnet,service=canary |
[production] |
| 20:02 |
<dzahn@cumin1001> |
conftool action : set/weight=1; selector: name=mw2278.codfw.wmnet,service=canary |
[production] |
| 19:50 |
<mutante> |
gerrit2001 - restarted apache2 as well for consistency |
[production] |
| 19:47 |
<mutante> |
gerrit - restarting apache2 after we dropped MaxClients config line. This should make us fall back to Debian default MaxRequestWorkers. (since we use event MPM we should not be using MaxClients in the first place, says #httpd) (T277127) |
[production] |
| 19:28 |
<James_F> |
Zuul: [mediawiki/services/function-orchestrator] Add code coverage job |
[releng] |
| 18:20 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 25247c9cbba3d3741908164f2d15fb8497ce8b5e: hrwiki: Configure mentorship for Growth team features (T275684) (duration: 01m 00s) |
[production] |
| 18:13 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 951601f7a4c887f21e209b32dbd1cfd3da084816: Grant enwiki pagemovers the delete-redirect right (T278131) (duration: 00m 59s) |
[production] |
| 18:12 |
<elukey> |
drop /srv/.hardsync* to clean up hardlinks not needed |
[analytics] |
| 18:07 |
<elukey> |
run rm -rfv .hardsync.*/archive/public-datasets/* on thorium:/srv to clean up files to drop (didn't work) |
[analytics] |
| 18:01 |
<elukey> |
drop /srv/.hardsync*trash* on thorium - old hardlinks that should have been trashed |
[analytics] |
| 17:30 |
<Trey314159> |
reindexing Italian wikis on elastic@eqiad, elastic@codfw, and cloudelastic (T274200) |
[production] |
| 16:49 |
<jayme@deploy1002> |
helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 16:48 |
<jayme@deploy1002> |
helmfile [staging-codfw] START helmfile.d/admin 'apply'. |
[production] |
| 16:47 |
<jayme@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 16:46 |
<jayme@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 16:37 |
<jayme@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 16:37 |
<jayme@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 16:12 |
<pt1979@cumin2001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 16:07 |
<pt1979@cumin2001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 100%: Slowly repool db1144:3314', diff saved to https://phabricator.wikimedia.org/P14990 and previous config saved to /var/cache/conftool/dbconfig/20210322-155808-root.json |
[production] |
| 15:52 |
<razzi> |
rebalance kafka partitions for webrequest_text partition 2 |
[analytics] |
| 15:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 75%: Slowly repool db1144:3314', diff saved to https://phabricator.wikimedia.org/P14989 and previous config saved to /var/cache/conftool/dbconfig/20210322-154304-root.json |
[production] |
| 15:38 |
<pt1979@cumin2001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 15:33 |
<pt1979@cumin2001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 50%: Slowly repool db1144:3314', diff saved to https://phabricator.wikimedia.org/P14988 and previous config saved to /var/cache/conftool/dbconfig/20210322-152800-root.json |
[production] |
| 15:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 25%: Slowly repool db1144:3314', diff saved to https://phabricator.wikimedia.org/P14987 and previous config saved to /var/cache/conftool/dbconfig/20210322-151257-root.json |
[production] |
| 14:26 |
<jayme@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 14:25 |
<jayme@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 14:23 |
<jayme@deploy1002> |
helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 14:22 |
<jayme@deploy1002> |
helmfile [staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 14:14 |
<elukey@deploy1002> |
helmfile [ml-serve-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 14:14 |
<elukey@deploy1002> |
helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 14:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1144:3314 for schema change', diff saved to https://phabricator.wikimedia.org/P14986 and previous config saved to /var/cache/conftool/dbconfig/20210322-141146-marostegui.json |
[production] |
| 14:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 100%: Slowly repool db1143', diff saved to https://phabricator.wikimedia.org/P14985 and previous config saved to /var/cache/conftool/dbconfig/20210322-140800-root.json |
[production] |
| 14:07 |
<XioNoX> |
rename cloud-hosts1-b-eqiad to cloud-hosts1-eqiad - T277771 |
[production] |
| 14:07 |
<XioNoX> |
rename cloud-hosts1-b-eqiad to cloud-hosts1-eqiad |
[production] |
| 13:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 75%: Slowly repool db1143', diff saved to https://phabricator.wikimedia.org/P14984 and previous config saved to /var/cache/conftool/dbconfig/20210322-135256-root.json |
[production] |
| 13:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 50%: Slowly repool db1143', diff saved to https://phabricator.wikimedia.org/P14983 and previous config saved to /var/cache/conftool/dbconfig/20210322-133753-root.json |
[production] |
| 13:26 |
<kharlan@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . |
[production] |
| 13:26 |
<kharlan@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . |
[production] |
| 13:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1143 (re)pooling @ 25%: Slowly repool db1143', diff saved to https://phabricator.wikimedia.org/P14982 and previous config saved to /var/cache/conftool/dbconfig/20210322-132249-root.json |
[production] |
| 13:20 |
<kharlan@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . |
[production] |
| 13:20 |
<kharlan@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . |
[production] |
| 13:16 |
<kharlan@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'linkrecommendation' for release 'staging' . |
[production] |
| 12:28 |
<jayme@deploy1002> |
helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |