|
2021-03-22
§
|
| 23:52 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudgw1002.eqiad.wmnet with reason: REIMAGE |
[production] |
| 23:49 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudgw1002.eqiad.wmnet with reason: REIMAGE |
[production] |
| 23:34 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts mw2250.codfw.wmnet |
[production] |
| 23:21 |
<pt1979@cumin2001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 23:18 |
<ebernhardson@deploy1002> |
Synchronized php-1.36.0-wmf.35/extensions/WikimediaEvents/modules/ext.wikimediaEvents/searchSatisfaction.js: T262612: Start glent m1 ab test (duration: 01m 53s) |
[production] |
| 23:18 |
<pt1979@cumin2001> |
START - Cookbook sre.dns.netbox |
[production] |
| 23:08 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission for hosts mw2250.codfw.wmnet |
[production] |
| 23:06 |
<James_F> |
Zuul: [labs/tools/majavah-bot] Run generic tox tests |
[releng] |
| 23:01 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts mw2249.codfw.wmnet |
[production] |
| 22:52 |
<mutante> |
decom mw2249 |
[production] |
| 22:44 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission for hosts mw2249.codfw.wmnet |
[production] |
| 21:18 |
<wm-bot> |
<lucaswerkmeister> deployed 2e4e3dca5a (improved Malayalam nouns [not verbs as it says in the commit message, oops] + i18n updates) |
[tools.lexeme-forms] |
| 21:08 |
<sbassett> |
Deployed security patch for T272244 |
[production] |
| 20:02 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2279.codfw.wmnet,service=canary |
[production] |
| 20:02 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2278.codfw.wmnet,service=canary |
[production] |
| 20:02 |
<dzahn@cumin1001> |
conftool action : set/weight=1; selector: name=mw2279.codfw.wmnet,service=canary |
[production] |
| 20:02 |
<dzahn@cumin1001> |
conftool action : set/weight=1; selector: name=mw2278.codfw.wmnet,service=canary |
[production] |
| 19:50 |
<mutante> |
gerrit2001 - restarted apache2 as well for consistency |
[production] |
| 19:47 |
<mutante> |
gerrit - restarting apache2 after we dropped MaxClients config line. This should make us fall back to Debian default MaxRequestWorkers. (since we use event MPM we should not be using MaxClients in the first place, says #httpd) (T277127) |
[production] |
| 19:28 |
<James_F> |
Zuul: [mediawiki/services/function-orchestrator] Add code coverage job |
[releng] |
| 18:20 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 25247c9cbba3d3741908164f2d15fb8497ce8b5e: hrwiki: Configure mentorship for Growth team features (T275684) (duration: 01m 00s) |
[production] |
| 18:13 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 951601f7a4c887f21e209b32dbd1cfd3da084816: Grant enwiki pagemovers the delete-redirect right (T278131) (duration: 00m 59s) |
[production] |
| 18:12 |
<elukey> |
drop /srv/.hardsync* to clean up hardlinks not needed |
[analytics] |
| 18:07 |
<elukey> |
run rm -rfv .hardsync.*/archive/public-datasets/* on thorium:/srv to clean up files to drop (didn't work) |
[analytics] |
| 18:01 |
<elukey> |
drop /srv/.hardsync*trash* on thorium - old hardlinks that should have been trashed |
[analytics] |
| 17:30 |
<Trey314159> |
reindexing Italian wikis on elastic@eqiad, elastic@codfw, and cloudelastic (T274200) |
[production] |
| 16:49 |
<jayme@deploy1002> |
helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 16:48 |
<jayme@deploy1002> |
helmfile [staging-codfw] START helmfile.d/admin 'apply'. |
[production] |
| 16:47 |
<jayme@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 16:46 |
<jayme@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 16:37 |
<jayme@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 16:37 |
<jayme@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 16:12 |
<pt1979@cumin2001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 16:07 |
<pt1979@cumin2001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 100%: Slowly repool db1144:3314', diff saved to https://phabricator.wikimedia.org/P14990 and previous config saved to /var/cache/conftool/dbconfig/20210322-155808-root.json |
[production] |
| 15:52 |
<razzi> |
rebalance kafka partitions for webrequest_text partition 2 |
[analytics] |
| 15:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 75%: Slowly repool db1144:3314', diff saved to https://phabricator.wikimedia.org/P14989 and previous config saved to /var/cache/conftool/dbconfig/20210322-154304-root.json |
[production] |
| 15:38 |
<pt1979@cumin2001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 15:33 |
<pt1979@cumin2001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 50%: Slowly repool db1144:3314', diff saved to https://phabricator.wikimedia.org/P14988 and previous config saved to /var/cache/conftool/dbconfig/20210322-152800-root.json |
[production] |
| 15:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 25%: Slowly repool db1144:3314', diff saved to https://phabricator.wikimedia.org/P14987 and previous config saved to /var/cache/conftool/dbconfig/20210322-151257-root.json |
[production] |
| 14:26 |
<jayme@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 14:25 |
<jayme@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 14:23 |
<jayme@deploy1002> |
helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |