|
2020-08-26
§
|
| 23:35 |
<eileen> |
civicrm revision changed from d2e80f7522 to 481ab742db, config revision is e2ab4d7c1f |
[production] |
| 23:00 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) |
[production] |
| 22:36 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 22:30 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 22:26 |
<dzahn@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 22:20 |
<dzahn@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 22:16 |
<bstorm> |
truncated file virgule.err since it had reached 97G in size and was just repeated output from a job (not actually errors) |
[tools.robokobot] |
| 21:29 |
<wm-bot> |
<jeanfred> jsub new categorize_images job to test 7b0a9d7 |
[tools.heritage] |
| 21:29 |
<wm-bot> |
<jeanfred> Deploy latest from Git master: 7b0a9d7 |
[tools.heritage] |
| 21:08 |
<bd808> |
Disabled puppet on tools-proxy-06 to test fixes for a bug in the new T251628 code |
[tools] |
| 20:39 |
<wm-bot> |
<root> Deleted old access.log files (6G total size) |
[tools.zoomviewer] |
| 20:36 |
<wm-bot> |
<root> Changed schedule of check.sh crontab to once per 5 minutes |
[tools.zoomviewer] |
| 20:35 |
<wm-bot> |
<root> Disabled crontab for "jlocal echo ${HOME}/check.sh >> /tmp/debugomat" |
[tools.zoomviewer] |
| 19:51 |
<XioNoX> |
standardize pfw3-eqiad |
[production] |
| 19:33 |
<marxarelli> |
1.36.0-wmf.6 promoted to group1 (T257974). logs show no new errors |
[production] |
| 19:24 |
<dduvall@deploy1001> |
Synchronized php: group1 wikis to 1.36.0-wmf.6 (duration: 01m 03s) |
[production] |
| 19:23 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.6 |
[production] |
| 18:21 |
<Urbanecm> |
Morning B&C done |
[production] |
| 18:19 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 945b97cff8b8a1e4bb43b613fc93b099f74945f7: Added import sources for mlwiktionary (T260716) (duration: 01m 05s) |
[production] |
| 18:12 |
<wm-bot> |
<root> Added `-o /dev/null -e /dev/null` to jsub commands generated by index.php (T248188) |
[tools.zoomviewer] |
| 18:12 |
<Urbanecm> |
Purge Thai and Greek taglines, URLs are at P12372 (T258552) |
[production] |
| 18:11 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 40092898d8c70191324e844d2c222469b954e9ef: Update Thai and Greek taglines (T258552) (duration: 01m 03s) |
[production] |
| 18:09 |
<urbanecm@deploy1001> |
Synchronized static/images/mobile/copyright/: 40092898d8c70191324e844d2c222469b954e9ef: Update Thai and Greek taglines (T258552) (duration: 01m 05s) |
[production] |
| 18:08 |
<herron> |
upgraded eqiad elk v7 cluster from 7.8.0 to 7.9.0 T234854 |
[production] |
| 18:02 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 18:02 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 17:41 |
<bstorm> |
deleting data in the old NFS location of paws/userhomes T261338 |
[tools.paws] |
| 17:41 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable client side error logging on hewiki (T255585) (duration: 01m 04s) |
[production] |
| 17:14 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Documentation-only change; sync for line sanity (duration: 01m 04s) |
[production] |
| 17:12 |
<bstorm> |
Running 'ionice -c 3 nice -19 find /srv/tools -type f -size +100M -printf "%k KB %p\n" > tools_large_files_20200826.txt' on labstore1004 T261336 |
[admin] |
| 17:12 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T254349 Set wgVisualEditorEnableBetaFeature true on wikis that need it (duration: 01m 03s) |
[production] |
| 16:29 |
<James_F> |
Zuul: [mediawiki/extensions/WikiLambda] Add code health job |
[releng] |
| 15:59 |
<oblivian@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:53 |
<oblivian@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:41 |
<oblivian@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:11 |
<oblivian@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 14:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1134 for MCR change', diff saved to https://phabricator.wikimedia.org/P12371 and previous config saved to /var/cache/conftool/dbconfig/20200826-145612-marostegui.json |
[production] |
| 14:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12370 and previous config saved to /var/cache/conftool/dbconfig/20200826-145531-marostegui.json |
[production] |
| 14:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12369 and previous config saved to /var/cache/conftool/dbconfig/20200826-144750-marostegui.json |
[production] |
| 14:45 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=inactive:weight=0; selector: name=schema1002.eqiad.wmnet |
[production] |
| 14:45 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=inactive:weight=0; selector: name=schema1001.eqiad.wmnet |
[production] |
| 14:45 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=inactive:weight=0; selector: name=schema2001.codfw.wmnet |
[production] |
| 14:44 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=inactive:weight=0; selector: name=schema2002.codfw.wmnet |
[production] |
| 14:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12368 and previous config saved to /var/cache/conftool/dbconfig/20200826-143623-marostegui.json |
[production] |
| 14:34 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=schema2003.codfw.wmnet |
[production] |
| 14:34 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=schema2004.codfw.wmnet |
[production] |
| 14:33 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=schema1004.eqiad.wmnet |
[production] |
| 14:33 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=schema1003.eqiad.wmnet |
[production] |