|
2020-08-27
§
|
| 04:53 |
<marostegui> |
Stop db1074 and db2107 in sync to fix drifts on s2 change_tag - T260042 |
[production] |
| 04:53 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1074', diff saved to https://phabricator.wikimedia.org/P12374 and previous config saved to /var/cache/conftool/dbconfig/20200827-045329-marostegui.json |
[production] |
| 04:04 |
<ryankemper@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=cloudelastic1006.wikimedia.org |
[production] |
| 04:03 |
<ryankemper@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=cloudelastic1005.wikimedia.org |
[production] |
| 04:01 |
<ryankemper@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cloudelastic1005.wikimedia.org |
[production] |
| 02:03 |
<mutante> |
shutting down install3001,install4001,install5001 VMs (no OS yet, but please also don't delete, debugging in progress, shutting them down until I continue on T254157) |
[production] |
|
2020-08-26
§
|
| 23:35 |
<eileen> |
civicrm revision changed from d2e80f7522 to 481ab742db, config revision is e2ab4d7c1f |
[production] |
| 23:00 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) |
[production] |
| 22:36 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 22:30 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 22:26 |
<dzahn@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 22:20 |
<dzahn@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 19:51 |
<XioNoX> |
standardize pfw3-eqiad |
[production] |
| 19:33 |
<marxarelli> |
1.36.0-wmf.6 promoted to group1 (T257974). logs show no new errors |
[production] |
| 19:24 |
<dduvall@deploy1001> |
Synchronized php: group1 wikis to 1.36.0-wmf.6 (duration: 01m 03s) |
[production] |
| 19:23 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.6 |
[production] |
| 18:21 |
<Urbanecm> |
Morning B&C done |
[production] |
| 18:19 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 945b97cff8b8a1e4bb43b613fc93b099f74945f7: Added import sources for mlwiktionary (T260716) (duration: 01m 05s) |
[production] |
| 18:12 |
<Urbanecm> |
Purge Thai and Greek taglines, URLs are at P12372 (T258552) |
[production] |
| 18:11 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 40092898d8c70191324e844d2c222469b954e9ef: Update Thai and Greek taglines (T258552) (duration: 01m 03s) |
[production] |
| 18:09 |
<urbanecm@deploy1001> |
Synchronized static/images/mobile/copyright/: 40092898d8c70191324e844d2c222469b954e9ef: Update Thai and Greek taglines (T258552) (duration: 01m 05s) |
[production] |
| 18:08 |
<herron> |
upgraded eqiad elk v7 cluster from 7.8.0 to 7.9.0 T234854 |
[production] |
| 18:02 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 18:02 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 17:41 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable client side error logging on hewiki (T255585) (duration: 01m 04s) |
[production] |
| 17:14 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Documentation-only change; sync for line sanity (duration: 01m 04s) |
[production] |
| 17:12 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T254349 Set wgVisualEditorEnableBetaFeature true on wikis that need it (duration: 01m 03s) |
[production] |
| 15:59 |
<oblivian@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:53 |
<oblivian@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:41 |
<oblivian@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:11 |
<oblivian@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 14:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1134 for MCR change', diff saved to https://phabricator.wikimedia.org/P12371 and previous config saved to /var/cache/conftool/dbconfig/20200826-145612-marostegui.json |
[production] |
| 14:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12370 and previous config saved to /var/cache/conftool/dbconfig/20200826-145531-marostegui.json |
[production] |
| 14:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12369 and previous config saved to /var/cache/conftool/dbconfig/20200826-144750-marostegui.json |
[production] |
| 14:45 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=inactive:weight=0; selector: name=schema1002.eqiad.wmnet |
[production] |
| 14:45 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=inactive:weight=0; selector: name=schema1001.eqiad.wmnet |
[production] |
| 14:45 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=inactive:weight=0; selector: name=schema2001.codfw.wmnet |
[production] |
| 14:44 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=inactive:weight=0; selector: name=schema2002.codfw.wmnet |
[production] |
| 14:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12368 and previous config saved to /var/cache/conftool/dbconfig/20200826-143623-marostegui.json |
[production] |
| 14:34 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=schema2003.codfw.wmnet |
[production] |
| 14:34 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=schema2004.codfw.wmnet |
[production] |
| 14:33 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=schema1004.eqiad.wmnet |
[production] |
| 14:33 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=schema1003.eqiad.wmnet |
[production] |
| 14:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12367 and previous config saved to /var/cache/conftool/dbconfig/20200826-142746-marostegui.json |
[production] |
| 14:25 |
<jgleeson> |
updated civicrm from 0f195c6cca to d2e80f7522 |
[production] |
| 14:21 |
<pt1979@cumin2001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 14:20 |
<marostegui> |
Upgrade mysql on db1091 after MCR changes |
[production] |
| 14:13 |
<pt1979@cumin2001> |
START - Cookbook sre.dns.netbox |
[production] |
| 13:37 |
<kormat@cumin1001> |
dbctl commit (dc=all): 'Repooling db1110 @ 100% T261276', diff saved to https://phabricator.wikimedia.org/P12366 and previous config saved to /var/cache/conftool/dbconfig/20200826-133753-kormat.json |
[production] |
| 13:18 |
<duesen> |
daniel@mwmaint1002:/srv/mediawiki/php-1.36.0-wmf.5$ mwscript maintenance/findBadBlobs.php dewiki --mark T205936 --revisions - < ~/T205936-dewiki-20050512070000.ids # marking known bad revisions for T205936 |
[production] |