|
2020-08-27
§
|
| 06:45 |
<elukey@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 06:34 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 06:31 |
<elukey@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 06:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1079,db1082', diff saved to https://phabricator.wikimedia.org/P12392 and previous config saved to /var/cache/conftool/dbconfig/20200827-060652-marostegui.json |
[production] |
| 05:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1079,db1082', diff saved to https://phabricator.wikimedia.org/P12391 and previous config saved to /var/cache/conftool/dbconfig/20200827-055815-marostegui.json |
[production] |
| 05:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1079,db1082', diff saved to https://phabricator.wikimedia.org/P12390 and previous config saved to /var/cache/conftool/dbconfig/20200827-055522-marostegui.json |
[production] |
| 05:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1079', diff saved to https://phabricator.wikimedia.org/P12389 and previous config saved to /var/cache/conftool/dbconfig/20200827-055126-marostegui.json |
[production] |
| 05:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1082', diff saved to https://phabricator.wikimedia.org/P12388 and previous config saved to /var/cache/conftool/dbconfig/20200827-055104-marostegui.json |
[production] |
| 05:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1079', diff saved to https://phabricator.wikimedia.org/P12387 and previous config saved to /var/cache/conftool/dbconfig/20200827-054259-marostegui.json |
[production] |
| 05:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1074 db1085 db1078', diff saved to https://phabricator.wikimedia.org/P12386 and previous config saved to /var/cache/conftool/dbconfig/20200827-054114-marostegui.json |
[production] |
| 05:38 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1082', diff saved to https://phabricator.wikimedia.org/P12385 and previous config saved to /var/cache/conftool/dbconfig/20200827-053814-marostegui.json |
[production] |
| 05:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1134 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12384 and previous config saved to /var/cache/conftool/dbconfig/20200827-053558-marostegui.json |
[production] |
| 05:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1074, db1085, db1079', diff saved to https://phabricator.wikimedia.org/P12383 and previous config saved to /var/cache/conftool/dbconfig/20200827-053509-marostegui.json |
[production] |
| 05:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1074, db1085, db1079', diff saved to https://phabricator.wikimedia.org/P12382 and previous config saved to /var/cache/conftool/dbconfig/20200827-053100-marostegui.json |
[production] |
| 05:29 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1134', diff saved to https://phabricator.wikimedia.org/P12381 and previous config saved to /var/cache/conftool/dbconfig/20200827-052925-marostegui.json |
[production] |
| 05:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1085', diff saved to https://phabricator.wikimedia.org/P12380 and previous config saved to /var/cache/conftool/dbconfig/20200827-052818-marostegui.json |
[production] |
| 05:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1074', diff saved to https://phabricator.wikimedia.org/P12379 and previous config saved to /var/cache/conftool/dbconfig/20200827-052413-marostegui.json |
[production] |
| 05:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1079', diff saved to https://phabricator.wikimedia.org/P12378 and previous config saved to /var/cache/conftool/dbconfig/20200827-051609-marostegui.json |
[production] |
| 05:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1134 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12377 and previous config saved to /var/cache/conftool/dbconfig/20200827-051546-marostegui.json |
[production] |
| 05:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1134 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12376 and previous config saved to /var/cache/conftool/dbconfig/20200827-050754-marostegui.json |
[production] |
| 05:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1085', diff saved to https://phabricator.wikimedia.org/P12375 and previous config saved to /var/cache/conftool/dbconfig/20200827-050727-marostegui.json |
[production] |
| 04:53 |
<marostegui> |
Stop db1074 and db2107 in sync to fix drifts on s2 change_tag - T260042 |
[production] |
| 04:53 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1074', diff saved to https://phabricator.wikimedia.org/P12374 and previous config saved to /var/cache/conftool/dbconfig/20200827-045329-marostegui.json |
[production] |
| 04:04 |
<ryankemper@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=cloudelastic1006.wikimedia.org |
[production] |
| 04:03 |
<ryankemper@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: name=cloudelastic1005.wikimedia.org |
[production] |
| 04:01 |
<ryankemper@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cloudelastic1005.wikimedia.org |
[production] |
| 02:03 |
<mutante> |
shutting down install3001,install4001,install5001 VMs (no OS yet, but please also don't delete, debugging in progress, shutting them down until I continue on T254157) |
[production] |
|
2020-08-26
§
|
| 23:35 |
<eileen> |
civicrm revision changed from d2e80f7522 to 481ab742db, config revision is e2ab4d7c1f |
[production] |
| 23:00 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) |
[production] |
| 22:36 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 22:30 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 22:26 |
<dzahn@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 22:20 |
<dzahn@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 19:51 |
<XioNoX> |
standardize pfw3-eqiad |
[production] |
| 19:33 |
<marxarelli> |
1.36.0-wmf.6 promoted to group1 (T257974). logs show no new errors |
[production] |
| 19:24 |
<dduvall@deploy1001> |
Synchronized php: group1 wikis to 1.36.0-wmf.6 (duration: 01m 03s) |
[production] |
| 19:23 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.6 |
[production] |
| 18:21 |
<Urbanecm> |
Morning B&C done |
[production] |
| 18:19 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 945b97cff8b8a1e4bb43b613fc93b099f74945f7: Added import sources for mlwiktionary (T260716) (duration: 01m 05s) |
[production] |
| 18:12 |
<Urbanecm> |
Purge Thai and Greek taglines, URLs are at P12372 (T258552) |
[production] |
| 18:11 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 40092898d8c70191324e844d2c222469b954e9ef: Update Thai and Greek taglines (T258552) (duration: 01m 03s) |
[production] |
| 18:09 |
<urbanecm@deploy1001> |
Synchronized static/images/mobile/copyright/: 40092898d8c70191324e844d2c222469b954e9ef: Update Thai and Greek taglines (T258552) (duration: 01m 05s) |
[production] |
| 18:08 |
<herron> |
upgraded eqiad elk v7 cluster from 7.8.0 to 7.9.0 T234854 |
[production] |
| 18:02 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 18:02 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 17:41 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable client side error logging on hewiki (T255585) (duration: 01m 04s) |
[production] |
| 17:14 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Documentation-only change; sync for line sanity (duration: 01m 04s) |
[production] |
| 17:12 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T254349 Set wgVisualEditorEnableBetaFeature true on wikis that need it (duration: 01m 03s) |
[production] |
| 15:59 |
<oblivian@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:53 |
<oblivian@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |