3401-3450 of 10000 results (30ms)
2020-08-27 §
04:04 <ryankemper@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: name=cloudelastic1006.wikimedia.org [production]
04:03 <ryankemper@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: name=cloudelastic1005.wikimedia.org [production]
04:01 <ryankemper@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cloudelastic1005.wikimedia.org [production]
02:03 <mutante> shutting down install3001,install4001,install5001 VMs (no OS yet, but please also don't delete, debugging in progress, shutting them down until I continue on T254157) [production]
2020-08-26 §
23:35 <eileen> civicrm revision changed from d2e80f7522 to 481ab742db, config revision is e2ab4d7c1f [production]
23:00 <ryankemper@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) [production]
22:36 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
22:30 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
22:26 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm [production]
22:20 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm [production]
19:51 <XioNoX> standardize pfw3-eqiad [production]
19:33 <marxarelli> 1.36.0-wmf.6 promoted to group1 (T257974). logs show no new errors [production]
19:24 <dduvall@deploy1001> Synchronized php: group1 wikis to 1.36.0-wmf.6 (duration: 01m 03s) [production]
19:23 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.6 [production]
18:21 <Urbanecm> Morning B&C done [production]
18:19 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 945b97cff8b8a1e4bb43b613fc93b099f74945f7: Added import sources for mlwiktionary (T260716) (duration: 01m 05s) [production]
18:12 <Urbanecm> Purge Thai and Greek taglines, URLs are at P12372 (T258552) [production]
18:11 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 40092898d8c70191324e844d2c222469b954e9ef: Update Thai and Greek taglines (T258552) (duration: 01m 03s) [production]
18:09 <urbanecm@deploy1001> Synchronized static/images/mobile/copyright/: 40092898d8c70191324e844d2c222469b954e9ef: Update Thai and Greek taglines (T258552) (duration: 01m 05s) [production]
18:08 <herron> upgraded eqiad elk v7 cluster from 7.8.0 to 7.9.0 T234854 [production]
18:02 <cmjohnson@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
18:02 <cmjohnson@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:41 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable client side error logging on hewiki (T255585) (duration: 01m 04s) [production]
17:14 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Documentation-only change; sync for line sanity (duration: 01m 04s) [production]
17:12 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: T254349 Set wgVisualEditorEnableBetaFeature true on wikis that need it (duration: 01m 03s) [production]
15:59 <oblivian@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
15:53 <oblivian@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
15:41 <oblivian@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
15:11 <oblivian@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
14:56 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1134 for MCR change', diff saved to https://phabricator.wikimedia.org/P12371 and previous config saved to /var/cache/conftool/dbconfig/20200826-145612-marostegui.json [production]
14:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12370 and previous config saved to /var/cache/conftool/dbconfig/20200826-145531-marostegui.json [production]
14:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12369 and previous config saved to /var/cache/conftool/dbconfig/20200826-144750-marostegui.json [production]
14:45 <elukey@puppetmaster1001> conftool action : set/pooled=inactive:weight=0; selector: name=schema1002.eqiad.wmnet [production]
14:45 <elukey@puppetmaster1001> conftool action : set/pooled=inactive:weight=0; selector: name=schema1001.eqiad.wmnet [production]
14:45 <elukey@puppetmaster1001> conftool action : set/pooled=inactive:weight=0; selector: name=schema2001.codfw.wmnet [production]
14:44 <elukey@puppetmaster1001> conftool action : set/pooled=inactive:weight=0; selector: name=schema2002.codfw.wmnet [production]
14:36 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12368 and previous config saved to /var/cache/conftool/dbconfig/20200826-143623-marostegui.json [production]
14:34 <elukey@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: name=schema2003.codfw.wmnet [production]
14:34 <elukey@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: name=schema2004.codfw.wmnet [production]
14:33 <elukey@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: name=schema1004.eqiad.wmnet [production]
14:33 <elukey@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: name=schema1003.eqiad.wmnet [production]
14:27 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1091 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12367 and previous config saved to /var/cache/conftool/dbconfig/20200826-142746-marostegui.json [production]
14:25 <jgleeson> updated civicrm from 0f195c6cca to d2e80f7522 [production]
14:21 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:20 <marostegui> Upgrade mysql on db1091 after MCR changes [production]
14:13 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
13:37 <kormat@cumin1001> dbctl commit (dc=all): 'Repooling db1110 @ 100% T261276', diff saved to https://phabricator.wikimedia.org/P12366 and previous config saved to /var/cache/conftool/dbconfig/20200826-133753-kormat.json [production]
13:18 <duesen> daniel@mwmaint1002:/srv/mediawiki/php-1.36.0-wmf.5$ mwscript maintenance/findBadBlobs.php dewiki --mark T205936 --revisions - < ~/T205936-dewiki-20050512070000.ids # marking known bad revisions for T205936 [production]
13:17 <kormat@cumin1001> dbctl commit (dc=all): 'Repooling db1110 @ 75% T261276', diff saved to https://phabricator.wikimedia.org/P12365 and previous config saved to /var/cache/conftool/dbconfig/20200826-131732-kormat.json [production]
13:16 <duesen> daniel@mwmaint1002:/srv/mediawiki/php-1.36.0-wmf.5$ mwscript maintenance/findBadBlobs.php oswiki --mark T205936 --revisions - < ~/T205936-oswiki-20090309200000.ids # marking known bad revisions for T205936 [production]