3801-3850 of 10000 results (38ms)
2016-11-03 ยง
20:10 <bblack> codfw cache_text - all pooled nodes are v4 (2x still depooled-but-upgraded) - T131503 [production]
20:04 <ejegg> enabled donations queue consumer [production]
20:02 <ejegg> disabled donations queue consumer [production]
20:01 <twentyafterfour@tin> rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.29.0-wmf.1 [production]
19:32 <ejegg> updated CiviCRM from 954bab421fa3435c5dd1a80f59f657407764b620 to 4ff64afd1ac10643f1f2c91c4aa7a5535512c33e [production]
19:31 <mobrovac> change-prop deploying f107669 [production]
19:28 <twentyafterfour@tin> Synchronized php-1.29.0-wmf.1/extensions/PageImages/includes/ApiQueryPageImages.php: T149849 (duration: 00m 47s) [production]
19:23 <mobrovac> restbase restarting to re-include wikidata domains for T149114 [production]
19:17 <Krenair> <twentyafterfour> !log In order to unblock the train for group2: deploying https://gerrit.wikimedia.org/r/#/c/319643/ refs T149059, T149849 [production]
18:46 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:319644|Fix comment refs (T148327)]] (duration: 00m 47s) [production]
18:39 <thcipriani@tin> Synchronized wmf-config: SWAT: [[gerrit:319605|LABS: Enable Map (GeoJSON) data on Commons (T149548)]] (housekeeping only sync) (duration: 00m 50s) [production]
18:33 <thcipriani@tin> Synchronized php-1.29.0-wmf.1/extensions/Kartographer/includes/ApiQueryMapData.php: SWAT: [[gerrit:319622|Fix warning (T149923)]] (duration: 00m 47s) [production]
18:26 <ema> repooling cp2016 (T131503) [production]
18:21 <thcipriani@tin> Synchronized php-1.28.0-wmf.23/extensions/EventBus/EventBus.php: SWAT: [[gerrit:319586|Log more EventBus HTTP request/response context for HTTP errors (T148251)]] (duration: 00m 52s) [production]
18:19 <thcipriani@tin> Synchronized php-1.29.0-wmf.1/extensions/EventBus/EventBus.php: SWAT: [[gerrit:319587|Log more EventBus HTTP request/response context for HTTP errors (T148251)]] (duration: 00m 49s) [production]
18:02 <hashar> Added security rule for "puppet3-diffs" labs project to allow ssh connection from contint1001 instead of gallium [production]
16:20 <jynus> stopping and upgrading labsdb1009,10,11 (also disabling temporarily puppet) [production]
16:16 <papaul> OS install on labstore2001 [production]
16:13 <mutante> mw1205 - service hhvm restart [production]
15:59 <jynus@tin> Synchronized wmf-config/db-eqiad.php: mariadb: Reduce db1051 load, it has hardware issues (duration: 00m 47s) [production]
15:41 <moritzm> installing memcached security updates on graphite hosts [production]
15:39 <mobrovac> scb in eqiad enabled puppet back [production]
15:26 <mobrovac> scb in eqiad disabling puppet [production]
14:57 <akosiaris> failover icinga from tegmen to einsteinium [production]
14:17 <moritzm> exim reenabled on fermium after mailman update [production]
14:14 <moritzm> temporarily stop exim on fermium for mailman update [production]
14:08 <bblack> cache_text: nginx lossless restarts for libssl update - T144626 - T148917 [production]
14:00 <mobrovac> restbase rolling restart for T149114 [production]
13:56 <bblack> cache_upload: nginx lossless restarts for libssl update - T144626 - T148917 [production]
13:49 <bblack> cache_maps + cache_misc: nginx lossless restarts for libssl update - T144626 - T148917 [production]
13:42 <bblack> cp*: upgrade libssl1.1 to 1.1.0b-1+wmf2 (but no nginx restart yet) - T144626 - T148917 [production]
13:35 <bblack> cp1065: upgrade libssl1.1 to 1.1.0b-1+wmf2 - T144626 - T148917 [production]
13:31 <mobrovac> change-prop deploying a1bd739 [production]
13:06 <hashar@tin> Synchronized wmf-config/CommonSettings.php: Add wmgRevisionSliderBetaFeature (default true) (duration: 00m 46s) [production]
13:05 <hashar@tin> Synchronized wmf-config/InitialiseSettings.php: Add wmgRevisionSliderBetaFeature (default true) (duration: 00m 47s) [production]
13:04 <paravoid> re-enabling cr2-esams:xe-0/1/3 + cr2-eqiad:xe-4/1/3 (esams-eqiad link) [production]
12:09 <marostegui> Deploying schema change s4 commonswiki.revision only codfw - https://phabricator.wikimedia.org/T147305 [production]
11:54 <mobrovac> change-prop deploying 15eae87 [production]
11:52 <mobrovac> restbase deploy end of 1ec3b129 [production]
11:48 <akosiaris> reenable puppet across the fleet on hosts that I had disabled it. https://gerrit.wikimedia.org/r/#/c/316032/1 merged successfully [production]
11:32 <mobrovac> restbase deploy start of 1ec3b129 [production]
11:16 <ema> depooling cp2016, cp2007, cp2019, cp2023: not caching properly (T131503) [production]
10:54 <akosiaris> disable puppet throughout the fleet. merging https://gerrit.wikimedia.org/r/#/c/316032/1 [production]
09:37 <moritzm> uploaded openssl 1.1.0b-1+wmf2 for jessie-wikimedia to apt.wikimedia.org (adding the read_ahead bugfix and dropping the chapoly_pref patch) [production]
09:36 <marostegui> Deploy schema change s5 dewiki.revision - only codfw https://phabricator.wikimedia.org/T148967 [production]
09:24 <ema> upgrading cp2016 (text-codfw) to varnish 4 -- T131503 [production]
09:17 <akosiaris@puppetmaster1001> conftool action : set/pooled=yes; selector: scb2003.codfw.wmnet (tags: ['dc=codfw', 'cluster=scb', 'service=apertium']) [production]
09:17 <akosiaris@puppetmaster1001> conftool action : set/pooled=yes; selector: scb2004.codfw.wmnet (tags: ['dc=codfw', 'cluster=scb', 'service=apertium']) [production]
09:16 <akosiaris@puppetmaster1001> conftool action : set/pooled=yes; selector: scb1004.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=apertium']) [production]
09:16 <akosiaris@puppetmaster1001> conftool action : set/pooled=yes; selector: scb1003.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=scb', 'service=apertium']) [production]