7901-7950 of 10000 results (74ms)
2017-10-27 §
11:04 <marostegui> Optimize cebwiki.templatelinks on db1103 - T174509 [production]
11:03 <mobrovac@tin> Started deploy [citoid/deploy@ff63420]: Update dependencies [production]
08:56 <marostegui> Optimize table commonswiki.recentchanges on dbstore1001 - T162789 [production]
08:11 <marostegui> Drop redundant indexes from pagelinks and templatelinks on s3 wikis only for dbstore2001 and dbstore2002 - T174509 [production]
05:59 <marostegui> Stop MySQL on db2084 to copy s5 to db2086 - T178359 [production]
05:55 <marostegui> dbstore1001: convert itwiki.pagelinks to TokuDB - T162789 [production]
05:53 <mutante> uploaded parsoid_0.8.0 to releases.wikimedia.org (for Subbu - T179134) [production]
05:46 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1060 - T174509 (duration: 00m 50s) [production]
03:02 <bblack> cp1067, cp4026 - backend restarts, mailbox lag [production]
2017-10-26 §
22:47 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: wikidata to wmf.4 [production]
22:39 <bblack> restarting varnish-backend on cp1053 (mailbox lag from ongoing issues elsewhere?) [production]
21:40 <bblack> raising backend max_connections for api.svc.eqiad.wmnet + appservers.svc.eqiad.wmnet from 1K to 10K on cp1053.eqiad.wmnet (current funnel for the bulk of the 503s) [production]
21:32 <hoo@tin> Synchronized wmf-config/InitialiseSettings.php: Temporary disable remex html (T178632) (duration: 00m 50s) [production]
21:30 <hoo@tin> Synchronized wmf-config/InitialiseSettings.php: Temporary disable remex html (T178632) (duration: 00m 50s) [production]
21:00 <hoo> Fully revert all changes related to T178180 [production]
20:58 <hoo@tin> Synchronized wmf-config/Wikibase.php: Revert "Add property for RDF mapping of external identifiers for Wikidata" (T178180) (duration: 00m 50s) [production]
20:02 <ladsgroup@tin> Synchronized wmf-config/InitialiseSettings.php: UBN! disbale ores for wikidata (T179107) (duration: 00m 50s) [production]
20:00 <ladsgroup@tin> Synchronized wmf-config/InitialiseSettings.php: UBN! disbale ores for wikidata (T179107) (duration: 00m 50s) [production]
19:41 <awight@tin> Finished deploy [ores/deploy@0adae70]: Increase extractor wikidata API timeout to 15s, T179107 (duration: 07m 25s) [production]
19:36 <aaron@tin> Started restart [jobrunner/jobrunner@a20d043]: (no justification provided) [production]
19:34 <awight@tin> Started deploy [ores/deploy@0adae70]: Increase extractor wikidata API timeout to 15s, T179107 [production]
19:33 <awight@tin> Finished deploy [ores/deploy@0adae70]: Increase extractor wikidata API timeout to 15s, T179107 (duration: 00m 10s) [production]
19:33 <awight@tin> Started deploy [ores/deploy@0adae70]: Increase extractor wikidata API timeout to 15s, T179107 [production]
19:27 <demon@tin> Synchronized php-1.31.0-wmf.5/includes/libs/filebackend/SwiftFileBackend.php: Better error grouping (duration: 00m 50s) [production]
19:14 <ladsgroup@tin> Synchronized php-1.31.0-wmf.5/extensions/WikibaseQualityConstraints/includes/ConstraintCheck/DelegatingConstraintChecker.php: Fix sorting of NullResults (T179038) (duration: 00m 49s) [production]
19:12 <ladsgroup@tin> Synchronized php-1.31.0-wmf.5/extensions/WikibaseQualityConstraints/tests/phpunit/DelegatingConstraintCheckerTest.php: Fix sorting of NullResults (T179038) (duration: 00m 50s) [production]
18:52 <ladsgroup@tin> Synchronized php-1.31.0-wmf.5/extensions/Wikidata/extensions/Constraints/tests/phpunit/DelegatingConstraintCheckerTest.php: Fix sorting of NullResults (T179038) (duration: 00m 49s) [production]
18:51 <ladsgroup@tin> Synchronized php-1.31.0-wmf.5/extensions/Wikidata/extensions/Constraints/includes/ConstraintCheck/DelegatingConstraintChecker.php: Fix sorting of NullResults (T179038) (duration: 01m 04s) [production]
18:41 <bblack> eqiad cp servers: rolling quick depool -> repool around ethtool parameter changes for -lro,-pause [production]
18:30 <urandom> Dropping and recreating keyspace enwiki_T_page__summary (restbase-ng cluster) [production]
18:27 <bblack> esams cp servers: rolling quick depool -> repool around ethtool parameter changes for -lro,-pause [production]
18:26 <urandom> T179105: Altering "enwiki_T_page__summary".data to use LZ4Compressor [production]
17:59 <bblack> codfw cp servers: rolling quick depool -> repool around ethtool parameter changes for -lro,-pause [production]
17:57 <bblack@neodymium> conftool action : set/pooled=no; selector: name=cp4024.ulsfo.wmnet [production]
17:49 <bblack> ulsfo cp servers: rolling quick depool -> repool around ethtool parameter changes for -lro,-pause [production]
17:47 <awight@tin> Finished deploy [ores/deploy@f5deb7f]: Revscoring 2 on ores1002 (non-production) (duration: 01m 45s) [production]
17:45 <awight@tin> Started deploy [ores/deploy@f5deb7f]: Revscoring 2 on ores1002 (non-production) [production]
17:36 <arlolra> Updated Parsoid to 8e99708a (T176728) [production]
17:30 <arlolra@tin> Finished deploy [parsoid/deploy@4882c59]: Updating Parsoid to 8e99708a (duration: 08m 40s) [production]
17:21 <arlolra@tin> Started deploy [parsoid/deploy@4882c59]: Updating Parsoid to 8e99708a [production]
17:17 <awight@tin> Finished deploy [ores/deploy@971be22]: Rolling back scb1002, T175180 T178441 (duration: 00m 32s) [production]
17:16 <awight@tin> Started deploy [ores/deploy@971be22]: Rolling back scb1002, T175180 T178441 [production]
17:14 <awight@tin> Finished deploy [ores/deploy@971be22]: ORES w/ revscoring 2 and Celery 4, T175180 T178441 (duration: 03m 20s) [production]
17:11 <awight@tin> Started deploy [ores/deploy@971be22]: ORES w/ revscoring 2 and Celery 4, T175180 T178441 [production]
17:09 <marostegui> Optimize ruwiki.recentchanges on dbstore1001 - T162789 [production]
17:08 <dcausse> [logstash] reimporting today logs [production]
17:08 <dcausse> [logstash] deleting today logs to reapply mapping template [production]
16:58 <ottomata> now mirroring main Kafka cluster topics to jumbo Kafka cluster, with MirrorMaker instances running on analytics-eqiad broker nodes. https://phabricator.wikimedia.org/T177216 [production]
16:44 <awight@tin> Finished deploy [ores/deploy@971be22]: Push ORES w/ Celery 4 support to new cluster (try to rebuild venv), T178441 (duration: 01m 02s) [production]
16:43 <awight@tin> Started deploy [ores/deploy@971be22]: Push ORES w/ Celery 4 support to new cluster (try to rebuild venv), T178441 [production]