5301-5350 of 10000 results (65ms)
2019-01-08 ยง
22:20 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: dblists: Load wikibaserepo (duration: 00m 52s) [production]
22:15 <jforrester@deploy1001> scap failed: average error rate on 9/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) [production]
22:14 <jforrester@deploy1001> Synchronized dblists/wikidata.dblist: dblists: Remove testcommons from wikidata list (duration: 00m 52s) [production]
22:13 <jforrester@deploy1001> Synchronized dblists/wikidatarepo.dblist: dblists: Add wikidatarepo list (duration: 00m 53s) [production]
22:12 <urandom> forcing removal of restbase1016-b (host down way too long to salvage) -- T212418 [production]
22:08 <marostegui> Drop valid_tag table from db2043 with replication (s3 codfw master - lag will be generated) - T212254 [production]
22:03 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: cleanup - Idfa129a65a41 (duration: 00m 53s) [production]
21:53 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1078 T212254 (duration: 00m 52s) [production]
21:49 <marostegui> Drop valid_tag table from db1078 (s3) - T212254 [production]
21:49 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1078 T212254 (duration: 00m 53s) [production]
21:43 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1123 T212254 (duration: 00m 53s) [production]
21:38 <marostegui> Drop valid_tag table from db1123 (s3) - T212254 [production]
21:38 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1123 T212254 (duration: 00m 53s) [production]
21:31 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: group0 to 1.33.0-wmf.12 [production]
21:03 <dduvall@deploy1001> Finished scap: testwiki to php-1.33.0-wmf.12 and rebuild l10n cache (duration: 39m 22s) [production]
20:42 <ejegg> updated payments-wiki from b8acb95a2a to c455bbc6bb [production]
20:24 <dduvall@deploy1001> Started scap: testwiki to php-1.33.0-wmf.12 and rebuild l10n cache [production]
20:24 <gehel> starting data copy from wdqs1004 to wdqs1007 (both will be depooled) - T213217 [production]
20:21 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: TestCommons: Don't enable entities, we're not Wikidata.org (duration: 01m 44s) [production]
20:11 <XioNoX> change BGP_fundraising_aggregates term nat from static to aggregate on pfw3-eqiad - T211028 [production]
19:51 <ejegg> updated fundraising CiviCRM from b8e3a71845 to 5580f0b11c [production]
19:48 <krinkle@deploy1001> Finished deploy [performance/navtiming@68fd54d]: (no justification provided) (duration: 00m 05s) [production]
19:48 <krinkle@deploy1001> Started deploy [performance/navtiming@68fd54d]: (no justification provided) [production]
19:48 <dduvall@deploy1001> Pruned MediaWiki: 1.33.0-wmf.12 (duration: 06m 26s) [production]
19:11 <arlolra> Updated Parsoid to 2c5dc7b (T197616, T205491, T209772, T199926, T209194, T204622) [production]
19:06 <marostegui> Drop valid_tag table from s1 - T212254 [production]
19:00 <arlolra@deploy1001> Finished deploy [parsoid/deploy@4b82683]: Updating Parsoid to 2c5dc7b (duration: 10m 40s) [production]
18:54 <XioNoX> make pfw3-codfw source NAT similar to pfw3-eqiad - T211028 [production]
18:54 <ejegg> updated SmashPig standalone install from fb3268897b to 25713ca232 [production]
18:50 <marostegui> Drop valid_tag table from s4 - T212254 [production]
18:50 <XioNoX> add NAT workaround to pfw3-eqiad - T211028 [production]
18:49 <arlolra@deploy1001> Started deploy [parsoid/deploy@4b82683]: Updating Parsoid to 2c5dc7b [production]
18:38 <XioNoX> temporarily permit ssh from frpm1001 to pfw3-eqiad on pfw3-eqiad [production]
18:28 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1105:3311 T86338 T202167 (duration: 00m 45s) [production]
18:27 <jynus> restarting s5 replication on labsdb1009/10/11 [production]
17:41 <moritzm> installing libseccomp updates from stretch point release [production]
17:40 <mobrovac@deploy1001> Finished deploy [restbase/deploy@503b29c]: Add test-commons and nap.wikisource, take #2 (duration: 02m 29s) [production]
17:38 <mobrovac@deploy1001> Started deploy [restbase/deploy@503b29c]: Add test-commons and nap.wikisource, take #2 [production]
17:37 <mobrovac@deploy1001> Finished deploy [restbase/deploy@503b29c]: Add test-commons and nap.wikisource - T210752 T197616 (duration: 96m 50s) [production]
17:33 <_joe_> applying the new apache configuration to jobrunners in eqiad [production]
17:24 <elukey> roll restart of aqs on aqs100* to pick up new Druid settings [production]
17:20 <_joe_> depooling mw1299 for testing of the apache change [production]
17:16 <SMalyshev> restarted Blazegraph wdqs1006 due to unresponsiveness (caused by load?) [production]
16:56 <urandom> forcing removal of restbase1016-a (host down way too long to salvage) -- T212418 [production]
16:56 <jynus> changing db1124:s5 replication to db2066 [production]
16:54 <marostegui> Deploy schema change on db1105:3311 T86338 T202167 [production]
16:54 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1105:3311 T86338 T202167 (duration: 00m 44s) [production]
16:54 <jynus> stopping s5 replication on labsdb1009/10/11 to prevent undoable mistakes [production]
16:34 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool es2019 - T212833 (duration: 02m 51s) [production]
16:29 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1089 T86338 T202167 (duration: 00m 45s) [production]