501-550 of 10000 results (25ms)
2015-09-17 ยง
19:46 <krenair@tin> Synchronized multiversion/MWMultiVersion.php: (no message) (duration: 00m 12s) [production]
19:41 <krenair@tin> Synchronized multiversion/MWMultiVersion.php: https://gerrit.wikimedia.org/r/#/c/239181/ (duration: 00m 14s) [production]
19:12 <mutante> powercycling unresponse mw1005 [production]
18:14 <twentyafterfour@tin> rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedia wikis to 1.26wmf23 [production]
17:38 <legoktm@tin> Synchronized php-1.26wmf22/includes/registration/ExtensionRegistry.php: registration: Fix merging of array_plus (duration: 00m 13s) [production]
17:35 <legoktm@tin> Synchronized php-1.26wmf23/includes/registration/ExtensionRegistry.php: registration: Fix merging of array_plus (duration: 00m 11s) [production]
16:43 <chasemp> restart elasticsearch on 1005 [production]
16:16 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/235900/ (duration: 00m 12s) [production]
15:15 <dcausse> elastic in eqiad plugin updates: restarting elastic1004 (take 2) [production]
15:06 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: CX: Enable Suggestions in ptwiki [[gerrit:238097]] (duration: 00m 13s) [production]
14:22 <mutante> analytics1029 - Failed to start Hadoop datanode [production]
14:20 <mutante> starting hadoop datanode on analytics1029 [production]
14:14 <_joe_> reimaging tmh1001 to mw1259 [production]
14:11 <jynus> stopping replication and applying schema change to db1051 [production]
14:05 <dcausse> elastic in eqiad plugin updates: can't restart elastic1004 (2 timeouts when disabling replication, too much load?), waiting for more shards to rebalance... [production]
13:58 <dcausse> elastic in eqiad plugin updates: restarting elastic1004 [production]
13:50 <moritzm> repooled mw1236-mw1239 (T104968) [production]
13:34 <moritzm> depooled mw1236-mw1239 (T104968) [production]
13:26 <moritzm> repooled mw1090-mw1099 (T104968) [production]
13:16 <moritzm> depooled mw1090-mw1099 (T104968) [production]
13:13 <moritzm> repooled mw1080-mw1089 (T104968) [production]
13:05 <moritzm> depooled mw1080-mw1089 (T104968) [production]
13:01 <moritzm> repooled mw1070-mw1079 (T104968) [production]
12:49 <moritzm> depooled mw1070-mw1079 (T104968) [production]
12:35 <moritzm> repooled mw1060 and mw1062-mw1069 (T104968) [production]
12:24 <moritzm> depooled mw1060 and mw1062-mw1069 (T104968) (not repooled) [production]
12:24 <moritzm> repooled mw1060 and mw1062-mw1069 (T104968) [production]
12:16 <moritzm> repooled mw1050-mw1059 [production]
12:04 <moritzm> depooled mw1050-mw1059 [production]
11:39 <moritzm> repooled mw1040 and mw1042-mw1049 (T104968) [production]
11:36 <dcausse> elastic in eqiad plugin updates: restarting elastic1003 [production]
11:26 <moritzm> typoed earlier entry: "mw1032-mw1039" instead of "mw1032-mw1239" [production]
11:26 <moritzm> depooled mw1040 and mw1042-mw1049 (T104968) [production]
11:18 <moritzm> repooled mw1030 and mw1032-mw1239 (T104968) [production]
11:03 <moritzm> depooled mw1030 and mw1032-mw1239 (T104968) [production]
10:35 <moritzm> repooled mw1250-mw1258 (T104968) [production]
10:27 <moritzm> depooled mw1250-mw1258 (T104968) [production]
10:24 <_joe_> killing temporarily subra [production]
10:24 <moritzm> repooled mw1240-mw1249 (T104968) [production]
10:19 <_joe_> experimenting with poolcounter issues on subra [production]
10:18 <oblivian@tin> Synchronized wmf-config/PoolCounterSettings-codfw.php: Use codfw poolcounters in codfw (duration: 00m 12s) [production]
10:12 <moritzm> depooled mw1240-mw1249 (T104968) [production]
10:11 <dcausse> elastic in eqiad plugin updates: restarting elastic1002 [production]
10:05 <hoo@tin> Synchronized wmf-config/: Set 'repoConceptBaseUri' for all Wikibase clients (duration: 00m 13s) [production]
10:00 <dcausse> elastic in eqiad plugin updates: unfreezing indices [production]
09:48 <dcausse> elastic in eqiad plugin updates: no more groovy in warmers, waiting for few more shards to move in elastic1001 and will unfreeze indices to test warmers [production]
09:39 <dcausse> elastic in eqiad plugin updates: deleting warmers manually for old unused indices (eswikisource_content_1415240352, ruwiki_content_1415302164, thwiki_content_1415318677). We will have to remove these indices. [production]
09:39 <paravoid> repooling ulsfo US-West traffic back to ulsfo for the first time since May :) [production]
09:01 <dcausse> elastic in eqiad plugin updates: updating warmers on all wikis [production]
08:58 <paravoid> penalizing ulsfo-eqiad direct MPLS links to higher OSPF weights [production]