| 
      
        2017-03-14
      
      §
     | 
  
    
  | 18:39 | 
  <gehel> | 
  removing swap from elasticsearch servers - T158884 | 
  [production] | 
            
  | 18:37 | 
  <ottomata> | 
  upgrading librdkafka to 0.9.4 and restarting varnishkafka on cache text hosts | 
  [production] | 
            
  | 18:19 | 
  <ottomata> | 
  upgrading librdkafka to 0.9.4 and restarting varnishkafka on cache upload hosts | 
  [production] | 
            
  | 18:13 | 
  <ottomata> | 
  upgrading librdkafka to 0.9.4 and restarting varnishkafka on cache misc hosts | 
  [production] | 
            
  | 18:11 | 
  <nuria@tin> | 
  Finished deploy [eventlogging/analytics@c3ccb4a]: (no justification provided) (duration: 00m 03s) | 
  [production] | 
            
  | 18:11 | 
  <nuria@tin> | 
  Started deploy [eventlogging/analytics@c3ccb4a]: (no justification provided) | 
  [production] | 
            
  | 17:07 | 
  <ottomata> | 
  upgrading librdkafka to 0.9.4 on cache misc and restarting varnishkafka | 
  [production] | 
            
  | 16:29 | 
  <jynus> | 
  no reponse from db1057 after powercycle- trying to hard reset it | 
  [production] | 
            
  | 16:10 | 
  <urandom> | 
  T111113: Restart Cassandra in RESTBase Staging to enable optional client encryption | 
  [production] | 
            
  | 15:39 | 
  <godog> | 
  shut ms-be2002 for idrac / bios troubleshooting T155689 | 
  [production] | 
            
  | 15:24 | 
  <chasemp> | 
  silence toolschecker precise job start check in anticipation of removal | 
  [production] | 
            
  | 15:18 | 
  <twentyafterfour> | 
  preparing to branch 1.29.0-wmf.16 refs T158997 | 
  [production] | 
            
  | 14:36 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1080 - T132416 (duration: 00m 40s) | 
  [production] | 
            
  | 14:36 | 
  <marostegui> | 
  Enabled parallel replication (5 threads) on db2033 (x1) - T160407 | 
  [production] | 
            
  | 14:20 | 
  <chasemp> | 
  labsdb100[9|10|11] 'maintain-views --all-databases --table page --replace-all --debug' | 
  [production] | 
            
  | 14:18 | 
  <chasemp> | 
  labsdb1003 time maintain-views --all-databases --table page --replace-all --debug | 
  [production] | 
            
  | 14:01 | 
  <Dereckson> | 
  Purged portals URL | 
  [production] | 
            
  | 13:56 | 
  <dereckson@tin> | 
  Synchronized portals: Resync portals/ directory after touch (duration: 00m 42s) | 
  [production] | 
            
  | 13:56 | 
  <chasemp> | 
  labsdb1001 maintain-views --all-databases --table page --replace-all --debug | 
  [production] | 
            
  | 13:46 | 
  <dereckson@tin> | 
  Synchronized portals: Bump to e576c18522ff (duration: 00m 41s) | 
  [production] | 
            
  | 13:45 | 
  <dereckson@tin> | 
  Synchronized portals/prod/wikipedia.org/assets: Bump to e576c18522ff (duration: 00m 41s) | 
  [production] | 
            
  | 13:18 | 
  <elukey> | 
  started redis-cli --bigkeys -i 0.1 on rdb1008 (eqiad jobqueue slave) | 
  [production] | 
            
  | 13:15 | 
  <dereckson@tin> | 
  Synchronized portals: (no justification provided) (duration: 00m 41s) | 
  [production] | 
            
  | 13:14 | 
  <dereckson@tin> | 
  Synchronized portals/prod/wikipedia.org/assets: (no justification provided) (duration: 00m 41s) | 
  [production] | 
            
  | 13:00 | 
  <gehel> | 
  restarting elasticsearch on relforge1001 to test gelf appender | 
  [production] | 
            
  | 12:41 | 
  <elukey> | 
  reimage analytics1043 to Debian Jessie | 
  [production] | 
            
  | 12:32 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1054 with full weight after warmup (duration: 00m 40s) | 
  [production] | 
            
  | 12:28 | 
  <jynus> | 
  stopping mariadb on db1057, preparing to backup and reimage | 
  [production] | 
            
  | 12:24 | 
  <addshore@tin> | 
  Synchronized dblists/: T150183 wmgUseInterwikiSorting true for all wikidata clients [[gerrit:341034|#1]] [[gerrit:342611|#2]] PT 4/4 (duration: 00m 41s) | 
  [production] | 
            
  | 12:23 | 
  <addshore@tin> | 
  Synchronized docroot/: T150183 wmgUseInterwikiSorting true for all wikidata clients [[gerrit:341034|#1]] [[gerrit:342611|#2]] PT 3/4 NOOP (duration: 00m 44s) | 
  [production] | 
            
  | 12:19 | 
  <addshore@tin> | 
  Synchronized wmf-config/CommonSettings.php: T150183 wmgUseInterwikiSorting true for all wikidata clients [[gerrit:341034|#1]] [[gerrit:342611|#2]] PT 2/4 (duration: 00m 41s) | 
  [production] | 
            
  | 12:18 | 
  <addshore@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: T150183 wmgUseInterwikiSorting true for all wikidata clients [[gerrit:341034|#1]] [[gerrit:342611|#2]] PT 1/4 (duration: 00m 52s) | 
  [production] | 
            
  | 09:15 | 
  <addshore@tin> | 
  Synchronized dblists/interwikisorting.dblist: [[gerrit:341033|wmgUseInterwikiSorting true for wikidata clients, excluding wikipedias]] T150183 (duration: 00m 42s) | 
  [production] | 
            
  | 08:38 | 
  <elukey> | 
  moved some log files from /var/log/upstart/$logname.log.1 to /var/log/upstart/$logname.log.1.bis on labvirt1014, labtestvirt2001, labtestnet2001, labnet1001 to reduce cronspam | 
  [production] | 
            
  | 08:15 | 
  <moritzm> | 
  installing icu security updates on trusty (jessie already fixed) | 
  [production] | 
            
  | 08:07 | 
  <moritzm> | 
  installing icoutils security update on trusty (jessie already fixed) | 
  [production] | 
            
  | 07:26 | 
  <moritzm> | 
  installing python-imaging/pillow security updates on trusty (jessie already fixed) | 
  [production] | 
            
  | 07:07 | 
  <marostegui> | 
  Deploy alter table enwiki.revision db1080 - T132416 | 
  [production] | 
            
  | 07:07 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1080 - T132416 (duration: 00m 41s) | 
  [production] | 
            
  | 07:01 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1083 - T132416 (duration: 00m 41s) | 
  [production] | 
            
  | 02:31 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.29.0-wmf.15) (duration: 12m 36s) | 
  [production] | 
            
  
    | 
      
        2017-03-13
      
      §
     | 
  
    
  | 23:32 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1054 after upgrade with low weight (duration: 00m 41s) | 
  [production] | 
            
  | 22:29 | 
  <bawolff@tin> | 
  Synchronized php-1.29.0-wmf.15/extensions/SemanticForms/includes/SF_ValuesUtils.php: Backport bb42c6f401b9 (duration: 00m 48s) | 
  [production] | 
            
  | 21:40 | 
  <bawolff> | 
  Deployed fix for T160266 | 
  [production] | 
            
  | 20:45 | 
  <addshore> | 
  InterwikiSorting deploy (to group0) done | 
  [production] | 
            
  | 20:43 | 
  <addshore@tin> | 
  Synchronized wmf-config/CommonSettings.php: T150183 Enable InterwikiSorting on group0 [[gerrit:341032|#1]] [[gerrit:342504|#2]] PT 4/4 (duration: 00m 40s) | 
  [production] | 
            
  | 20:42 | 
  <addshore@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: T150183 Enable InterwikiSorting on group0 [[gerrit:341032|#1]] [[gerrit:342504|#2]] PT 3/4 (duration: 00m 41s) | 
  [production] | 
            
  | 20:41 | 
  <addshore@tin> | 
  Synchronized docroot/noc/conf/interwikisorting.dblist: T150183 Enable InterwikiSorting on group0 [[gerrit:341032|#1]] [[gerrit:342504|#2]] PT 2/4 NOOP (duration: 00m 42s) | 
  [production] | 
            
  | 20:39 | 
  <addshore@tin> | 
  Synchronized dblists/interwikisorting.dblist: T150183 Enable InterwikiSorting on group0 [[gerrit:341032|#1]] [[gerrit:342504|#2]] PT 1/4 (duration: 00m 51s) | 
  [production] | 
            
  | 18:37 | 
  <dcausse@tin> | 
  Synchronized php-1.29.0-wmf.15/extensions/CirrusSearch/: Make incoming link counting compatible with 5.x (duration: 00m 53s) | 
  [production] |