| 
      
        2016-12-02
      
      §
     | 
  
    
  | 18:48 | 
  <godog> | 
  deploy thumbor 0.1.30 to thumbor100[12] | 
  [production] | 
            
  | 17:18 | 
  <mobrovac> | 
  restbase deploy end of 1651e35 | 
  [production] | 
            
  | 16:49 | 
  <mobrovac> | 
  restbase deploy start of 1651e35 | 
  [production] | 
            
  | 15:39 | 
  <mobrovac@tin> | 
  Finished deploy [changeprop/deploy@8f53dc6]: (no message) (duration: 00m 54s) | 
  [production] | 
            
  | 15:38 | 
  <mobrovac@tin> | 
  Starting deploy [changeprop/deploy@8f53dc6]: (no message) | 
  [production] | 
            
  | 15:02 | 
  <_joe_> | 
  rolling restart of API appservers to catch up with the new jemalloc arenas config T151702 | 
  [production] | 
            
  | 14:44 | 
  <oblivian@tin> | 
  Synchronized php-1.29.0-wmf.4/api.php: Reverting the API block after template has been protected (duration: 00m 45s) | 
  [production] | 
            
  | 14:33 | 
  <marostegui> | 
  Deploy alter table wikidatawiki.revision in db2066 - T150644 | 
  [production] | 
            
  | 13:45 | 
  <marostegui> | 
  Deploy alter table wikidatawiki.revision in db2059 -T150644 | 
  [production] | 
            
  | 13:23 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1060 (duration: 00m 45s) | 
  [production] | 
            
  | 11:56 | 
  <jynus> | 
  mysql restart for db1060 T152188 | 
  [production] | 
            
  | 11:51 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Really depool db1060 && pool db1074 with full load after warmup (duration: 00m 44s) | 
  [production] | 
            
  | 11:40 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1060 (duration: 00m 45s) | 
  [production] | 
            
  | 11:39 | 
  <marostegui> | 
  Stop MySQL db1095 for maintenance - T150802 | 
  [production] | 
            
  | 11:21 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1071 - T148967 (duration: 00m 44s) | 
  [production] | 
            
  | 11:11 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1074 with low load (duration: 00m 45s) | 
  [production] | 
            
  | 11:10 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1071 - T148967 (duration: 00m 45s) | 
  [production] | 
            
  | 10:36 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool db2045 - T150644 (duration: 00m 44s) | 
  [production] | 
            
  | 10:34 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1076 with full load (duration: 00m 44s) | 
  [production] | 
            
  | 10:30 | 
  <marostegui> | 
  Deploy alter table wikidatawiki.revision in db2052 -T150644 | 
  [production] | 
            
  | 10:28 | 
  <jynus> | 
  mysql restart and upgrade for db1074 T152188 | 
  [production] | 
            
  | 10:09 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1074 (duration: 00m 45s) | 
  [production] | 
            
  | 09:52 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1076 with low load (duration: 00m 49s) | 
  [production] | 
            
  | 09:18 | 
  <jynus> | 
  mysql restart and upgrade for db1076 T152188 | 
  [production] | 
            
  | 09:18 | 
  <marostegui> | 
  Deploy alter table wikidatawiki.revision in db2045 -T150644 | 
  [production] | 
            
  | 09:16 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Depool db2045 - T150644 (duration: 00m 47s) | 
  [production] | 
            
  | 09:03 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1076 (duration: 00m 48s) | 
  [production] | 
            
  | 08:43 | 
  <oblivian@tin> | 
  Synchronized php-1.29.0-wmf.4/api.php: API bandaid (duration: 00m 48s) | 
  [production] | 
            
  | 08:37 | 
  <elukey> | 
  restarting hhvm (/usr/local/bin/restart-hhvm) on G@cluster:api_appserver and G@site:eqiad (batch 10%) | 
  [production] | 
            
  | 07:59 | 
  <marostegui> | 
  Deploy alter table db1071 - dewiki.revision - T148967 | 
  [production] | 
            
  | 07:51 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1071 - T148967 (duration: 02m 22s) | 
  [production] | 
            
  | 05:55 | 
  <mutante> | 
  thumbor - also got upgraded to imagemagick deb8u6+wmf1 | 
  [production] | 
            
  | 05:49 | 
  <mutante> | 
  imagescalers - upgraded imagemagick 8:6.8.9.9-5+deb8u5+wmf1 -> 8:6.8.9.9-5+deb8u6+wmf1  (https://www.debian.org/security/2016/dsa-3726) | 
  [production] | 
            
  | 03:39 | 
  <mutante> | 
  mw1293 - upgrade imagemagick to 8:6.8.9.9-5+deb8u6+wmf1 | 
  [production] | 
            
  | 00:47 | 
  <ebernhardson@tin> | 
  Synchronized php-1.29.0-wmf.4/extensions/PageImages/maintenance/initImageData.php: T152155: Maintenance script updates for re-initializing page images (duration: 00m 44s) | 
  [production] | 
            
  | 00:41 | 
  <demon@tin> | 
  Synchronized wmf-config: Removing some old ExtensionMessages files (duration: 00m 47s) | 
  [production] | 
            
  | 00:30 | 
  <aaron@tin> | 
  Synchronized wmf-config/CommonSettings.php: Bump $wgJobBackoffThrottling for cache purges (duration: 00m 45s) | 
  [production] | 
            
  | 00:20 | 
  <ebernhardson@tin> | 
  Synchronized php-1.29.0-wmf.4/extensions/PageImages/includes/ApiQueryPageImages.php: T152155: Thumbnails are not showing in search on multiple platforms (duration: 00m 45s) | 
  [production] | 
            
  | 00:00 | 
  <bsitzmann@tin> | 
  Finished deploy [mobileapps/deploy@b545699]: Update mobileapps to 04a6e84 (duration: 01m 17s) | 
  [production] | 
            
  
    | 
      
        2016-12-01
      
      §
     | 
  
    
  | 23:58 | 
  <bsitzmann@tin> | 
  Starting deploy [mobileapps/deploy@b545699]: Update mobileapps to 04a6e84 | 
  [production] | 
            
  | 22:12 | 
  <godog> | 
  upgrade scap to 3.4.1-1 on tin and mira | 
  [production] | 
            
  | 22:07 | 
  <Krenair> | 
  Dropped ar_usertext_timestamp indexes from the archive tables of all 4 wikis created since September (olowiki, ecwikimedia, projectcomwiki, fiwikivoyage) - replaced with usertext_timestamp index to match all older wikis. MW fix to follow. see -tech | 
  [production] | 
            
  | 21:15 | 
  <ottomata> | 
  rolling bounce of main kafka brokers and then eventbus service  to pick up api_version change, and to apply min.insync.replicas=1 to kafka | 
  [production] | 
            
  | 21:01 | 
  <ottomata> | 
  bouncing kafka broker on kafka1002 to troubleshoot production only missing messages | 
  [production] | 
            
  | 20:55 | 
  <otto@tin> | 
  Finished deploy [eventlogging/eventbus@948765d]: accept api_version parameter (duration: 00m 09s) | 
  [production] | 
            
  | 20:55 | 
  <otto@tin> | 
  Starting deploy [eventlogging/eventbus@948765d]: accept api_version parameter | 
  [production] | 
            
  | 20:53 | 
  <otto@tin> | 
  Finished deploy [eventlogging/eventbus@948765d]: (no message) (duration: 00m 08s) | 
  [production] | 
            
  | 20:53 | 
  <otto@tin> | 
  Starting deploy [eventlogging/eventbus@948765d]: (no message) | 
  [production] | 
            
  | 20:27 | 
  <ottomata> | 
  bouncing kafka broker on kafka1018 to test config changes to eventlogging analytics kafka clients | 
  [production] | 
            
  | 20:03 | 
  <demon@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group2 to wmf.4 | 
  [production] |