| 
      
        2017-03-02
      
      ยง
     | 
  
    
  | 19:49 | 
  <maxsem@tin> | 
  Finished deploy [tilerator/deploy@0fe5a1d]: Reverting to previous version | 
  [production] | 
            
  | 19:49 | 
  <maxsem@tin> | 
  Started deploy [tilerator/deploy@0fe5a1d]: Reverting to previous version | 
  [production] | 
            
  | 19:46 | 
  <maxsem@tin> | 
  Finished deploy [tilerator/deploy@edb97c5]: https://gerrit.wikimedia.org/r/#/c/340607/ (duration: 00m 05s) | 
  [production] | 
            
  | 19:46 | 
  <maxsem@tin> | 
  Started deploy [tilerator/deploy@edb97c5]: https://gerrit.wikimedia.org/r/#/c/340607/ | 
  [production] | 
            
  | 19:43 | 
  <maxsem@tin> | 
  Finished deploy [tilerator/deploy@edb97c5]: https://gerrit.wikimedia.org/r/#/c/340607/ (duration: 00m 03s) | 
  [production] | 
            
  | 19:43 | 
  <maxsem@tin> | 
  Started deploy [tilerator/deploy@edb97c5]: https://gerrit.wikimedia.org/r/#/c/340607/ | 
  [production] | 
            
  | 19:42 | 
  <maxsem@tin> | 
  Finished deploy [tilerator/deploy@edb97c5]: https://gerrit.wikimedia.org/r/#/c/340607/ (duration: 00m 03s) | 
  [production] | 
            
  | 19:42 | 
  <maxsem@tin> | 
  Started deploy [tilerator/deploy@edb97c5]: https://gerrit.wikimedia.org/r/#/c/340607/ | 
  [production] | 
            
  | 19:42 | 
  <maxsem@tin> | 
  Finished deploy [tilerator/deploy@edb97c5]: https://gerrit.wikimedia.org/r/#/c/340607/ (duration: 00m 23s) | 
  [production] | 
            
  | 19:42 | 
  <maxsem@tin> | 
  Started deploy [tilerator/deploy@edb97c5]: https://gerrit.wikimedia.org/r/#/c/340607/ | 
  [production] | 
            
  | 19:16 | 
  <addshore@tin> | 
  Synchronized dblists/all-labs.dblist: [[gerrit:340559|Add beta hewiktionary]] T158628 2/2 NOOP (duration: 00m 39s) | 
  [production] | 
            
  | 19:15 | 
  <addshore@tin> | 
  Synchronized wikiversions-labs.json: [[gerrit:340559|Add beta hewiktionary]] T158628 1/2 NOOP (duration: 00m 42s) | 
  [production] | 
            
  | 19:06 | 
  <awight> | 
  reenabling donation and recurring queue consumers | 
  [production] | 
            
  | 19:05 | 
  <addshore@tin> | 
  Synchronized wmf-config/throttle.php: [[gerrit:340765|Add new rules for WMUK]] T159454 T159461 (duration: 00m 43s) | 
  [production] | 
            
  | 19:04 | 
  <awight> | 
  update civicrm from fb91fa8abe11dab53402300fc84dd37f79b3c690 to d012767f4f80ea6ca368d9c03aa84473b05ee980 | 
  [production] | 
            
  | 18:21 | 
  <demon@tin> | 
  Synchronized php-1.29.0-wmf.14/includes/changes/EnhancedChangesList.php: T159466 (duration: 00m 40s) | 
  [production] | 
            
  | 17:51 | 
  <bblack> | 
  disabling puppet on authdns prod machines for hacky discovery testing | 
  [production] | 
            
  | 17:44 | 
  <bblack@puppetmaster1001> | 
  conftool action : set/pooled=true; selector: dnsdisc=appservers-ro,name=codfw | 
  [production] | 
            
  | 17:44 | 
  <bblack@puppetmaster1001> | 
  conftool action : set/pooled=true; selector: dnsdisc=appservers-rw,name=eqiad | 
  [production] | 
            
  | 17:38 | 
  <oblivian@puppetmaster1001> | 
  conftool action : set/pooled=true; selector: name=eqiad | 
  [production] | 
            
  | 16:52 | 
  <bblack> | 
  puppet re-enabled on authdns production boxes | 
  [production] | 
            
  | 16:27 | 
  <bblack> | 
  puppet disabled on authdns production boxes, for hacky testing of discovery-related commits | 
  [production] | 
            
  | 16:00 | 
  <jynus> | 
  restarting db1001 for kernel and mariadb upgrade | 
  [production] | 
            
  | 15:49 | 
  <moritzm> | 
  uploaded 6.8.9.9-5+deb8u7+wmf1 to apt.wikimedia.org (CMYK sharpen bugfix rebased on latest Debian update) | 
  [production] | 
            
  | 15:42 | 
  <moritzm> | 
  installing libfcgi-perl security updates | 
  [production] | 
            
  | 14:47 | 
  <phuedx@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: T157700: Re-enable Page Previews instrumentation (duration: 00m 40s) | 
  [production] | 
            
  | 14:37 | 
  <filippo@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=ms-fe1008.eqiad.wmnet | 
  [production] | 
            
  | 14:37 | 
  <filippo@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=ms-fe1007.eqiad.wmnet | 
  [production] | 
            
  | 14:37 | 
  <filippo@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=ms-fe1006.eqiad.wmnet | 
  [production] | 
            
  | 14:32 | 
  <phuedx@tin> | 
  Synchronized portals: (no justification provided) (duration: 00m 41s) | 
  [production] | 
            
  | 14:32 | 
  <phuedx@tin> | 
  Synchronized portals/prod/wikipedia.org/assets: (no justification provided) (duration: 00m 40s) | 
  [production] | 
            
  | 14:26 | 
  <jynus> | 
  running alter table on db2040 T147747 | 
  [production] | 
            
  | 14:22 | 
  <elukey@tin> | 
  Finished deploy [analytics/refinery@c3dd129]: (no justification provided) (duration: 02m 18s) | 
  [production] | 
            
  | 14:20 | 
  <elukey@tin> | 
  Started deploy [analytics/refinery@c3dd129]: (no justification provided) | 
  [production] | 
            
  | 14:12 | 
  <phuedx@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Remove Page Previews experiment config (duration: 00m 40s) | 
  [production] | 
            
  | 14:10 | 
  <phuedx@tin> | 
  Synchronized wmf-config/CommonSettings.php: Remove Page Previews experiment config (duration: 01m 06s) | 
  [production] | 
            
  | 13:47 | 
  <moritzm> | 
  removed obsolete kernels on ocg1002 | 
  [production] | 
            
  | 13:46 | 
  <moritzm> | 
  removed obsolete kernels on eventlog1001 | 
  [production] | 
            
  | 13:03 | 
  <filippo@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=ms-fe1005.eqiad.wmnet | 
  [production] | 
            
  | 12:52 | 
  <moritzm> | 
  installing shadow security updates on jessie hosts | 
  [production] | 
            
  | 12:43 | 
  <jynus> | 
  running ANALYZE table on revision at db1051 (depooled) T159319 | 
  [production] | 
            
  | 12:36 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1051 for maintenance (duration: 00m 42s) | 
  [production] | 
            
  | 11:58 | 
  <hashar> | 
  CI composer based builds are now ok. Only operations/mediawiki-config was impacted as far as I can tell. | 
  [production] | 
            
  | 11:10 | 
  <kartik@tin> | 
  Finished deploy [cxserver/deploy@5101090]: (no justification provided) (duration: 02m 24s) | 
  [production] | 
            
  | 11:07 | 
  <kartik@tin> | 
  Started deploy [cxserver/deploy@5101090]: (no justification provided) | 
  [production] | 
            
  | 10:51 | 
  <hashar> | 
  CI composer based builds are sometime broken since composer got upgraded to 1.1.0 .  See https://phabricator.wikimedia.org/T159431 | 
  [production] | 
            
  | 10:22 | 
  <moritzm> | 
  installing bind updates (we're using client-side libs/tools) | 
  [production] | 
            
  | 10:04 | 
  <moritzm> | 
  installing tiff security updates on trusty hosts (jessie already fixed) | 
  [production] | 
            
  | 09:55 | 
  <elukey> | 
  increased PHP memory_limit on bohrium for Piwik (T154558) | 
  [production] | 
            
  | 09:26 | 
  <moritzm> | 
  installing glibc updates from jessie point release | 
  [production] |