| 
      
        2016-02-11
      
      §
     | 
  
    
  | 01:58 | 
  <tgr@mira> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.13 again | 
  [production] | 
            
  | 01:47 | 
  <mutante> | 
  labservices1001 - out of disk | 
  [production] | 
            
  | 01:46 | 
  <mutante> | 
  ms-be1008 - delete/gzip large syslog, out of disk | 
  [production] | 
            
  | 01:41 | 
  <tgr@mira> | 
  Synchronized wmf-config/mobile.php: fix for T49647 (duration: 02m 15s) | 
  [production] | 
            
  | 01:05 | 
  <krenair@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/269883/1 (duration: 02m 11s) | 
  [production] | 
            
  | 00:53 | 
  <krenair@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/263051/ (duration: 02m 13s) | 
  [production] | 
            
  | 00:48 | 
  <krenair@mira> | 
  Synchronized w/static/images/project-logos/wuuwiki.png: https://gerrit.wikimedia.org/r/#/c/263051/ (duration: 02m 13s) | 
  [production] | 
            
  | 00:44 | 
  <krenair@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264937/ (duration: 02m 15s) | 
  [production] | 
            
  | 00:35 | 
  <krenair@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/269827/1 (duration: 02m 13s) | 
  [production] | 
            
  | 00:32 | 
  <krenair@mira> | 
  Synchronized w/static/images/project-logos/hiwikiquote.png: https://gerrit.wikimedia.org/r/#/c/269827/1 (duration: 02m 13s) | 
  [production] | 
            
  | 00:12 | 
  <krenair@mira> | 
  Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/259439/ (duration: 02m 20s) | 
  [production] | 
            
  
    | 
      
        2016-02-10
      
      §
     | 
  
    
  | 23:49 | 
  <tgr> | 
  switched mw1017 to wmf.13 (all groups) | 
  [production] | 
            
  | 23:38 | 
  <ori@mira> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis back to php-1.27.0-wmf.12 | 
  [production] | 
            
  | 23:34 | 
  <ori> | 
  Restarted HHVM on mw1017 | 
  [production] | 
            
  | 23:27 | 
  <urandom> | 
  performing rolling restart of Cassandra in restbase staging (experimental gc settings) | 
  [production] | 
            
  | 23:15 | 
  <ori@mira> | 
  Synchronized wmf-config/mobile.php: I6946eccf9c: Better hack for T49647 (duration: 02m 19s) | 
  [production] | 
            
  | 23:10 | 
  <ori@tin> | 
  Synchronized php-1.27.0-wmf.12/includes/WebResponse.php: I13fcc3ce4: Allow changing cookie options in WebResponseSetCookie hook (duration: 01m 30s) | 
  [production] | 
            
  | 23:08 | 
  <ori@tin> | 
  Synchronized php-1.27.0-wmf.13/includes/WebResponse.php: I13fcc3ce4: Allow changing cookie options in WebResponseSetCookie hook (duration: 01m 37s) | 
  [production] | 
            
  | 22:49 | 
  <krinkle@mira> | 
  Synchronized w/static.php: (no message) (duration: 02m 18s) | 
  [production] | 
            
  | 22:40 | 
  <krenair@mira> | 
  Synchronized php-1.27.0-wmf.13/extensions/MobileFrontend/resources/mobile.loggingSchemas/SchemaEdit.js: https://gerrit.wikimedia.org/r/#/c/269755/ (duration: 02m 23s) | 
  [production] | 
            
  | 22:36 | 
  <urandom> | 
  rolling restart of Cassandra staging complete (experimental gc settings) | 
  [production] | 
            
  | 22:35 | 
  <krenair@mira> | 
  Synchronized php-1.27.0-wmf.12/extensions/MobileFrontend/resources/mobile.loggingSchemas/SchemaEdit.js: https://gerrit.wikimedia.org/r/#/c/269848/ (duration: 02m 18s) | 
  [production] | 
            
  | 22:32 | 
  <yurik> | 
  deployed and restarted kartotherian services | 
  [production] | 
            
  | 22:28 | 
  <urandom> | 
  performing rolling restart of Cassandra in restbase staging (experimental gc settings) | 
  [production] | 
            
  | 22:23 | 
  <yurik> | 
  deployed and restarted tilerator & tileratorui services | 
  [production] | 
            
  | 22:15 | 
  <ori> | 
  Restarted apache on palladium and strontium | 
  [production] | 
            
  | 21:42 | 
  <urandom> | 
  ephemerally lowering compactor thread count on restbase1002 from 10 to 8 (limit combined working space) | 
  [production] | 
            
  | 21:21 | 
  <subbu> | 
  finished deploying parsoid version 8976ab93 | 
  [production] | 
            
  | 21:16 | 
  <hashar> | 
  CI dust have settled.  Krinkle and I have pooled a lot more Trusty slaves to accommodate for the overload caused by switching to php55 (jobs run on Trusty) | 
  [production] | 
            
  | 21:14 | 
  <subbu> | 
  synced code + restarted parsoid on wtp1001 as a canary | 
  [production] | 
            
  | 21:10 | 
  <subbu> | 
  starting parsoid deploy  | 
  [production] | 
            
  | 20:24 | 
  <chasemp> | 
  tc per client shaping for labstore1001 test | 
  [production] | 
            
  | 20:21 | 
  <demon@mira> | 
  Synchronized php-1.27.0-wmf.13/includes/interwiki/Interwiki.php: fix cache stuff (duration: 02m 18s) | 
  [production] | 
            
  | 20:17 | 
  <apergos> | 
  cleaned up integrations slave trusty 1001,10012,10013, 1016, missed in first round.  | 
  [production] | 
            
  | 20:10 | 
  <demon@mira> | 
  rebuilt wikiversions.php and synchronized wikiversions files: rebuild | 
  [production] | 
            
  | 20:10 | 
  <demon@mira> | 
  Synchronized multiversion/MWWikiversions.php: rm newline addition (duration: 02m 19s) | 
  [production] | 
            
  | 20:03 | 
  <demon@mira> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.13 | 
  [production] | 
            
  | 19:45 | 
  <apergos> | 
  did cleanup across all integration slaves, some were very close to out of room. results: https://phabricator.wikimedia.org/P2587 | 
  [production] | 
            
  | 19:26 | 
  <mutante> | 
  ms-be1008 - powercycling - the known XFS issue | 
  [production] | 
            
  | 19:04 | 
  <mobrovac> | 
  restbase disabled puppet in staging, testing brotli compression which requires JAVA_OPTS tuning | 
  [production] | 
            
  | 18:30 | 
  <mutante> | 
  puppetstoredconfigclean.rb iodine.wikimedia.org, revoke puppet cert, delete salt key on new master | 
  [production] | 
            
  | 18:15 | 
  <mutante> | 
  iodine - shutdown, decom | 
  [production] | 
            
  | 18:10 | 
  <mutante> | 
  iodine - schedule downtime, stop puppet, stop salt, .. | 
  [production] | 
            
  | 17:21 | 
  <demon@mira> | 
  Synchronized multiversion/MWWikiversions.php: newlines in wikiversion.json (duration: 02m 21s) | 
  [production] | 
            
  | 17:15 | 
  <demon@mira> | 
  Synchronized errorpages/404.html: minor html fix (duration: 02m 17s) | 
  [production] | 
            
  | 17:12 | 
  <thcipriani@mira> | 
  Finished scap: VE, Cite, and Citoid bumps [[gerrit:269592]] [[gerrit:269593]] [[gerrit:269575]] [[gerrit:269590]] (duration: 29m 53s) | 
  [production] | 
            
  | 17:02 | 
  <elukey> | 
  readded mc1006 back into redis/memcached pool | 
  [production] | 
            
  | 16:42 | 
  <thcipriani@mira> | 
  Started scap: VE, Cite, and Citoid bumps [[gerrit:269592]] [[gerrit:269593]] [[gerrit:269575]] [[gerrit:269590]] | 
  [production] | 
            
  | 16:35 | 
  <thcipriani@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: Revert "Enable Math extension on Wikitech" (duration: 02m 14s) | 
  [production] | 
            
  | 16:24 | 
  <thcipriani@mira> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable Math extension on Wikitech [[gerrit:269694]] (duration: 02m 14s) | 
  [production] |