| 2014-10-17
      
      § | 
    
  | 23:46 | <bd808> | Ran trebuchet to create initial tag for iegreview/iegreview | [production] | 
            
  | 23:35 | <ori> | pooled mw1114 (hhvm api server) to test whether new package resolves overload behavior | [production] | 
            
  | 23:29 | <ori> | updating hhvm app servers to 3.3.0-20140925+wmf3 | [production] | 
            
  | 22:18 | <mutante> | graceful Apache on antimony - svn fixed, gitblit behind varnish | [production] | 
            
  | 22:09 | <mutante> | graceful Apache on neon - icinga and tendril done, ishmael = misc-web | [production] | 
            
  | 21:47 | <mutante> | graceful Apache on stat1001 | [production] | 
            
  | 21:25 | <springle> | Synchronized wmf-config/db-eqiad.php: repool es1004, es1007, es1010 (duration: 00m 07s) | [production] | 
            
  | 21:19 | <mutante> | graceful Apache on netmon1001 | [production] | 
            
  | 21:13 | <mutante> | graceful Apache on puppetmaster | [production] | 
            
  | 20:34 | <andrewbogott> | removed some stray .zip files from /tmp on ocg1002 | [production] | 
            
  | 20:16 | <ori> | disabled puppet on osmium to debug hhvm | [production] | 
            
  | 15:36 | <paravoid> | killed tampa config remnants on all cr1/cr2s | [production] | 
            
  | 15:30 | <bblack> | restarted puppetmasters | [production] | 
            
  | 14:44 | <_joe_> | load test on hhvm done | [production] | 
            
  | 14:26 | <_joe_> | load test on the hhvm cluster | [production] | 
            
  | 10:41 | <akosiaris> | uploaded apertium 3.3 on apt.wikimedia.org (trusty-wikimedia) | [production] | 
            
  | 09:17 | <_joe_> | manually killed long-running stuck processes on ocg1001, moving to the rest of the cluster | [production] | 
            
  | 08:56 | <_joe_> | restarted the ocg cluster | [production] | 
            
  | 08:44 | <akosiaris> | uploaded cg3 and lttoolbox on apt.wikimedia.org | [production] | 
            
  | 05:37 | <_joe_> | depooling both hhvm api appservers | [production] | 
            
  | 04:52 | <springle> | xtrabackup clone es1010 to es2008 | [production] | 
            
  | 04:51 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Fri Oct 17 04:51:02 UTC 2014 (duration 51m 1s) | [production] | 
            
  | 04:17 | <springle> | Synchronized wmf-config/db-eqiad.php: reduce load on db1059 (duration: 00m 21s) | [production] | 
            
  | 04:05 | <springle> | upgrade es1010 to trusty (clone failed, needs trusty) | [production] | 
            
  | 03:54 | <springle> | xtrabackup clone es1010 to es2008 | [production] | 
            
  | 03:53 | <springle> | xtrabackup clone es1007 to es2006 | [production] | 
            
  | 03:37 | <springle> | Synchronized wmf-config/db-eqiad.php: depool es1007 and es1010 (duration: 00m 09s) | [production] | 
            
  | 03:13 | <LocalisationUpdate> | completed (1.25wmf4) at 2014-10-17 03:13:33+00:00 | [production] | 
            
  | 02:39 | <LocalisationUpdate> | completed (1.25wmf3) at 2014-10-17 02:39:26+00:00 | [production] | 
            
  | 01:16 | <mutante> | powering up server formerly known as cp1001 | [production] | 
            
  | 00:47 | <hoo> | Synchronized php-1.25wmf4/extensions/Wikidata/: Fix ORMTable usage, IE 11 freeze bug and adopt to further core changes (duration: 00m 14s) | [production] | 
            
  
    | 2014-10-16
      
      § | 
    
  | 22:15 | <K4-713> | undo last payments settings change | [production] | 
            
  | 22:09 | <K4-713> | payments settings updates | [production] | 
            
  | 20:15 | <reedy> | Synchronized database lists: (no message) (duration: 00m 18s) | [production] | 
            
  | 20:15 | <reedy> | Synchronized wmf-config/: (no message) (duration: 00m 17s) | [production] | 
            
  | 20:10 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.24wmf4 | [production] | 
            
  | 20:07 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedias to 1.25wmf3 | [production] | 
            
  | 20:06 | <reedy> | Synchronized database lists: (no message) (duration: 00m 19s) | [production] | 
            
  | 20:06 | <reedy> | Finished scap: testwiki to 1.25wmf4 (duration: 46m 21s) | [production] | 
            
  | 19:36 | <K4-713> | payments localsettings updates - supported countries and fraud filter settings | [production] | 
            
  | 19:19 | <reedy> | Started scap: testwiki to 1.25wmf4 | [production] | 
            
  | 19:09 | <K4-713> | updated payments to 14c415fcfc3cade9a1 | [production] | 
            
  | 18:22 | <ottomata> | restarted varnishkafka on cp3021 | [production] | 
            
  | 17:47 | <ejegg> | Synchronized php-1.25wmf2/extensions/CentralNotice/: Update CentralNotice (duration: 00m 04s) | [production] | 
            
  | 17:29 | <ejegg> | Synchronized php-1.25wmf3/extensions/CentralNotice/: Update CentralNotice (duration: 00m 08s) | [production] | 
            
  | 17:29 | <bblack> | depooled mw1114 for api | [production] | 
            
  | 15:31 | <marktraceur> | Synchronized wmf-config/Wikibase.php: [SWAT] Define client CSS classes for new wikidata badges (duration: 00m 05s) | [production] | 
            
  | 15:30 | <marktraceur> | Sorry, that was in fact adding NS_PROPERTY to the search configuration, mistyped. | [production] |