| 
      
        2012-12-11
      
      §
     | 
  
    
  | 07:42 | 
  <aaron> | 
  synchronized php-1.21wmf6/includes/job/JobQueueDB.php  'deployed 26b9a472df430dc3abaead0e6f5a9f640b075b99' | 
  [production] | 
            
  | 07:34 | 
  <binasher> | 
  restarting all job runners | 
  [production] | 
            
  | 07:32 | 
  <Aaron|home> | 
  Also deployed 919b9d8ba3ee7d6a16214b0853e82fa733a0605a and  dcd314e837ccf3d2f7c2ca7dc94b7d1e6accfde4 | 
  [production] | 
            
  | 07:31 | 
  <aaron> | 
  synchronized php-1.21wmf5/includes/job/JobQueueDB.php  'reverted live hacks to use slave DB' | 
  [production] | 
            
  | 07:02 | 
  <catrope> | 
  synchronized php-1.21wmf6/extensions/VisualEditor/modules/ve/init/mw/targets/ve.init.mw.ViewPageTarget.js  'touch' | 
  [production] | 
            
  | 07:02 | 
  <catrope> | 
  synchronized php-1.21wmf5/extensions/VisualEditor/modules/ve/init/mw/targets/ve.init.mw.ViewPageTarget.js  'touch' | 
  [production] | 
            
  | 07:00 | 
  <olivneh> | 
  synchronized php-1.21wmf5/extensions/E3Experiments/Experiments.php | 
  [production] | 
            
  | 06:56 | 
  <olivneh> | 
  synchronized php-1.21wmf5/extensions/EventLogging  'Syncing to avoid obsolete CACHE_MEMCACHE use in prod.' | 
  [production] | 
            
  | 06:49 | 
  <asher> | 
  synchronized wmf-config/db.php  'pulling db39 from s3, crazy behind on purging' | 
  [production] | 
            
  | 05:52 | 
  <aaron> | 
  synchronized php-1.21wmf5/includes/objectcache/MemcachedPhpBagOStuff.php | 
  [production] | 
            
  | 05:52 | 
  <aaron> | 
  synchronized php-1.21wmf5/includes/objectcache/MemcachedPhpBagOStuff.php | 
  [production] | 
            
  | 05:22 | 
  <binasher> | 
  killed all wikiadmin threads on db39 | 
  [production] | 
            
  | 05:11 | 
  <catrope> | 
  synchronized php-1.21wmf6/extensions/VisualEditor  'Updating VisualEditor to master' | 
  [production] | 
            
  | 05:11 | 
  <catrope> | 
  synchronized php-1.21wmf5/extensions/VisualEditor  'Updating VisualEditor to master' | 
  [production] | 
            
  | 04:41 | 
  <binasher> | 
  same with db66 | 
  [production] | 
            
  | 04:38 | 
  <binasher> | 
  killed 22 long running FlaggedRevsStats::getEditReviewTimes on db39, run from hume against enwikinews | 
  [production] | 
            
  | 04:05 | 
  <RoanKattouw> | 
  Updating Parsoid to master | 
  [production] | 
            
  | 03:53 | 
  <mutante> | 
  powercycling mexia | 
  [production] | 
            
  | 03:29 | 
  <aaron> | 
  synchronized php-1.21wmf5/includes/job/JobQueueDB.php | 
  [production] | 
            
  | 03:29 | 
  <aaron> | 
  synchronized php-1.21wmf5/includes/job/jobs/RefreshLinksJob.php | 
  [production] | 
            
  | 03:29 | 
  <RoanKattouw> | 
  mexia still hasn't come back up. That combined with the weird behavior I saw before rebooting it makes me suspect a hardware issue | 
  [production] | 
            
  | 03:12 | 
  <aaron> | 
  synchronized php-1.21wmf6/includes/job/JobQueueDB.php | 
  [production] | 
            
  | 03:08 | 
  <aaron> | 
  synchronized php-1.21wmf6/includes/job/jobs/RefreshLinksJob.php | 
  [production] | 
            
  | 03:02 | 
  <RoanKattouw> | 
  Rebooting mexia, seems to have been hit by some kernel bug | 
  [production] | 
            
  | 02:00 | 
  <LocalisationUpdate> | 
  failed: git pull of core failed | 
  [production] | 
            
  | 01:12 | 
  <pgehres> | 
  synchronized wmf-config/CommonSettings.php  'Putting donation pipeline back to normal' | 
  [production] | 
            
  | 01:02 | 
  <cmjohnson1> | 
  replacing disk on slot 8 db1008 | 
  [production] | 
            
  | 00:48 | 
  <pgehres> | 
  synchronized wmf-config/CommonSettings.php  'Putting donation pipeline in maintenance mode, part 2' | 
  [production] | 
            
  | 00:44 | 
  <pgehres> | 
  synchronized wmf-config/CommonSettings.php  'Putting donation pipeline in maintenance mode' | 
  [production] | 
            
  | 00:32 | 
  <K4-713> | 
  synchronized payments cluster to 325a919967709359c8 | 
  [production] | 
            
  | 00:05 | 
  <pgehres> | 
  synchronized php-1.21wmf6/extensions/FundraiserLandingPage/  'Updating FundraiserLandingPage to master' | 
  [production] | 
            
  | 00:04 | 
  <pgehres> | 
  synchronized php-1.21wmf6/extensions/ContributionTracking/  'Updating ContributionTracking to master' | 
  [production] | 
            
  
    | 
      
        2012-12-10
      
      §
     | 
  
    
  | 23:17 | 
  <pgehres> | 
  synchronized php-1.21wmf5/extensions/ContributionTracking/  'Updating ContributionTracking to master' | 
  [production] | 
            
  | 23:10 | 
  <K4-713> | 
  synchronized payments to 72c1985a2b9b1db | 
  [production] | 
            
  | 22:45 | 
  <LeslieCarr> | 
  removed /home/wikipedia/common/php-1.21wmf1/ from fenari | 
  [production] | 
            
  | 22:08 | 
  <mutante> | 
  mobile and non-mobile Wikipedia now use just ONE certificate.. a new "star.wp" with an ".m." in it, valid until 2016 by DigiCert | 
  [production] | 
            
  | 21:57 | 
  <mutante> | 
  one more nginx restart on ssl boxes for mobile cert | 
  [production] | 
            
  | 21:18 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: wikidatawiki & mediawikiwiki to 1.21wmf6 | 
  [production] | 
            
  | 21:03 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: test2wiki also on 1.21wmf6 | 
  [production] | 
            
  | 21:03 | 
  <demon> | 
  Finished syncing Wikimedia installation... : | 
  [production] | 
            
  | 20:27 | 
  <demon> | 
  Started syncing Wikimedia installation... : | 
  [production] | 
            
  | 20:19 | 
  <asher> | 
  synchronized wmf-config/CommonSettings.php  're-enabling pc1 for sql: the bag o stuffening' | 
  [production] | 
            
  | 20:11 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: | 
  [production] | 
            
  | 20:07 | 
  <demon> | 
  Finished syncing Wikimedia installation... : | 
  [production] | 
            
  | 20:03 | 
  <demon> | 
  Started syncing Wikimedia installation... : | 
  [production] | 
            
  | 20:01 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki back to 1.21wmf5 | 
  [production] | 
            
  | 19:59 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki to 1.21wmf6 | 
  [production] | 
            
  | 19:58 | 
  <demon> | 
  ran sync-common-all | 
  [production] | 
            
  | 19:50 | 
  <mutante> | 
  replaced wikipedia SSL cert with new DigiCert (rather than RapidSSL) (RT-3639) | 
  [production] | 
            
  | 19:40 | 
  <mutante> | 
  restarting nginx on all ssl hosts | 
  [production] |