| 
      
        2012-07-16
      
      §
     | 
  
    
  | 15:56 | 
  <mutante> | 
  fixing pt.planet after missing locale has been added | 
  [production] | 
            
  | 15:48 | 
  <mutante> | 
  installing package upgrades on singer (planet) | 
  [production] | 
            
  | 15:43 | 
  <cmjohnson1> | 
  mw8 shutting down and taking offline  to run Dell's dset program | 
  [production] | 
            
  | 15:38 | 
  <mutante> | 
  planet - svn up the config | 
  [production] | 
            
  | 15:29 | 
  <mark> | 
  Reinstalling lvs3 with Ubuntu Precise | 
  [production] | 
            
  | 15:17 | 
  <mark> | 
  lvs5 is back up and serving traffic | 
  [production] | 
            
  | 14:49 | 
  <mark> | 
  Reinstalling lvs5 with Ubuntu Precise | 
  [production] | 
            
  | 14:45 | 
  <mark> | 
  Stopped PyBal on lvs5 to failover traffic to lvs1 | 
  [production] | 
            
  | 14:35 | 
  <maplebed> | 
  adjusted swift rings; set new object servers to 20, new container servers to 100 | 
  [production] | 
            
  | 14:33 | 
  <mark> | 
  lvs6 is back up and serving traffic | 
  [production] | 
            
  | 14:12 | 
  <mark> | 
  Reinstalling lvs6 with Ubuntu Precise | 
  [production] | 
            
  | 14:02 | 
  <mark> | 
  Stopped PyBal on lvs6 to failover traffic to lvs2 | 
  [production] | 
            
  | 13:58 | 
  <mutante> | 
  mw8 - alright, most likely just needs new DIMM per cmjohnson | 
  [production] | 
            
  | 13:50 | 
  <mutante> | 
  mw8 PHP fatal errors, running out of memory | 
  [production] | 
            
  | 12:56 | 
  <mutante> | 
  sync-apache, graceful-all to fix wikipedia.cz redirect | 
  [production] | 
            
  | 05:32 | 
  <apergos> | 
  reboot to upgrade kernel etc on mw8 since it's been flapping anyways | 
  [production] | 
            
  | 02:46 | 
  <LocalisationUpdate> | 
  completed (1.20wmf6) at Mon Jul 16 02:46:14 UTC 2012 | 
  [production] | 
            
  | 02:23 | 
  <LocalisationUpdate> | 
  completed (1.20wmf7) at Mon Jul 16 02:23:04 UTC 2012 | 
  [production] | 
            
  
    | 
      
        2012-07-13
      
      §
     | 
  
    
  | 18:08 | 
  <reedy> | 
  synchronized php-1.20wmf7/includes/GlobalFunctions.php  | 
  [production] | 
            
  | 18:08 | 
  <reedy> | 
  synchronized php-1.20wmf6/includes/GlobalFunctions.php  | 
  [production] | 
            
  | 17:58 | 
  <reedy> | 
  synchronized wmf-config/  | 
  [production] | 
            
  | 17:57 | 
  <reedy> | 
  synchronized php-1.20wmf7/  | 
  [production] | 
            
  | 17:56 | 
  <reedy> | 
  synchronized php-1.20wmf6/  | 
  [production] | 
            
  | 17:32 | 
  <reedy> | 
  synchronized wmf-config/  | 
  [production] | 
            
  | 17:21 | 
  <AaronSchulz> | 
  Running copyFileBackend.php for commons (shards 1-4) (actually started yesterday) | 
  [production] | 
            
  | 15:28 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Disable aftv5 on en_labswikimedia due to out of date schema' | 
  [production] | 
            
  | 15:21 | 
  <reedy> | 
  synchronized php-1.20wmf7/includes/GlobalFunctions.php  | 
  [production] | 
            
  | 15:20 | 
  <reedy> | 
  synchronized php-1.20wmf6/includes/GlobalFunctions.php  | 
  [production] | 
            
  | 15:20 | 
  <reedy> | 
  synchronized php-1.20wmf7/includes/DefaultSettings.php  | 
  [production] | 
            
  | 15:19 | 
  <reedy> | 
  synchronized php-1.20wmf6/includes/DefaultSettings.php  | 
  [production] | 
            
  | 15:17 | 
  <reedy> | 
  synchronized wmf-config/CommonSettings.php  'Setting wgDBerrorLogInUTC = true' | 
  [production] | 
            
  | 15:07 | 
  <hasharDeadmau5> | 
  unknown column 'af_is_featured' on en_labswikimedia@db25 | 
  [production] | 
            
  | 14:55 | 
  <mark> | 
  Increased all mobile varnish server weights from 10 to 100 to aid chash | 
  [production] | 
            
  | 14:54 | 
  <mark> | 
  Added cp1041.eqiad.wmnet back into the mobile LVS pool | 
  [production] | 
            
  | 10:10 | 
  <mutante> | 
  mw1016 - was down, reinstalling with precise | 
  [production] | 
            
  | 02:46 | 
  <LocalisationUpdate> | 
  completed (1.20wmf6) at Fri Jul 13 02:45:59 UTC 2012 | 
  [production] |