| 
      
        2012-07-17
      
      §
     | 
  
    
  | 19:09 | 
  <mlitn> | 
  Started syncing Wikimedia installation... :  | 
  [production] | 
            
  | 15:18 | 
  <mark> | 
  lvs1002 is back up and idling | 
  [production] | 
            
  | 15:15 | 
  <mark> | 
  Fixed serial console redirection after boot to OFF on lvs1002 | 
  [production] | 
            
  | 15:06 | 
  <mark> | 
  lvs4 is back up and serving traffic | 
  [production] | 
            
  | 14:52 | 
  <mark> | 
  Reinstalling lvs4 with Ubuntu Precise | 
  [production] | 
            
  | 14:48 | 
  <mark> | 
  Stopped PyBal on lvs4, failing over traffic to lvs3 | 
  [production] | 
            
  | 14:47 | 
  <mark> | 
  Reinstalling lvs1002 with Ubuntu Precise | 
  [production] | 
            
  | 14:46 | 
  <mark> | 
  Stopped PyBal on lvs1002, failing over traffic to lvs1005 | 
  [production] | 
            
  | 14:44 | 
  <mark> | 
  lvs1001 is back up and serving traffic | 
  [production] | 
            
  | 14:11 | 
  <mark> | 
  Stopped PyBal on lvs1001, failing over traffic to lvs1004 | 
  [production] | 
            
  | 13:09 | 
  <maplebed> | 
  changed auth URL for swift to use load balancer rather than round robin DNS | 
  [production] | 
            
  | 13:08 | 
  <mark> | 
  lvs1003 is back up and serving traffic | 
  [production] | 
            
  | 13:07 | 
  <maplebed> | 
  changed order on ganglia swift view graphs to group by metric rather than host | 
  [production] | 
            
  | 13:02 | 
  <reedy> | 
  synchronized wmf-config/  | 
  [production] | 
            
  | 12:53 | 
  <mark> | 
  Fixed boot order on lvs1003 | 
  [production] | 
            
  | 12:34 | 
  <mark> | 
  Reinstalling lvs1003 with Ubuntu Precise | 
  [production] | 
            
  | 12:28 | 
  <mark> | 
  Stopped PyBal on lvs1003, failing over traffic to lvs1006 | 
  [production] | 
            
  | 12:10 | 
  <mark> | 
  amslvs1 is back up and serving traffic | 
  [production] | 
            
  | 11:44 | 
  <mark> | 
  Reinstalling amslvs1 with Ubuntu Precise | 
  [production] | 
            
  | 11:40 | 
  <mark> | 
  Stopped PyBal on amslvs1, failing over traffic to amslvs3 | 
  [production] | 
            
  | 11:32 | 
  <mark> | 
  amslvs2 is back up and serving traffic | 
  [production] | 
            
  | 10:47 | 
  <mark> | 
  Reinstalling amslvs2 with Ubuntu Precise | 
  [production] | 
            
  | 10:36 | 
  <mark> | 
  Stopped PyBal on amslvs2, failing over traffic to amslvs4 | 
  [production] | 
            
  | 09:17 | 
  <hashar> | 
  deployment-prep sync upload6 dirs again. root@deployment-nfs-memc:$ rsync -a --progress --inplace /mnt/export/upload6 /data/project/upload6 | 
  [production] | 
            
  | 07:30 | 
  <Tim> | 
  testing envvars/apache2.conf change on srv193 | 
  [production] | 
            
  | 07:09 | 
  <Tim> | 
  restarted apache on srv193 | 
  [production] | 
            
  | 02:58 | 
  <Tim> | 
  graceful restart of all apaches | 
  [production] | 
            
  | 02:52 | 
  <reedy> | 
  synchronized wmf-config/CommonSettings.php  | 
  [production] | 
            
  | 02:47 | 
  <LocalisationUpdate> | 
  completed (1.20wmf6) at Tue Jul 17 02:47:06 UTC 2012 | 
  [production] | 
            
  | 02:23 | 
  <LocalisationUpdate> | 
  completed (1.20wmf7) at Tue Jul 17 02:23:41 UTC 2012 | 
  [production] | 
            
  | 00:44 | 
  <Tim> | 
  deploying new redirects.conf | 
  [production] | 
            
  | 00:28 | 
  <Tim> | 
  testing new redirects.conf on mw1 | 
  [production] |