| 
      
        2010-12-18
      
      §
     | 
  
    
  | 23:11 | 
  <RobH> | 
  srv187-srv189 relocated and powering up | 
  [production] | 
            
  | 22:27 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db.php  'srv181-srv186 in' | 
  [production] | 
            
  | 22:26 | 
  <mark> | 
  Updated smokeping with new rack layouts and contents | 
  [production] | 
            
  | 22:21 | 
  <RobH> | 
  srv181-srv186 relocated and coming back online | 
  [production] | 
            
  | 22:21 | 
  <RobH> | 
  srv187-srv189 coming down for relocation | 
  [production] | 
            
  | 22:00 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db.php  'srv153 srv154 in' | 
  [production] | 
            
  | 21:58 | 
  <RobH> | 
  srv153, srv154 powering back up | 
  [production] | 
            
  | 21:53 | 
  <RobH> | 
  shutting down srv153, srv154 power redistribution | 
  [production] | 
            
  | 21:51 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db.php  'srv154-srv153 out' | 
  [production] | 
            
  | 21:28 | 
  <RobH> | 
  srv181-srv186 going offline for relocation | 
  [production] | 
            
  | 21:27 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db.php  'srv181-srv186 pulled when applicable' | 
  [production] | 
            
  | 21:23 | 
  <hashar> | 
  synchronized php-1.5/wmf-config/abusefilter.php  'resyncing abusefilter.php for the offline apaches' | 
  [production] | 
            
  | 21:20 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db.php  | 
  [production] | 
            
  | 21:15 | 
  <RobH> | 
  srv157 accidental reboot, opps =P | 
  [production] | 
            
  | 21:13 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db.php  'commented out srv156, its coming down for power rebalancing' | 
  [production] | 
            
  | 21:11 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db.php  'srv178-srv180 back in' | 
  [production] | 
            
  | 21:05 | 
  <RobH> | 
  srv175-srv180 relocated to new rack and coming online | 
  [production] | 
            
  | 20:53 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db.php  'srv177 back in' | 
  [production] | 
            
  | 20:42 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db.php  'srv175 srv176 back in' | 
  [production] | 
            
  | 20:06 | 
  <hashar> | 
  synchronized php-1.5/wmf-config/abusefilter.php  'bug 26364: abuse filter privacy issue on hiwiki' | 
  [production] | 
            
  | 20:02 | 
  <RobH> | 
  srv175-srv180 coming down for relocation | 
  [production] | 
            
  | 20:02 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db.php  'putting ms2 back into ES rotation, removing srv175-srv180' | 
  [production] | 
            
  | 19:39 | 
  <hashar> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'bug 26364: abuse filter privacy issue on hiwiki' | 
  [production] | 
            
  | 19:28 | 
  <RobH> | 
  hume back online in new rack | 
  [production] | 
            
  | 18:38 | 
  <mark> | 
  DRBD failover to nfs2 successful, nfs2 is now primary | 
  [production] | 
            
  | 18:38 | 
  <RobH> | 
  pdf2 relocated, powering up | 
  [production] | 
            
  | 18:32 | 
  <mark> | 
  Starting DRBD failover of nfs1 to nfs2 | 
  [production] | 
            
  | 18:20 | 
  <RobH> | 
  nfs2 reracked, powered up, online (replication check delayed until later) | 
  [production] | 
            
  | 18:19 | 
  <RobH> | 
  pdf2 coming down for relocation, will be back online shortly | 
  [production] | 
            
  | 17:51 | 
  <RobH> | 
  shutting down nfs2 for relocation | 
  [production] |