| 
      
        2013-01-04
      
      §
     | 
  
    
  | 23:29 | 
  <RobH> | 
  authdns-update for xenon and caesium dns | 
  [production] | 
            
  | 22:46 | 
  <binasher> | 
  temporarily setting up mysql replication on shard es1 again (es([1-4]|100[1-4]) for mha testing | 
  [production] | 
            
  | 22:35 | 
  <notpeter> | 
  es1001 back in prodcution. all external store are innodb! | 
  [production] | 
            
  | 22:17 | 
  <RobH> | 
  updated asw-a-eqiad vlan for 4/0/44 | 
  [production] | 
            
  | 21:25 | 
  <mutante> | 
  mauntius/torrus: http://wikitech.wikimedia.org/view/Torrus#Deadlock_problem | 
  [production] | 
            
  | 21:23 | 
  <RobH> | 
  authdns-update for some mgmt name changes | 
  [production] | 
            
  | 21:23 | 
  <mutante> | 
  restarting torrus-common on manutius | 
  [production] | 
            
  | 21:01 | 
  <cmjohnson1> | 
  db1042 removing/replacing bad hdd from slot 9 | 
  [production] | 
            
  | 20:52 | 
  <cmjohnson1> | 
  db1038 removing bad hdd from slot6 for replacement | 
  [production] | 
            
  | 20:02 | 
  <Ryan_Lane> | 
  depooling ssl1001 | 
  [production] | 
            
  | 18:29 | 
  <robh> | 
  synchronized wmf-config/PoolCounterSettings.php | 
  [production] | 
            
  | 15:29 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: wikidatawiki to 1.21wmf7 | 
  [production] | 
            
  | 15:08 | 
  <demon> | 
  Finished syncing Wikimedia installation... : | 
  [production] | 
            
  | 14:25 | 
  <demon> | 
  Started syncing Wikimedia installation... : | 
  [production] | 
            
  | 02:49 | 
  <LocalisationUpdate> | 
  completed (1.21wmf7) at Fri Jan  4 02:49:14 UTC 2013 | 
  [production] | 
            
  | 02:47 | 
  <asher> | 
  synchronized wmf-config/db.php  'returning db64 to s3' | 
  [production] | 
            
  | 02:27 | 
  <binasher> | 
  restarted replication on db64 from db34 | 
  [production] | 
            
  | 02:26 | 
  <LocalisationUpdate> | 
  completed (1.21wmf6) at Fri Jan  4 02:26:53 UTC 2013 | 
  [production] | 
            
  | 02:20 | 
  <binasher> | 
  truncated ( de | it )wikivoyage.searchindex tables | 
  [production] | 
            
  | 00:37 | 
  <bsitu> | 
  Finished syncing Wikimedia installation... : Update Echo to Master | 
  [production] | 
            
  | 00:20 | 
  <notpeter> | 
  starting innobackupex from es1002 to es1001 | 
  [production] | 
            
  | 00:15 | 
  <RobH> | 
  modifying kuo bios power profile from DAPC to OS performance per watt setting in attempt to resolve runaway process issue | 
  [production] | 
            
  | 00:13 | 
  <notpeter> | 
  converting es1001 to innodb | 
  [production] | 
            
  | 00:11 | 
  <binasher> | 
  rebuilding db64 via hotbackup of db1010 | 
  [production] | 
            
  | 00:03 | 
  <asher> | 
  synchronized wmf-config/db.php  'pulling db64 from s3' | 
  [production] | 
            
  | 00:02 | 
  <RobH> | 
  taking down kuo, its power settings are freaking it out, will fix and bring back into servie | 
  [production] | 
            
  
    | 
      
        2013-01-03
      
      §
     | 
  
    
  | 23:10 | 
  <olivneh> | 
  synchronized php-1.21wmf6/extensions/EventLogging  'Updating display of rev IDs in schema subtitles' | 
  [production] | 
            
  | 23:02 | 
  <RobH> | 
  authdns-updateexit | 
  [production] | 
            
  | 22:59 | 
  <bsitu> | 
  Started syncing Wikimedia installation... : Update Echo to Master | 
  [production] | 
            
  | 22:34 | 
  <aaron> | 
  synchronized wmf-config/CommonSettings.php  'switched captchas back to swift' | 
  [production] | 
            
  | 22:18 | 
  <binasher> | 
  reprepro includedeb (lucid|precise)-wikimedia mha4mysql-manager_0.55-0_all.deb mha4mysql-node_0.54-0_all.deb | 
  [production] | 
            
  | 21:30 | 
  <reedy> | 
  synchronized php-1.21wmf7/cache/interwiki.cdb  'Updating 1.21wmf7 interwiki cache' | 
  [production] | 
            
  | 21:30 | 
  <reedy> | 
  synchronized php-1.21wmf6/cache/interwiki.cdb  'Updating 1.21wmf6 interwiki cache' | 
  [production] | 
            
  | 21:30 | 
  <RobH> | 
  reinstalling tarin to internal ip for secondary poolcounter server in tampa | 
  [production] | 
            
  | 21:13 | 
  <paravoid> | 
  powercycling cp1026, load spike, unresponsive | 
  [production] | 
            
  | 20:38 | 
  <reedy> | 
  synchronized php-1.21wmf7/cache/interwiki.cdb  'Updating interwiki cache' | 
  [production] |