| 
      
        2012-07-13
      
      §
     | 
  
    
  | 17:58 | 
  <reedy> | 
  synchronized wmf-config/  | 
  [production] | 
            
  | 17:57 | 
  <reedy> | 
  synchronized php-1.20wmf7/  | 
  [production] | 
            
  | 17:56 | 
  <reedy> | 
  synchronized php-1.20wmf6/  | 
  [production] | 
            
  | 17:32 | 
  <reedy> | 
  synchronized wmf-config/  | 
  [production] | 
            
  | 17:21 | 
  <AaronSchulz> | 
  Running copyFileBackend.php for commons (shards 1-4) (actually started yesterday) | 
  [production] | 
            
  | 15:28 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Disable aftv5 on en_labswikimedia due to out of date schema' | 
  [production] | 
            
  | 15:21 | 
  <reedy> | 
  synchronized php-1.20wmf7/includes/GlobalFunctions.php  | 
  [production] | 
            
  | 15:20 | 
  <reedy> | 
  synchronized php-1.20wmf6/includes/GlobalFunctions.php  | 
  [production] | 
            
  | 15:20 | 
  <reedy> | 
  synchronized php-1.20wmf7/includes/DefaultSettings.php  | 
  [production] | 
            
  | 15:19 | 
  <reedy> | 
  synchronized php-1.20wmf6/includes/DefaultSettings.php  | 
  [production] | 
            
  | 15:17 | 
  <reedy> | 
  synchronized wmf-config/CommonSettings.php  'Setting wgDBerrorLogInUTC = true' | 
  [production] | 
            
  | 15:07 | 
  <hasharDeadmau5> | 
  unknown column 'af_is_featured' on en_labswikimedia@db25 | 
  [production] | 
            
  | 14:55 | 
  <mark> | 
  Increased all mobile varnish server weights from 10 to 100 to aid chash | 
  [production] | 
            
  | 14:54 | 
  <mark> | 
  Added cp1041.eqiad.wmnet back into the mobile LVS pool | 
  [production] | 
            
  | 10:10 | 
  <mutante> | 
  mw1016 - was down, reinstalling with precise | 
  [production] | 
            
  | 02:46 | 
  <LocalisationUpdate> | 
  completed (1.20wmf6) at Fri Jul 13 02:45:59 UTC 2012 | 
  [production] | 
            
  
    | 
      
        2012-07-12
      
      §
     | 
  
    
  | 22:15 | 
  <Aaron|home> | 
  Running copyFileBackend.php for commons (shards 0,c-f) | 
  [production] | 
            
  | 20:58 | 
  <Aaron|home> | 
  Running copyFileBackend.php for commons (shards 0,8-b) | 
  [production] | 
            
  | 14:22 | 
  <mutante> | 
  apache children on srv193 keep segfaulting since yesterday (test.wp) | 
  [production] | 
            
  | 14:00 | 
  <RobHalsell> | 
  shutting down srv206 for chris per rt241 | 
  [production] | 
            
  | 11:55 | 
  <reedy> | 
  synchronized wmf-config/  | 
  [production] | 
            
  | 10:40 | 
  <mark> | 
  Depooled mobile varnish server cp1041 for vlan/hostname change and reinstallation with precise | 
  [production] | 
            
  | 04:38 | 
  <binasher> | 
  started hot backup of db48 to db1048 | 
  [production] | 
            
  | 02:50 | 
  <LocalisationUpdate> | 
  completed (1.20wmf6) at Thu Jul 12 02:50:49 UTC 2012 | 
  [production] | 
            
  | 02:29 | 
  <maplebed> | 
  adjusted the swift rings a bit to move some more traffic to the new hosts.  set object partitions to weight 10, account and container 60. | 
  [production] | 
            
  | 02:27 | 
  <LocalisationUpdate> | 
  completed (1.20wmf7) at Thu Jul 12 02:26:57 UTC 2012 | 
  [production] | 
            
  
    | 
      
        2012-07-11
      
      §
     | 
  
    
  | 22:52 | 
  <AaronSchulz> | 
  Running copyFileBackend.php for commons (shards 0,4-7) | 
  [production] | 
            
  | 22:06 | 
  <AaronSchulz> | 
  Running copyFileBackend.php for commons (shards 0,0-3) | 
  [production] | 
            
  | 21:36 | 
  <binasher> | 
  running hotbackup of db48 -> db49 (otrs / external misc) | 
  [production] | 
            
  | 21:31 | 
  <binasher> | 
  rebooting db49 for kernel upgrade | 
  [production] | 
            
  | 21:23 | 
  <binasher> | 
  rebuilding db49 (otrs slave) | 
  [production] | 
            
  | 21:01 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  | 
  [production] | 
            
  | 20:16 | 
  <reedy> | 
  synchronized php-1.20wmf7/extensions/CentralNotice  | 
  [production] | 
            
  | 19:33 | 
  <cmjohnson1> | 
  db10 swapping disk1 out rt3251 | 
  [production] | 
            
  | 19:33 | 
  <RobH> | 
  shutting down srv203 for hardware checking per rt3110 | 
  [production] | 
            
  | 19:12 | 
  <hashar> | 
  finally stopped using my production ssh key on labs. | 
  [production] | 
            
  | 19:11 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: wikiquote and wikiversity to 1.20wmf7 | 
  [production] | 
            
  | 19:08 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: wikibooks and wikinews to 1.20wmf7 | 
  [production] | 
            
  | 19:06 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: wiktionary and wikisource to 1.20wmf7 | 
  [production] | 
            
  | 19:03 | 
  <cmjohnson1> | 
  search35 shutting down to reseat DIMM rt-3260 | 
  [production] | 
            
  | 18:29 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: Switch special, private and closed wikis from 1.20wmf6 to 1.20wmf7 | 
  [production] | 
            
  | 15:16 | 
  <robh> | 
  synchronized wmf-config/mc.php  'moving srv208 to down' | 
  [production] | 
            
  | 15:14 | 
  <robh> | 
  synchronized wmf-config/mc.php  | 
  [production] | 
            
  | 14:30 | 
  <cmjohnson1> | 
  shutting down search32 to reseat DIMM rt-3076 | 
  [production] | 
            
  | 11:16 | 
  <mutante> | 
  yay @ svn removal | 
  [production] | 
            
  | 11:13 | 
  <hashar> | 
  /home/wikipedia/conf/httpd cleaned out the svn repository. That directory is now 100% under git tracking yeah!!! Thanks TIm! | 
  [production] | 
            
  | 11:11 | 
  <mutante> | 
  rebooting srv266 | 
  [production] | 
            
  | 11:05 | 
  <mutante> | 
  purged wikipedie.cz/Experti_na_prirodu URLs with purgeList.php | 
  [production] | 
            
  | 10:52 | 
  <mutante> | 
  apache-graceful-all | 
  [production] | 
            
  | 10:48 | 
  <mutante> | 
  sync-apache to push fixed wikipedie.cz redirect | 
  [production] |