| 
      
        2010-01-04
      
      §
     | 
  
    
  | 14:31 | 
  <Rob> | 
  decommissioned srv120, srv121 to make room for new search servers. | 
  [production] | 
            
  | 14:29 | 
  <Rob> | 
  pulling srv34 from rack to decommission, need the space for new search servers. | 
  [production] | 
            
  | 14:09 | 
  <mark> | 
  synchronized php-1.5/wmf-config/mc.php  'Replace memcached on just decommissioned host srv82' | 
  [production] | 
            
  | 13:57 | 
  <Rob> | 
  srv82, srv83, srv84 decommissioned to make room for new search servers in rack | 
  [production] | 
            
  | 12:56 | 
  <Tim> | 
  started squid-frontend on knsq8, knsq10, knsq11, knsq13, knsq14, knsq15, all crashed at roughly the same time as knsq9 | 
  [production] | 
            
  | 12:52 | 
  <Tim> | 
  started squid-frontend on knsq9, died at ~17:30 on the 3rd. Syslog shows many crashes, followed by "out of socket memory" a couple of hundred times, then silence | 
  [production] | 
            
  
    | 
      
        2009-12-30
      
      §
     | 
  
    
  | 23:37 | 
  <tfinc> | 
  synchronized php-1.5/extensions/ContributionReporting/FundraiserStatistics_body.php  | 
  [production] | 
            
  | 23:35 | 
  <tfinc> | 
  synchronized php-1.5/extensions/ContributionReporting/ContributionReporting.i18n.php  | 
  [production] | 
            
  | 12:51 | 
  <mark> | 
  Set /proc/sys/vm/min_free_kbytes to 65535 on brewster and streber, to see if it helps with the swapper page allocation failure bug | 
  [production] | 
            
  | 07:57 | 
  <Tim> | 
  fixed private+secure file access, URLs were broken for months at least | 
  [production] | 
            
  | 07:50 | 
  <tstarling> | 
  synchronized php-1.5/wmf-config/secure.php  | 
  [production] | 
            
  | 07:48 | 
  <tstarling> | 
  synchronized php-1.5/wmf-config/secure.php  'possible resolution for boardwiki problem' | 
  [production] | 
            
  | 07:45 | 
  <tstarling> | 
  synchronized php-1.5/wmf-config/secure.php  'debugging hack' | 
  [production] | 
            
  | 07:34 | 
  <Tim> | 
  created an account for myself on boardwiki to debug file upload issue | 
  [production] | 
            
  
    | 
      
        2009-12-28
      
      §
     | 
  
    
  | 21:02 | 
  <RoanKattouw> | 
  Restarting refreshLinks for enwikibooks on hume | 
  [production] | 
            
  | 20:49 | 
  <mark> | 
  Created 10G of swap space on fenari | 
  [production] | 
            
  | 20:07 | 
  <domas> | 
  powercycled fenari, console not responsive | 
  [production] | 
            
  | 15:00 | 
  <mark> | 
  Installed Karmic on LVS1, setting up a test network for LVS performance testing | 
  [production] | 
            
  | 14:55 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'bug 17338: Disable $wgRestrictDisplayTitle and enable subpages in the main namespace on rmwiki' | 
  [production] | 
            
  | 13:54 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'bug 21566: Enable Collection on viwiki' | 
  [production] | 
            
  | 13:42 | 
  <RoanKattouw> | 
  Running refreshLinks on enwikinews per bug 19404 | 
  [production] | 
            
  | 13:41 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'bug 19404: Set $wgCategoryPrefixedDefaultSortkey=false on enwikinews' | 
  [production] | 
            
  | 13:18 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'bug 21958: Create Book namespace on enwiki' | 
  [production] | 
            
  
    | 
      
        2009-12-24
      
      §
     | 
  
    
  | 22:52 | 
  <RoanKattouw> | 
  Running updateArticleCount.php on arwiki and arwiktionary | 
  [production] | 
            
  | 20:21 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'wrong var AGAIN' | 
  [production] | 
            
  | 20:19 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'wrong var' | 
  [production] | 
            
  | 20:18 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'bug 21859: Enable Doublewiki on frwiktionary' | 
  [production] | 
            
  | 19:03 | 
  <catrope> | 
  ran sync-common-all  | 
  [production] | 
            
  | 18:56 | 
  <RoanKattouw> | 
  Running sync-docroot, sync-apache, sync-common-all, apache-graceful-all | 
  [production] | 
            
  | 18:45 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'Explicitly add langcodes for recently created wikis' | 
  [production] | 
            
  | 18:27 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'bug 19534: Add Page and Index namespaces on sourceswiki' | 
  [production] | 
            
  | 17:02 | 
  <catrope> | 
  ran sync-common-all  | 
  [production] | 
            
  | 16:57 | 
  <RoanKattouw> | 
  Syncing updates for wiki creations | 
  [production] |