| 2009-07-29
      
      § | 
    
  | 23:43 | <brion> | temporarily bumping max connections to 6k on db16 | [production] | 
            
  | 23:39 | <brion> | new boxes need to be fixed so apache-restart family of scripts work, or else the scripts replaced | [production] | 
            
  | 18:17 | <Rob> | srv131 back online | [production] | 
            
  | 18:17 | <Rob> | srv118 and srv130 back online | [production] | 
            
  | 18:15 | <Rob> | rebooted srv131 for lockup | [production] | 
            
  | 18:13 | <Rob> | rebooting srv118 & srv130, locked up | [production] | 
            
  | 18:11 | <Rob> | srv110 and srv113 back online | [production] | 
            
  | 18:08 | <Rob> | rebooted srv110, srv113, both locked up | [production] | 
            
  | 16:18 | <river> | removed stale fingerprint for ns1.wikimedia.org on bayle | [production] | 
            
  | 14:01 | <Rob> | shutting down mysql on db12, when done will shut down system and replace bad battery on controller | [production] | 
            
  | 13:30 | <Rob> | upgraded dobson memory for squid ssh testing | [production] | 
            
  | 13:24 | <Rob> | shutdown eiximenis, upgraded ram | [production] | 
            
  | 04:36 | <Tim> | running svn cleanup on locked directories as indicated by find -name lock | [production] | 
            
  | 04:19 | <Tim> | resharing ms1:/export/upload with root allowed, so that the above command actually works | [production] | 
            
  | 04:15 | <Tim> | fixing ExtensionDistributor permissions with: find -not -user extdist -exec chown extdist {} \\; | [production] | 
            
  | 04:11 | <Tim> | reduced the number of svn-invoker threads on zwinger to 1, to hopefully avoid permanent "lock held" errors (bug 19889) | [production] | 
            
  
    | 2009-07-24
      
      § | 
    
  | 21:51 | <hcatlin> | pushed new mobile code to support multiple notices | [production] | 
            
  | 20:08 | <brion> | added $wgUseXVO = true to default config in InitialiseSettings so it'll be there when bug 19845 fix goes live | [production] | 
            
  | 19:11 | <tomaszf> | starting three xml snapshot threads | [production] | 
            
  | 18:59 | <tomaszf> | pointing storage2 to /export/dumps instead of /export/archive | [production] | 
            
  | 17:06 | <domas> | removed henbane & amaryllis from nagios | [production] | 
            
  | 10:54 | <midom> | synchronized php-1.5/wmf-config/../includes/specials/SpecialSearch.php  'livehack per rainmans suggestion' | [production] | 
            
  | 05:31 | <Tim> | download.wikimedia.org was totally broken, just showing a few empty directories. Changed the document root on storage2 to /export/archive/public, where there appears to be some useful files | [production] |