| 2010-02-13
      
      § | 
    
  | 15:10 | <mark> | Added DNS entry for api.svc.pmpta.wmnet (LVS ip) | [production] | 
            
  | 14:18 | <mark> | Enabled more verbose logging on all squids: debug message level 1 for all message categories | [production] | 
            
  | 14:18 | <mark> | Disabled cache digests on all squids | [production] | 
            
  | 13:58 | <mark> | Removing sq30 and lower from the Text CARP pool | [production] | 
            
  | 13:51 | <mark> | Reducing persistent_connection_timeout and request_timeout from 1 min to 20s on all frontend squids, to reduce FD usage | [production] | 
            
  | 12:50 | <mark> | Restarting knsq* frontend squids with FD limit raised from 32k to 64k | [production] | 
            
  | 12:23 | <mark> | Grown filesystem /dev/data/ES (/a) on ms3 by 90G, leaving another 98G free | [production] | 
            
  | 00:17 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/UsabilityInitiative.hooks.php  'Sync CSS style versions' | [production] | 
            
  | 00:16 | <catrope> | synchronized php-1.5/wmf-config/CommonSettings.php  'Bump style version appendix' | [production] | 
            
  | 00:10 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js  'r62403' | [production] | 
            
  | 00:02 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/WikiEditor/WikiEditor.combined.min.js  'r62400' | [production] | 
            
  | 00:01 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js  'r62400' | [production] | 
            
  
    | 2010-02-12
      
      § | 
    
  | 23:55 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js  'Try deploying r62397 again' | [production] | 
            
  | 23:46 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/css/combined.min.css  'r62397' | [production] | 
            
  | 23:46 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js  'r62397' | [production] | 
            
  | 23:22 | <mark> | Deployed sq86 | [production] | 
            
  | 22:31 | <catrope> | synchronized php-1.5/wmf-config/CommonSettings.php  'Bump style version appendix' | [production] | 
            
  | 22:30 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js  'r62388' | [production] | 
            
  | 21:12 | <mark> | Implemented new routing policy for AS43821 | [production] | 
            
  | 21:01 | <catrope> | synchronized php-1.5/wmf-config/CommonSettings.php  'Bump style version appendix' | [production] | 
            
  | 20:59 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js  'r62382' | [production] | 
            
  | 19:15 | <Rob> | sq67 has both disks installed now (had to get one replaced) and has no OS installation yet.  holding off on install, not sure if mark wants this up as a squid or for varnish development | [production] | 
            
  | 18:22 | <atglenn> | resstarting transfer of data (one chunk) from ms8->ms5->ms6, running in screen as root on all three hosts using nc | [production] | 
            
  | 16:18 | <Rob> | setup project1 and project2 boxes for flaggedrevsdevelopment and project mangement suite testing | [production] | 
            
  | 15:12 | <mark> | Upgraded our AMS-IX connection from 2x 1G to 10G along the way | [production] | 
            
  | 15:12 | <mark> | Reloaded br1-knams twice to fix a CAM partitioning problem; not sufficient IP next hops for 8-path routes | [production] | 
            
  | 11:34 | <mark> | Filtering all prefixes to/from AMS-IX peers | [production] | 
            
  | 02:55 | <tstarling> | synchronized php-1.5/wmf-config/CommonSettings.php  're-enabling WikiEditor' | [production] | 
            
  | 02:04 | <Tim> | installed openssh-server on bayes, Erik Z apparently uninstalled it with aptitude | [production] | 
            
  | 01:37 | <tstarling> | synchronized php-1.5/wmf-config/CommonSettings.php  'disabled WikiEditor due to complaints about bug 22428' | [production] | 
            
  | 01:06 | <atglenn> | shoveling over some data from ms8 to ms6 via ms5 :-/ | [production] | 
            
  | 00:48 | <andrew> | synchronized php-1.5/extensions/LiquidThreads/lqt.js  'Disable live preview entirely, broken by CentralNotice and DismissableSiteNotice' | [production] | 
            
  | 00:37 | <andrew> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Disable DismissableSiteNotice for lqt.labs, breaks LiquidThreads live preview' | [production] | 
            
  | 00:17 | <andrew> | synchronized php-1.5/wmf-config/CommonSettings.php  'Bump style version' | [production] | 
            
  | 00:07 | <Andrew> | Running scap | [production] | 
            
  | 00:02 | <Andrew> | Updating LiquidThreads alpha to trunk state, deploying with scap in a few minutes | [production] | 
            
  
    | 2010-02-11
      
      § | 
    
  | 22:06 | <Fred> | implemented disk space check for tridge | [production] | 
            
  | 21:10 | <mark> | Pooled sq71-85 with full CARP weight (30) | [production] | 
            
  | 21:06 | <mark> | Pooled sq79-85 frontends in LVS with full load (30) | [production] | 
            
  | 20:59 | <mark> | Pooled sq79-85 frontends in LVS with load 1 | [production] | 
            
  | 20:51 | <mark> | Pooling sq79-85 in CARP on all frontends, with low CARP weight 10 instead of 30 | [production] | 
            
  | 20:30 | <mark> | Pooling sq79-85 in CARP on frontend sq51 to seed the cache | [production] | 
            
  | 19:39 | <atglenn> | replication every 15 minutes enabled from ms7 to ms8 | [production] | 
            
  | 19:31 | <mark> | Removed sq16-30 from the text frontend squid pool in LVS - these servers will soon be decommissioned | [production] | 
            
  | 19:28 | <mark> | Pooled sq71-78 frontend squids in LVS with full load (30) | [production] | 
            
  | 19:21 | <mark> | Pooled sq71-78 frontend squids in LVS with low load (1) to seed caches | [production] | 
            
  | 18:30 | <mark> | Pooling sq71-78 backend squids on all frontends, with lower CARP weight (10 instead of 30) to seed the caches | [production] | 
            
  | 18:04 | <mark> | Deployed new squid config to sq58-66 frontend squid, to seed the caches of backend squids sq71-78 | [production] | 
            
  | 17:27 | <mark> | Deployed new squid config to sq66 frontend squid, to seed the caches of backend squids sq71-78 | [production] | 
            
  | 17:24 | <mark> | Deployed Squid with correct configuration on sq71-78 | [production] |