| 2009-11-24
      
      § | 
    
  | 19:07 | <mark> | Resized JFS filesystem of /home on nfs2 to fit the new LV size on nfs1/nfs2 | [production] | 
            
  | 19:03 | <mark> | Moved cronned rsync of /home from db20 to nfs1 and nfs2, managed by puppet | [production] | 
            
  | 18:46 | <mark> | MediaWiki logging, broken by yesterday's NFS service IP migration, restored (using puppet) | [production] | 
            
  | 18:15 | <mark> | Puppetised rsyncd setup on nfs1/nfs2 | [production] | 
            
  | 18:00 | <andrew> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Flood flag for strategywiki bug 20347' | [production] | 
            
  | 01:43 | <Tim> | enabled write-back caching on db14 despite broken controller battery, due to excessive lag and MASTER_POS_WAIT() | [production] | 
            
  | 00:24 | <fvassard> | synchronized php-1.5/wmf-config/CommonSettings.php  'It turns out that this time FiveFacts is actually referenced as Five_Facts_About_Wikipedia... who knew...' | [production] | 
            
  | 00:10 | <fvassard> | synchronized php-1.5/wmf-config/CommonSettings.php  'It turns out that this time FiveFacts is actually referenced as Five_Facts_About_Wikipedia... who knew...' | [production] | 
            
  | 00:01 | <fvassard> | synchronized php-1.5/wmf-config/CommonSettings.php  'Adding Notice_[36,37,28] and FiveFacts landing page' | [production] | 
            
  
    | 2009-11-23
      
      § | 
    
  | 21:42 | <fvassard> | synchronized php-1.5/wmf-config/CommonSettings.php  'Adding RU and IL for GEOIP location.' | [production] | 
            
  | 21:27 | <fvassard> | synchronized php-1.5/wmf-config/CommonSettings.php  'Adding RU and IL for GEOIP location.' | [production] | 
            
  | 20:53 | <river> | moving ptolemy to TS vlan to reinstall it as the TS OSM db | [production] | 
            
  | 20:11 | <mark> | Finished migrating NFS off db20 to a DRBD cluster of nfs2 (primary) and nfs1 (secondary) | [production] | 
            
  | 19:14 | <RoanKattouw> | prototype got stuck, rebooted it. Linode sucks and we should not use it ever again | [production] | 
            
  | 16:53 | <mark> | Re-signed puppet cert for fenari... why was it gone? | [production] | 
            
  | 12:39 | <domas> | spence was taken down by runaway rrdtool process, http://p.defau.lt/?8FAf9FtfQTbkSQKCcP_qmw | [production] | 
            
  | 11:00 | <Andrew> | Scapping to prod message updates | [production] | 
            
  | 10:49 | <andrew> | synchronized php-1.5/extensions/LiquidThreads_alpha/pages/NewUserMessagesView.php  'Merge r59353' | [production] | 
            
  | 10:35 | <Andrew> | Scapping | [production] | 
            
  | 10:34 | <Andrew> | Updating LiquidThreads to trunk state on alpha only | [production] | 
            
  | 08:28 | <domas> | sq14 has failed sdc | [production] | 
            
  | 02:05 | <Tim> | fixed clear-profile so that it works on fenari and other ubuntu hosts | [production] | 
            
  | 01:54 | <tstarling> | synchronized php-1.5/index.php  'test' | [production] | 
            
  | 01:54 | <Tim> | set up fenari's /root/.ssh/config like the one on zwinger, with its user known hosts file pointing to the global one | [production] | 
            
  | 01:41 | <Tim> | Moved logmsgbot to fenari. Opened up its UDP port to zwinger and added firewall rules to /etc/rc.local for honour-system access control. | [production] | 
            
  | 01:21 | <Tim> | commented out the nagios-specific nickserv stuff in ircecho, so that logmsgbot doesn't ghost nagios-wm | [production] | 
            
  | 01:11 | <Tim> | installed udprec at fenari:/usr/local/bin/udprec | [production] | 
            
  | 00:55 | <Tim> | on fenari, symlinked ddsh -> dsh (dancer's) so that the sync scripts can work | [production] | 
            
  
    | 2009-11-20
      
      § | 
    
  | 22:11 | <fvassard> | synchronized php-1.5/wmf-config/CommonSettings.php  'Adding 2009_Notice35 to whitelist' | [production] | 
            
  | 22:10 | <fvassard> | synchronized php-1.5/wmf-config/CommonSettings.php  'Adding Notice30_EML to whitelist' | [production] | 
            
  | 19:13 | <fvassard> | synchronized php-1.5/wmf-config/CommonSettings.php  'Added Help_Us_Change_the_World to allowed targets' | [production] | 
            
  | 18:44 | <fvassard> | synchronized php-1.5/wmf-config/CommonSettings.php  'Added Help_Us_Change_the_World to allowed targets' | [production] | 
            
  | 18:41 | <fvassard> | synchronized php-1.5/wmf-config/CommonSettings.php  'Enabling tracking of 2009_Notice30_EML' | [production] | 
            
  | 13:56 | <mark> | OS-installed nfs1 | [production] | 
            
  | 13:09 | <mark> | Renamed auth1 to nfs1 in DNS | [production] | 
            
  | 13:00 | <mark> | apt-get upgrade and reboot on fenari | [production] | 
            
  | 12:48 | <catrope> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Re-enabling ClickTracking with throttle 1:1000' | [production] | 
            
  | 12:46 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/ClickTracking/ClickTracking.php  'Deploying r59284' | [production] | 
            
  | 12:46 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/ClickTracking/ClickTracking.js  'Deploying r59284' | [production] | 
            
  | 04:51 | <midom> | synchronized php-1.5/wmf-config/db.php  'load test' | [production] | 
            
  | 04:49 | <midom> | synchronized php-1.5/wmf-config/db.php  'load test' | [production] |