| 2010-09-03
      
      § | 
    
  | 23:44 | <Ryan_Lane> | added philippe to the wmffundraising alias | [production] | 
            
  | 18:56 | <RobH> | srv123 pulled, wiping.  also all memcached in updated pool test fine. | [production] | 
            
  | 18:56 | <robh> | synchronized php-1.5/wmf-config/mc.php  'removing srv123 from memcached due to decommissioning' | [production] | 
            
  | 18:38 | <RobH> | rt#54 db16 seems to not be throwing errors... installs ok, passed all VTS testing and memtesting.  reinstalled the OS | [production] | 
            
  | 11:45 | <tomaszf> | leaving lighttpd in non ipv6 mode till currently open sockets clear | [production] | 
            
  | 10:39 | <tomaszf> | disabling ipv6 for lighttpd on dataset1 as its not working post upgrade | [production] | 
            
  | 10:20 | <tomaszf> | upgrading to ubuntu 10.04 LTS on dataset1 | [production] | 
            
  | 01:44 | <Ryan_Lane> | added pre-commit and post-commit hooks to puppet's svn to check syntax on pre-commit, and svn up on post-commit | [production] | 
            
  
    | 2010-09-02
      
      § | 
    
  | 23:09 | <Ryan_Lane> | restarted mysql on db10 | [production] | 
            
  | 23:09 | <Ryan_Lane> | set expire_logs_days to 14 on db10 | [production] | 
            
  | 23:04 | <Ryan_Lane> | fixed password issue for debian-sys-maint on db10 | [production] | 
            
  | 22:56 | <Ryan_Lane> | commented out skip_slave_start in the my.cnf on db10 | [production] | 
            
  | 22:54 | <Ryan_Lane> | restarted db10 (specifically the mysql database) | [production] | 
            
  | 22:54 | <Ryan_Lane> | restarted db10 | [production] | 
            
  | 19:38 | <tfinc> | synchronized php-1.5/wmf-config/CommonSettings.php  'adding new thursday test banner to whitelist' | [production] | 
            
  | 16:53 | <RobH> | reinstalling storage2 | [production] | 
            
  | 16:09 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/PrefSwitch/SpecialPrefSwitch.php  'Attempt to fix DB error' | [production] | 
            
  | 16:06 | <RobH> | db16 is running VTS, DO NOT TOUCH THIS SERVER | [production] | 
            
  | 15:51 | <RobH> | db34 unracked, going back for repair with Oracle Field Engineer. | [production] | 
            
  | 15:02 | <RobH> | ignore errrors from db16, db28, db34.  Today is sun server troubleshooting day. | [production] | 
            
  | 14:58 | <RobH> | db28 is not in rotation, taking it down to continue hardware testing/repair | [production] | 
            
  | 14:25 | <RobH> | srv206 back online and in cluster. | [production] | 
            
  | 14:25 | <robh> | synchronized php-1.5/wmf-config/mc.php  'moved srv206 from down to spare' | [production] | 
            
  | 13:49 | <RobH> | srv206 offline, looking into it. | [production] | 
            
  
    | 2010-09-01
      
      § | 
    
  | 22:42 | <mark> | Shutdown zwinger | [production] | 
            
  | 22:10 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/PrefSwitch/SpecialPrefSwitch.php  'Fix fatal' | [production] | 
            
  | 21:58 | <Ryan_Lane> | created article assessment wiki on prototype | [production] | 
            
  | 21:31 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/PrefSwitch/SpecialPrefSwitch.php  'Attempted fix for skin overwriting problem' | [production] | 
            
  | 21:19 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/PrefSwitch/SpecialPrefSwitch.php  'Attempted fix for DB issue' | [production] | 
            
  | 20:56 | <Ryan_Lane> | removed unneeded files from /var/tmp on prototype to free up space | [production] | 
            
  | 20:43 | <Ryan_Lane> | svn upped bugzilla/skins/custom/buglist.css and bugzilla/skins/custom/show_bug.css | [production] | 
            
  | 20:42 | <Ryan_Lane> | added an older version of PatchReader to kaulen for bugzilla | [production] | 
            
  | 20:37 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/SpecialNoticeText.php  'Picking up changes from r72135' | [production] |