| 2010-03-31
      
      § | 
    
  | 20:30 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/UsabilityInitiative.hooks.php  'r64455' | [production] | 
            
  | 20:30 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/css/combined.min.css  'r64455' | [production] | 
            
  | 20:29 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js  'r64455' | [production] | 
            
  | 20:29 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/WikiEditor/WikiEditor.combined.min.js  'r64455' | [production] | 
            
  | 19:55 | <catrope> | synchronized php-1.5/wmf-config/CommonSettings.php  'Disable collapsiblenav explicitly in wake of usability deployment' | [production] | 
            
  | 18:19 | <RoanKattouw> | Manually corrected time on prototype, was 6 hours behind. Guess we need NTP there too | [production] | 
            
  | 16:56 | <Rob> | rebooted srv119 | [production] | 
            
  | 16:31 | <Rob> | srv89 had temp warnings and failure to respond to console, replaced two bad fans in chassis with fans from srv127 | [production] | 
            
  | 16:22 | <Rob> | srv98 was unresponsive to console, rebooted and appears to be back online and working just fine for now. | [production] | 
            
  | 16:20 | <Rob> | srv127 decomissioned.  dead hard disks, dead fans, temp warnings, no lom, and out of warranty.  will be parted out to other servers | [production] | 
            
  | 16:08 | <Rob> | srv127 throwing both hard disk errors and temp warnings | [production] | 
            
  | 15:00 | <Rob> | db28 is not in any current use to to memory issue, investigating memory issue now for hardware warranty replacement | [production] | 
            
  | 05:33 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Tim says this helps' | [production] | 
            
  | 05:02 | <jeluf> | ran sync-common-all | [production] | 
            
  | 04:59 | <jeluf> | ran sync-common-all | [production] | 
            
  | 00:50 | <Tim> | increased apache TimeOut setting on singer from 5 to 30 seconds, to hopefully fix multiple reports of high error rates on secure.wikimedia.org | [production] | 
            
  
    | 2010-03-30
      
      § | 
    
  | 20:18 | <Rob> | new server gilman racked for future secure gateway use and dns updated for server fqdn | [production] | 
            
  | 20:17 | <Rob> | back to work morebots | [production] | 
            
  | 17:22 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '22650 - bureaucrats should only be allowed to remove abusefilter and bot, not the other stuff' | [production] | 
            
  | 09:21 | <andrew> | synchronized php-1.5/wmf-config/CommonSettings.php  'Bump style version' | [production] | 
            
  | 09:20 | <andrew> | synchronized php-1.5/extensions/LiquidThreads/lqt.css  'Deploy r64384' | [production] | 
            
  | 09:20 | <andrew> | synchronized php-1.5/extensions/LiquidThreads_alpha/lqt.css  'Deploy r64384' | [production] | 
            
  | 05:28 | <jeluf> | ran sync-common-all | [production] | 
            
  | 04:39 | <tstarling> | synchronized php-1.5/extensions/InputBox/InputBox.classes.php | [production] | 
            
  | 04:29 | <Tim> | removed the ex-fedora instances from srv122, srv124, srv142 from ms1 | [production] | 
            
  | 00:29 | <Tim> | copying half the data back from ms1 to ms2 so I can have a backup, with half the data on each server | [production] | 
            
  
    | 2010-03-26
      
      § | 
    
  | 21:46 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '#22919 -  Enable FlaggedRevs for pl.wikisource' | [production] | 
            
  | 21:45 | <jeluf> | synchronized flaggedrevs.dblist | [production] | 
            
  | 21:44 | <jeluf> | synchronized php-1.5/wmf-config/flaggedrevs.php | [production] | 
            
  | 21:41 | <catrope> | synchronized php-1.5/wmf-config/CommonSettings.php  'Bump style version appendix' | [production] | 
            
  | 21:41 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/js/plugins/jquery.wikiEditor.html  'r64223' | [production] | 
            
  | 21:39 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '#22965 -  wgUploadNavigationUrl for nl.wikipedia' | [production] | 
            
  | 20:40 | <Fred> | restarted DNS server on ns2 | [production] | 
            
  | 20:34 | <Rob> | added dns support for new misc server tarin | [production] | 
            
  | 19:44 | <Rob> | ignore nagioswarnings for sq11-sq30, they are being decommissioned | [production] |