| 
      
        2010-03-31
      
      §
     | 
  
    
  | 21:16 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'Don't totally disable collapsiblenav, keep it as a hidden pref' | 
  [production] | 
            
  | 20:31 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'Bump style version appendix' | 
  [production] | 
            
  | 20:30 | 
  <catrope> | 
  synchronized php-1.5/extensions/UsabilityInitiative/UsabilityInitiative.hooks.php  'r64455' | 
  [production] | 
            
  | 20:30 | 
  <catrope> | 
  synchronized php-1.5/extensions/UsabilityInitiative/css/combined.min.css  'r64455' | 
  [production] | 
            
  | 20:29 | 
  <catrope> | 
  synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js  'r64455' | 
  [production] | 
            
  | 20:29 | 
  <catrope> | 
  synchronized php-1.5/extensions/UsabilityInitiative/WikiEditor/WikiEditor.combined.min.js  'r64455' | 
  [production] | 
            
  | 19:55 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'Disable collapsiblenav explicitly in wake of usability deployment' | 
  [production] | 
            
  | 18:19 | 
  <RoanKattouw> | 
  Manually corrected time on prototype, was 6 hours behind. Guess we need NTP there too | 
  [production] | 
            
  | 16:56 | 
  <Rob> | 
  rebooted srv119 | 
  [production] | 
            
  | 16:31 | 
  <Rob> | 
  srv89 had temp warnings and failure to respond to console, replaced two bad fans in chassis with fans from srv127 | 
  [production] | 
            
  | 16:22 | 
  <Rob> | 
  srv98 was unresponsive to console, rebooted and appears to be back online and working just fine for now. | 
  [production] | 
            
  | 16:20 | 
  <Rob> | 
  srv127 decomissioned.  dead hard disks, dead fans, temp warnings, no lom, and out of warranty.  will be parted out to other servers | 
  [production] | 
            
  | 16:08 | 
  <Rob> | 
  srv127 throwing both hard disk errors and temp warnings | 
  [production] | 
            
  | 15:00 | 
  <Rob> | 
  db28 is not in any current use to to memory issue, investigating memory issue now for hardware warranty replacement | 
  [production] | 
            
  | 05:33 | 
  <jeluf> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'Tim says this helps' | 
  [production] | 
            
  | 05:02 | 
  <jeluf> | 
  ran sync-common-all  | 
  [production] | 
            
  | 04:59 | 
  <jeluf> | 
  ran sync-common-all  | 
  [production] | 
            
  | 00:50 | 
  <Tim> | 
  increased apache TimeOut setting on singer from 5 to 30 seconds, to hopefully fix multiple reports of high error rates on secure.wikimedia.org | 
  [production] | 
            
  
    | 
      
        2010-03-30
      
      §
     | 
  
    
  | 20:18 | 
  <Rob> | 
  new server gilman racked for future secure gateway use and dns updated for server fqdn | 
  [production] | 
            
  | 20:17 | 
  <Rob> | 
  back to work morebots | 
  [production] | 
            
  | 17:22 | 
  <jeluf> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  '22650 - bureaucrats should only be allowed to remove abusefilter and bot, not the other stuff' | 
  [production] | 
            
  | 09:21 | 
  <andrew> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'Bump style version' | 
  [production] | 
            
  | 09:20 | 
  <andrew> | 
  synchronized php-1.5/extensions/LiquidThreads/lqt.css  'Deploy r64384' | 
  [production] | 
            
  | 09:20 | 
  <andrew> | 
  synchronized php-1.5/extensions/LiquidThreads_alpha/lqt.css  'Deploy r64384' | 
  [production] | 
            
  | 05:28 | 
  <jeluf> | 
  ran sync-common-all  | 
  [production] | 
            
  | 04:39 | 
  <tstarling> | 
  synchronized php-1.5/extensions/InputBox/InputBox.classes.php  | 
  [production] | 
            
  | 04:29 | 
  <Tim> | 
  removed the ex-fedora instances from srv122, srv124, srv142 from ms1 | 
  [production] | 
            
  | 00:29 | 
  <Tim> | 
  copying half the data back from ms1 to ms2 so I can have a backup, with half the data on each server | 
  [production] |