| 2012-01-11
      
      § | 
    
  | 19:40 | <RobH> | mw1103 hardware issues, disregard nagios flapping | [production] | 
            
  | 19:37 | <RobH> | mw1102 offline due to bad mainboard until replacement arrives tomorrow or next | [production] | 
            
  | 19:30 | <RobH> | working on mw1102, disregard flapping | [production] | 
            
  | 18:40 | <notpeter> | running authdns-update on dobson to pick up new dns temps | [production] | 
            
  | 18:28 | <RobH> | lvs1003 repaired, now needs install and setup.  rt1549 and rt 2241 | [production] | 
            
  | 18:26 | <notpeter> | gracefulling apache on spence to deactivate nmis.w.o (abandoned install of nedi) | [production] | 
            
  | 16:23 | <mark> | Started rsync of lily:/var/lib/mailman/archives to sodium (in a screen on sodium) | [production] | 
            
  | 15:49 | <mark> | Started rsync of lily:/var/lib/mailman/data to sodium (in a screen on sodium) | [production] | 
            
  | 15:39 | <reedy> | synchronized php-1.18/includes/  '[[rev:108626|r108626]]' | [production] | 
            
  | 15:34 | <reedy> | synchronized php-1.18/includes/  'revert [[rev:108625|r108625]]' | [production] | 
            
  | 15:32 | <reedy> | synchronized php-1.18/includes/  '[[rev:108625|r108625]]' | [production] | 
            
  | 15:23 | <reedy> | synchronized php-1.18/extensions/CodeReview/  '[[rev:108623|r108623]]' | [production] | 
            
  | 15:22 | <reedy> | synchronized php-1.18/extensions/ArticleFeedbackv5/  '[[rev:108623|r108623]]' | [production] | 
            
  | 15:22 | <reedy> | synchronized php-1.18/extensions/ApiSandbox/  '[[rev:108623|r108623]]' | [production] | 
            
  | 15:20 | <reedy> | synchronized php-1.18/resources/mediawiki.action/  '[[rev:108622|r108622]]' | [production] | 
            
  | 15:19 | <reedy> | synchronized php-1.18/includes/  '[[rev:108622|r108622]]' | [production] | 
            
  | 14:11 | <mutante> | nagios https now serves real SSL cert | [production] | 
            
  | 14:09 | <mutante> | fixed Apache VirtualHost warnings on spence, NameVirtualHost *:443 in ports.conf, <VirtualHost *:443> in sites-available,.. | [production] | 
            
  | 02:04 | <LocalisationUpdate> | completed (1.18) at Wed Jan 11 02:04:39 UTC 2012 | [production] | 
            
  | 01:42 | <preilly> | pushing updates to Zero Rated Mobile Access extension | [production] | 
            
  | 01:41 | <preilly> | synchronized php-1.18/extensions/ZeroRatedMobileAccess/  'push updates to ZeroRatedMobileAccess extension' | [production] | 
            
  | 01:18 | <preilly> | only activate Zero Rated Mobile Access Extension for test wiki | [production] | 
            
  | 00:38 | <preilly> | synchronized wmf-config/InitialiseSettings.php  'add ZeroRatedMobileAccess extension only on test' | [production] | 
            
  | 00:34 | <preilly> | pushing ZeroRatedMobileAccess extension to production | [production] | 
            
  | 00:34 | <preilly> | synchronized wmf-config/InitialiseSettings.php  'add ZeroRatedMobileAccess extension' | [production] | 
            
  | 00:34 | <preilly> | synchronized wmf-config/CommonSettings.php  'add ZeroRatedMobileAccess extension' | [production] | 
            
  | 00:33 | <preilly> | synchronized wmf-config/extension-list  'add ZeroRatedMobileAccess extension' | [production] | 
            
  | 00:30 | <preilly> | push ZeroRatedMobileAccess extension | [production] | 
            
  | 00:30 | <preilly> | synchronized php-1.18/extensions/ZeroRatedMobileAccess/  'initial push of ZeroRatedMobileAccess extension' | [production] | 
            
  | 00:23 | <LeslieCarr> | applying new loopback filter to cr1-eqiad - higher risk of issues | [production] | 
            
  
    | 2012-01-10
      
      § | 
    
  | 23:58 | <preilly> | weekly mobile frontend push | [production] | 
            
  | 23:58 | <preilly> | synchronized php-1.18/extensions/MobileFrontend/  'weekly update to mobile frontend' | [production] | 
            
  | 22:10 | <lcarr> | broke ganglia redirect on nickel, fixing with next push | [production] | 
            
  | 19:45 | <LeslieCarr> | stopping gmetad on spence and unmounting the tmpfs drive | [production] | 
            
  | 18:32 | <LeslieCarr> | restarting gmetad on nickel | [production] | 
            
  | 11:39 | <nikerabbit> | synchronized php-1.18/extensions/Translate/MessageGroups.php  'Translate bugfix [[rev:108500|r108500]]' | [production] | 
            
  | 07:20 | <nikerabbit> | synchronized php-1.18/extensions/Translate/TranslateEditAddons.php  '[[rev:108497|r108497]]' | [production] | 
            
  | 02:05 | <LocalisationUpdate> | completed (1.18) at Tue Jan 10 02:05:15 UTC 2012 | [production] | 
            
  | 01:31 | <binasher> | all varnish servers have been upgraded to 3.0.2 | [production] | 
            
  | 00:38 | <RobH> | db1004 pd8 set to offline per rt 2140, will place call to dell for replacement | [production] | 
            
  | 00:23 | <RobH> | correction for typo, mw1102, not mw1002 | [production] | 
            
  | 00:23 | <binasher> | repooled cp3002 | [production] | 
            
  | 00:23 | <RobH> | mw1002 coming down for hw testing rt 1656 | [production] | 
            
  | 00:19 | <binasher> | depooling cp3002, upgrading varnish | [production] |