| 2011-08-01
      
      § | 
    
  | 23:25 | <aaron> | synchronized php/extensions/MobileFrontend/library/WURFL/CustomDeviceRepository.php  'pre-OOM logging' | [production] | 
            
  | 23:19 | <notpeter> | upgrading squid and squid-frontend on sq33 and sq34 | [production] | 
            
  | 23:15 | <notpeter> | restarting broken torrus again again. | [production] | 
            
  | 22:58 | <aaron> | synchronized live-1.5/robots.php  'commented out redundant code' | [production] | 
            
  | 22:57 | <notpeter> | upgrading squid and squid-frontend on sq35 and sq36 | [production] | 
            
  | 22:49 | <aaron> | synchronized multiversion/MWVersion.php  'fix for robots.php hits on bogus wikis' | [production] | 
            
  | 22:47 | <aaron> | synchronized multiversion/MWVersion.php | [production] | 
            
  | 22:26 | <notpeter> | upgrading squid and squid-frontend on sq37 and sq38 | [production] | 
            
  | 22:09 | <notpeter> | upgrading squid and squid-frontend on sq39 and sq40 | [production] | 
            
  | 21:51 | <notpeter> | restarting broken torrus again | [production] | 
            
  | 21:18 | <Reedy> | srv187 although up and alive, isn't reporting any metrics to ganglia, and uptime is completely wrong | [production] | 
            
  | 21:15 | <notpeter> | upgrading squid and squid-frontend on sq59 and sq60 | [production] | 
            
  | 21:12 | <aaron> | synchronized multiversion/MWVersion.php | [production] | 
            
  | 21:10 | <aaron> | synchronized wmf-config  'multiversion/MWVersion.php missing.php fix' | [production] | 
            
  | 20:46 | <aaron> | synchronized wmf-config/CommonSettings.php  'moved two lines of code down a bit to allow usage of missing.php' | [production] | 
            
  | 20:33 | <aaron> | synchronized multiversion/MWMultiVersion.php  'removed transition hack (cdb rebuilt)' | [production] | 
            
  | 20:29 | <AaronSchulz> | wikiversions.cdb rebuilt | [production] | 
            
  | 20:21 | <aaron> | synchronizing Wikimedia installation... Revision: 93132: | [production] | 
            
  | 20:17 | <aaron> | synchronized multiversion/MWMultiVersion.php  'hardcode to 1.17 for safe transition' | [production] | 
            
  | 20:09 | <aaron> | synchronized live-1.5/MWVersion.php | [production] | 
            
  | 19:59 | <notpeter> | upgrading squid and squid-frontend on sq61 and sq62 | [production] | 
            
  | 19:37 | <notpeter> | upgrading squid and squid-frontend on sq64 | [production] | 
            
  | 16:15 | <mark> | Deployed puppet generated pybal.conf on amslvs1,2,3,4 | [production] | 
            
  | 15:47 | <mutante> | amssq55: power cycle, fsck forces add. reboot, clean squid cache, dist-upgrade,.. | [production] | 
            
  | 15:43 | <mark> | Deployed PyBal generated pybal.conf on amslvs3 | [production] | 
            
  | 15:25 | <mutante> | amssq53: cleaned squid cache, now dist-upgrade and another reboot | [production] | 
            
  | 15:07 | <mark> | Temporarily filtering traffic from 10.0.0.0/16 to puppet on sockpuppet | [production] | 
            
  | 14:56 | <mutante> | power cycle amssq53 | [production] | 
            
  | 12:36 | <mutante> | restarted ntpd on ms3 to fix a Nagios CRIT | [production] | 
            
  | 12:14 | <mark> | Changed IP address of bits.pmtpa.wikimedia.org from .118 to 208.80.152.210 | [production] | 
            
  | 11:03 | <mark> | Moved LVS services configuration out of site.pp into lvs.pp, in puppet | [production] | 
            
  | 10:03 | <mark> | Started db20 | [production] | 
            
  
    | 2011-07-30
      
      § | 
    
  | 23:54 | <reedy> | synchronized php-1.17/extensions/MobileFrontend/MobileFrontend.php  'htmlspecialchars url' | [production] | 
            
  | 23:48 | <reedy> | synchronized php-1.17/extensions/MobileFrontend/MobileFrontend.php  'htmlspecialchars url' | [production] | 
            
  | 23:39 | <reedy> | synchronized php-1.17/extensions/MobileFrontend/MobileFrontend.php  'Urlencode url' | [production] | 
            
  | 23:32 | <reedy> | synchronized php-1.17/extensions/MobileFrontend/MobileFrontend.php  'Urlencode url' | [production] | 
            
  | 23:02 | <reedy> | synchronized wmf-config/CommonSettings.php  'Bump wgWikimediaMobileVersion to .3' | [production] | 
            
  | 22:59 | <Reedy> | WikimediaMobile javascript giving 404 error whilst log out. Going to bump $wgWikimediaMobileVersion | [production] | 
            
  | 22:15 | <jeluf> | synchronized wmf-config/InitialiseSettings.php  '30122 - Add "WP" namespace alias to Test Wikipedia' | [production] | 
            
  | 22:15 | <jeluf> | synchronized wmf-config/InitialiseSettings.php  '30122' | [production] | 
            
  | 22:08 | <jeluf> | synchronized wmf-config/InitialiseSettings.php  '30138 - Please enable AbuseFilter for ee.wikipedia' | [production] | 
            
  | 22:06 | <jeluf> | synchronized wmf-config/abusefilter.php  '30138 - Please enable AbuseFilter for ee.wikipedia' | [production] | 
            
  | 20:35 | <jeluf> | synchronized wmf-config/InitialiseSettings.php  '30121 - Test Wikipedia request for file mover user group' | [production] | 
            
  | 20:27 | <jeluf> | synchronized wmf-config/InitialiseSettings.php  '25181 - Enable transwiki import from zhwiki to other zh projects' | [production] | 
            
  | 20:20 | <jeluf> | synchronized wmf-config/InitialiseSettings.php  '30136 - Enable recent changes patrolling on Portuguese Wikibooks' | [production] | 
            
  | 19:09 | <binasher> | db18 mysqld is back up and catching up on replication. may have a bad dimm (lots of ecc errors), leaving out of rotation for now | [production] | 
            
  | 18:48 | <binasher> | pulling db18 from rotation, appears to be having hw troubles | [production] |