| 2012-07-19
      
      § | 
    
  | 18:52 | <preilly> | fixing subdomain check for zero vs m | [production] | 
            
  | 18:52 | <preilly> | synchronized wmf-config/mobile.php  'fix subdomain check' | [production] | 
            
  | 18:21 | <RobH> | updating dns with new zonefiles for legally won domain names | [production] | 
            
  | 18:15 | <Jeff_Green> | storage3 dist-upgrade and reboot | [production] | 
            
  | 17:01 | <AaronSchulz> | Running copyFileBackend.php for commons (shards c-f) | [production] | 
            
  | 16:28 | <cmjohnson1> | mw60 powering down to replace DIMM B1 rt3287 | [production] | 
            
  | 15:48 | <RobH> | hydrogen repaired per rt 3243 | [production] | 
            
  | 15:37 | <RobH> | rebooting hydrogen to set bios redirection | [production] | 
            
  | 14:18 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 13:42 | <Jeff_Green> | dist-upgrade and reboot hosts in payments cluster | [production] | 
            
  | 13:12 | <Reedy> | Pointed /h/w/c/php to php-1.20wmf7 | [production] | 
            
  | 13:10 | <Reedy> | Deleted php-1.20wmf6/cache/l10n from mediawiki-installation | [production] | 
            
  | 02:31 | <LocalisationUpdate> | completed (1.20wmf6) at Thu Jul 19 02:31:02 UTC 2012 | [production] | 
            
  | 02:23 | <LocalisationUpdate> | completed (1.20wmf7) at Thu Jul 19 02:23:27 UTC 2012 | [production] | 
            
  
    | 2012-07-18
      
      § | 
    
  | 22:02 | <notpeter> | ok, changed my mind one more time. removing srv194 from apaches pool for precise testing | [production] | 
            
  | 21:41 | <notpeter> | scratch that. removing srv289 from apaches pool for precise testing, not mw1 | [production] | 
            
  | 21:39 | <notpeter> | removing mw1 from apaches pool to do precise test install | [production] | 
            
  | 21:28 | <notpeter> | adding php5 packages to precise-wikimedia repo | [production] | 
            
  | 19:45 | <cmjohnson1> | srv281 powering down for HW checks | [production] | 
            
  | 19:09 | <cmjohnson1> | srv278 powering down for hardware problems -- random reboot system already depooled | [production] | 
            
  | 18:05 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: 295 remaining wikis to 1.20wmf7 | [production] | 
            
  | 17:37 | <cmjohnson1> | mw8 powering down to replace DIMM A3 rt 3273 | [production] | 
            
  | 17:25 | <cmjohnson1> | search32 powering down to replace DIMM B1 rt 3076 | [production] | 
            
  | 16:55 | <reedy> | synchronized php-1.20wmf7/includes/api/ApiQuerySiteinfo.php | [production] | 
            
  | 12:52 | <pp-pdf1> | upgraded mwlib to 0.13.11, restarted all services | [production] | 
            
  | 07:58 | <hashar> | gallium: added Firefox 14 to Testswarm, disabled Firefox 13. | [production] | 
            
  | 02:47 | <LocalisationUpdate> | completed (1.20wmf6) at Wed Jul 18 02:47:23 UTC 2012 | [production] | 
            
  | 02:24 | <LocalisationUpdate> | completed (1.20wmf7) at Wed Jul 18 02:24:14 UTC 2012 | [production] | 
            
  
    | 2012-07-17
      
      § | 
    
  | 23:24 | <Tim> | on srv193: removed core dump files, disabled core dumping, restarted apache | [production] | 
            
  | 19:52 | <mlitn> | Finished syncing Wikimedia installation... : | [production] | 
            
  | 19:09 | <mlitn> | Started syncing Wikimedia installation... : | [production] | 
            
  | 15:18 | <mark> | lvs1002 is back up and idling | [production] | 
            
  | 15:15 | <mark> | Fixed serial console redirection after boot to OFF on lvs1002 | [production] | 
            
  | 15:06 | <mark> | lvs4 is back up and serving traffic | [production] | 
            
  | 14:52 | <mark> | Reinstalling lvs4 with Ubuntu Precise | [production] | 
            
  | 14:48 | <mark> | Stopped PyBal on lvs4, failing over traffic to lvs3 | [production] | 
            
  | 14:47 | <mark> | Reinstalling lvs1002 with Ubuntu Precise | [production] | 
            
  | 14:46 | <mark> | Stopped PyBal on lvs1002, failing over traffic to lvs1005 | [production] | 
            
  | 14:44 | <mark> | lvs1001 is back up and serving traffic | [production] | 
            
  | 14:11 | <mark> | Stopped PyBal on lvs1001, failing over traffic to lvs1004 | [production] | 
            
  | 13:09 | <maplebed> | changed auth URL for swift to use load balancer rather than round robin DNS | [production] | 
            
  | 13:08 | <mark> | lvs1003 is back up and serving traffic | [production] | 
            
  | 13:07 | <maplebed> | changed order on ganglia swift view graphs to group by metric rather than host | [production] | 
            
  | 13:02 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 12:53 | <mark> | Fixed boot order on lvs1003 | [production] | 
            
  | 12:34 | <mark> | Reinstalling lvs1003 with Ubuntu Precise | [production] | 
            
  | 12:28 | <mark> | Stopped PyBal on lvs1003, failing over traffic to lvs1006 | [production] | 
            
  | 12:10 | <mark> | amslvs1 is back up and serving traffic | [production] | 
            
  | 11:44 | <mark> | Reinstalling amslvs1 with Ubuntu Precise | [production] | 
            
  | 11:40 | <mark> | Stopped PyBal on amslvs1, failing over traffic to amslvs3 | [production] |