| 
      
        2012-07-19
      
      §
     | 
  
    
  | 22:48 | 
  <catrope> | 
  Finished syncing Wikimedia installation... :  | 
  [production] | 
            
  | 22:10 | 
  <catrope> | 
  Started syncing Wikimedia installation... :  | 
  [production] | 
            
  | 22:02 | 
  <binasher> | 
  deploying new mobile redirector to pmtpa text squids (currently inactive) | 
  [production] | 
            
  | 21:39 | 
  <kaldari> | 
  Started syncing Wikimedia installation... :  | 
  [production] | 
            
  | 21:36 | 
  <binasher> | 
  deploying new mobile redirector to eqiad text squids | 
  [production] | 
            
  | 21:36 | 
  <catrope> | 
  synchronized php-1.20wmf7/extensions/LastModified  'Remove E3Experiments cruft from LastModified' | 
  [production] | 
            
  | 21:07 | 
  <binasher> | 
  deploying new mobile redirector to esams text squids | 
  [production] | 
            
  | 18:52 | 
  <preilly> | 
  fixing subdomain check for zero vs m | 
  [production] | 
            
  | 18:52 | 
  <preilly> | 
  synchronized wmf-config/mobile.php  'fix subdomain check' | 
  [production] | 
            
  | 18:21 | 
  <RobH> | 
  updating dns with new zonefiles for legally won domain names | 
  [production] | 
            
  | 18:15 | 
  <Jeff_Green> | 
  storage3 dist-upgrade and reboot | 
  [production] | 
            
  | 17:01 | 
  <AaronSchulz> | 
  Running copyFileBackend.php for commons (shards c-f) | 
  [production] | 
            
  | 16:28 | 
  <cmjohnson1> | 
  mw60 powering down to replace DIMM B1 rt3287 | 
  [production] | 
            
  | 15:48 | 
  <RobH> | 
  hydrogen repaired per rt 3243 | 
  [production] | 
            
  | 15:37 | 
  <RobH> | 
  rebooting hydrogen to set bios redirection | 
  [production] | 
            
  | 14:18 | 
  <reedy> | 
  synchronized wmf-config/  | 
  [production] | 
            
  | 13:42 | 
  <Jeff_Green> | 
  dist-upgrade and reboot hosts in payments cluster | 
  [production] | 
            
  | 13:12 | 
  <Reedy> | 
  Pointed /h/w/c/php to php-1.20wmf7 | 
  [production] | 
            
  | 13:10 | 
  <Reedy> | 
  Deleted php-1.20wmf6/cache/l10n from mediawiki-installation | 
  [production] | 
            
  | 02:31 | 
  <LocalisationUpdate> | 
  completed (1.20wmf6) at Thu Jul 19 02:31:02 UTC 2012 | 
  [production] | 
            
  | 02:23 | 
  <LocalisationUpdate> | 
  completed (1.20wmf7) at Thu Jul 19 02:23:27 UTC 2012 | 
  [production] | 
            
  
    | 
      
        2012-07-18
      
      §
     | 
  
    
  | 22:02 | 
  <notpeter> | 
  ok, changed my mind one more time. removing srv194 from apaches pool for precise testing | 
  [production] | 
            
  | 21:41 | 
  <notpeter> | 
  scratch that. removing srv289 from apaches pool for precise testing, not mw1 | 
  [production] | 
            
  | 21:39 | 
  <notpeter> | 
  removing mw1 from apaches pool to do precise test install | 
  [production] | 
            
  | 21:28 | 
  <notpeter> | 
  adding php5 packages to precise-wikimedia repo | 
  [production] | 
            
  | 19:45 | 
  <cmjohnson1> | 
  srv281 powering down for HW checks | 
  [production] | 
            
  | 19:09 | 
  <cmjohnson1> | 
  srv278 powering down for hardware problems -- random reboot system already depooled | 
  [production] | 
            
  | 18:05 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: 295 remaining wikis to 1.20wmf7 | 
  [production] | 
            
  | 17:37 | 
  <cmjohnson1> | 
  mw8 powering down to replace DIMM A3 rt 3273 | 
  [production] | 
            
  | 17:25 | 
  <cmjohnson1> | 
  search32 powering down to replace DIMM B1 rt 3076 | 
  [production] | 
            
  | 16:55 | 
  <reedy> | 
  synchronized php-1.20wmf7/includes/api/ApiQuerySiteinfo.php  | 
  [production] | 
            
  | 12:52 | 
  <pp-pdf1> | 
  upgraded mwlib to 0.13.11, restarted all services | 
  [production] | 
            
  | 07:58 | 
  <hashar> | 
  gallium: added Firefox 14 to Testswarm, disabled Firefox 13. | 
  [production] | 
            
  | 02:47 | 
  <LocalisationUpdate> | 
  completed (1.20wmf6) at Wed Jul 18 02:47:23 UTC 2012 | 
  [production] | 
            
  | 02:24 | 
  <LocalisationUpdate> | 
  completed (1.20wmf7) at Wed Jul 18 02:24:14 UTC 2012 | 
  [production] | 
            
  
    | 
      
        2012-07-17
      
      §
     | 
  
    
  | 23:24 | 
  <Tim> | 
  on srv193: removed core dump files, disabled core dumping, restarted apache | 
  [production] | 
            
  | 19:52 | 
  <mlitn> | 
  Finished syncing Wikimedia installation... :  | 
  [production] | 
            
  | 19:09 | 
  <mlitn> | 
  Started syncing Wikimedia installation... :  | 
  [production] | 
            
  | 15:18 | 
  <mark> | 
  lvs1002 is back up and idling | 
  [production] | 
            
  | 15:15 | 
  <mark> | 
  Fixed serial console redirection after boot to OFF on lvs1002 | 
  [production] | 
            
  | 15:06 | 
  <mark> | 
  lvs4 is back up and serving traffic | 
  [production] | 
            
  | 14:52 | 
  <mark> | 
  Reinstalling lvs4 with Ubuntu Precise | 
  [production] | 
            
  | 14:48 | 
  <mark> | 
  Stopped PyBal on lvs4, failing over traffic to lvs3 | 
  [production] | 
            
  | 14:47 | 
  <mark> | 
  Reinstalling lvs1002 with Ubuntu Precise | 
  [production] | 
            
  | 14:46 | 
  <mark> | 
  Stopped PyBal on lvs1002, failing over traffic to lvs1005 | 
  [production] | 
            
  | 14:44 | 
  <mark> | 
  lvs1001 is back up and serving traffic | 
  [production] |