| 2011-09-17
      
      § | 
    
  | 13:00 | <domas> | manually placed /etc/php5/conf.d/apc-aux.conf on lucid apaches, apparently working APC cache helps a lot with request processing | [production] | 
            
  | 11:26 | <mark> | PyBal on lvs1-3 was in a BGP colliding loop with cr2-pmtpa, restarted PyBal on all three hosts | [production] | 
            
  | 10:44 | <domas> | made profile hitrate collection less crashy, restarted it (from ~tstarling/src/udpprofile/, not sure where else it should've been) | [production] | 
            
  | 10:44 | <mark> | Upgrade of cr2-eqiad complete | [production] | 
            
  | 10:34 | <mark> | Upgrading re1.cr2-eqiad to junos 10.4R7.5 | [production] | 
            
  | 10:28 | <mark> | Performing failover from cr2-eqiad re1 to re0 | [production] | 
            
  | 10:18 | <mark> | Upgrade of cr1-eqiad complete | [production] | 
            
  | 10:08 | <mark> | Upgrading re0.cr1-eqiad to junos 10.4R7.5 | [production] | 
            
  | 09:55 | <mark> | Upgrading re0.cr2-eqiad to junos 10.4R7.5 | [production] | 
            
  | 09:46 | <mark> | Performing failover from cr1-eqiad re0 to re1 | [production] | 
            
  | 09:46 | <mark> | Performing failover from cr1-eqiad re1 to re2 | [production] | 
            
  | 09:33 | <mark_> | Upgrading re1.cr1-eqiad to junos 10.4R7.5 | [production] | 
            
  | 02:22 | <LocalisationUpdate> | completed (1.17=aawiki 1.17-test) at Sat Sep 17 02:25:20 UTC 2011 | [production] | 
            
  | 00:00 | <notpeter> | removing srv288 from the apaches pool for upgrade to lucid | [production] | 
            
  
    | 2011-09-16
      
      § | 
    
  | 23:48 | <maplebed> | started copying data from all srv hosts with external storage databases to es1001.  I expect it to run all weekend. | [production] | 
            
  | 23:39 | <notpeter> | putting srv289 back in the apache pool | [production] | 
            
  | 23:12 | <aaron> | synchronized wmf-config/CommonSettings.php  'added require for Math extension, checks $wmgUseMath and version compare with $wmfVersionNumber so it does nothing yet' | [production] | 
            
  | 22:50 | <aaron> | synchronized wmf-config/InitialiseSettings.php  'added wmgUseMath setting (on by default); not yet used' | [production] | 
            
  | 21:27 | <AaronSchulz> | fixed usage of $mwVersionNums in set-group-write2 (same double quotes bug the sync scripts had before) | [production] | 
            
  | 20:51 | <aaron> | synchronized php-1.17/extensions/FlaggedRevs/specialpages/ReviewedVersions_body.php  'deployed [[rev:97331|r97331]]' | [production] | 
            
  | 20:51 | <Jeff_Green> | putting http redirect back in front of donate.wikimedia.org b/c fundraisers aren't ready for it | [production] | 
            
  | 20:51 | <aaron> | synchronized php-1.17/extensions/FlaggedRevs/api/FlaggedRevsApi.hooks.php  'deployed [[rev:97331|r97331]]' | [production] | 
            
  | 20:29 | <AaronSchulz> | fixed geshi on test2 & secure testwiki | [production] | 
            
  | 20:25 | <aaron> | ran sync-common-all | [production] | 
            
  | 20:22 | <AaronSchulz> | changed mwscript & mwscriptwikiset to use the /home version of scripts (people usually did this before and some things like TrustedXFF expect it) | [production] | 
            
  | 20:07 | <aaron> | synchronized wmf-config/InitialiseSettings.php  'Set wgAutopromoteOnceLogInRC for plwiki,plwikisource ahead of time - no effect yet - [[bugzilla:29655|bug 29655]]' | [production] | 
            
  | 20:02 | <aaron> | synchronized php-1.17/extensions/FlaggedRevs/FlaggedRevision.php  'deployed [[rev:97323|r97323]]' | [production] | 
            
  | 19:55 | <Jeff_Green> | repooled payments4 after its long strange testing hiatus | [production] | 
            
  | 19:55 | <aaron> | synchronized php-1.17-test/extensions/FlaggedRevs/FlaggedRevision.php  'deployed [[rev:97323|r97323]]' | [production] | 
            
  | 19:03 | <preilly> | fix for undefined method on wml card | [production] | 
            
  | 19:02 | <preilly> | synchronized php/extensions/MobileFrontend/MobileFrontend.php  'fix method call to undefined method' | [production] | 
            
  | 18:45 | <AaronSchulz> | added php-1.17-test/libs/geshi symlink to common/libs geshi, working on testwiki now | [production] | 
            
  | 18:43 | <AaronSchulz> | created common/lib directory and placed a copy of GeSHi-1.0.8.10 inside | [production] | 
            
  | 18:19 | <preilly> | add version to mobile application js to bust cache | [production] | 
            
  | 18:18 | <preilly> | synchronized php/extensions/MobileFrontend/views/layout/application.html.php  'remove jquery add version id to bust cache' | [production] | 
            
  | 17:55 | <preilly> | size correction for mobile icons | [production] | 
            
  | 17:55 | <preilly> | synchronized wmf-config/InitialiseSettings.php  'config project icons for mobile size corrections' | [production] | 
            
  | 15:19 | <reedy> | synchronized php/extensions/OggHandler/OggHandler_body.php  '[[rev:97289|r97289]]' | [production] | 
            
  | 15:18 | <reedy> | synchronized php/extensions/OggHandler/OggPlayer.js  '[[rev:97289|r97289]]' | [production] | 
            
  | 15:17 | <reedy> | synchronized php/languages/messages/MessagesKk_cyrl.php  '[[rev:97287|r97287]]' | [production] | 
            
  | 15:17 | <reedy> | synchronized php/includes/HttpFunctions.php  '[[rev:97287|r97287]]' | [production] | 
            
  | 14:53 | <domas> | killed whoever was running full table scans with subselects on enwiki and hitting http://bugs.mysql.com/bug.php?id=46947 - thanks, you locked up enwiki for many users :) | [production] | 
            
  | 12:26 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Bug 30928 - Make Portal namespace for shwikipedia' | [production] | 
            
  | 11:47 | <Reedy> | Run namespaceDupes on hywiki | [production] | 
            
  | 02:22 | <LocalisationUpdate> | completed (1.17=aawiki 1.17-test) at Fri Sep 16 02:24:43 UTC 2011 | [production] | 
            
  | 01:50 | <aaron> | synchronizing Wikimedia installation... : | [production] | 
            
  | 01:41 | <AaronSchulz> | moved test2wiki to php-1.17-test | [production] | 
            
  | 01:36 | <AaronSchulz> | testing with testwiki on php-1.17-test | [production] | 
            
  | 01:04 | <aaron> | synchronizing Wikimedia installation... : | [production] | 
            
  | 00:47 | <aaron> | synchronized wikiversions.dat  'set testwiki to 1.17-test' | [production] |