| 2011-09-22
      
      § | 
    
  | 23:07 | <binasher> | running schema migrations against eqiad and analytics slaves | [production] | 
            
  | 22:19 | <Ryan_Lane> | tweeking performance settings for gerrit, restarting | [production] | 
            
  | 21:50 | <Ryan_Lane> | rebooting virt3, and virt2 | [production] | 
            
  | 21:50 | <Ryan_Lane> | rebooting virt4 | [production] | 
            
  | 20:59 | <aaron> | synchronized php-1.18/skins/common/edit.js | [production] | 
            
  | 20:49 | <notpeter> | putting srv274 and srv273 back in the apaches pool | [production] | 
            
  | 20:20 | <binasher> | additive liquidthread 1.18 migrations completed on all slaves, still need to do masters. some slaves could use an index drop | [production] | 
            
  | 19:58 | <mark> | Set static routes of 10.4.0.0/24 and 208.80.153.192/32 (in addition to 208.80.153.192/28) to 10.4.16.3 on cr1-sdtpa and cr2-pmtpa | [production] | 
            
  | 19:55 | <ben> | synchronized wmf-config/db.php  'db13 is now caught up with replication.  Putting it back in rotation.' | [production] | 
            
  | 19:49 | <mark> | Deconfigured cr1-sdtpa and cr2-pmtpa for vlan 103; they are no longer having an interface in that vlan | [production] | 
            
  | 19:40 | <binasher> | preparing to run liquidthreads 1.18 schema migrations against slaves | [production] | 
            
  | 19:21 | <Jeff_Green> | rebooting silicon to test firewall rule setup | [production] | 
            
  | 18:16 | <robh> | installing mw1-mw27 | [production] | 
            
  | 18:14 | <robh> | updated netboot apaches.cfg to include a tmp partition of 2gb | [production] | 
            
  | 17:59 | <ben> | synchronized wmf-config/db.php  'pulling db13 out of s2 because it looks like the check slave status cache does not work - the host is being pummelled by show slave status checks.' | [production] | 
            
  | 17:55 | <ben> | synchronized wmf-config/db.php  'db13 back in rotation now that slaving is fixed, though it\\'s still lagged,which is ok because MW will not use lagged slaves' | [production] | 
            
  | 17:31 | <AaronSchulz> | Hacked in common/edit.js symlink into 1.18 for LQT | [production] | 
            
  | 17:22 | <robh> | updating dns for new mw1-mw74 | [production] | 
            
  | 17:18 | <maplebed> | pulled db13 out of rotation (cluster s2) because slaving is broken | [production] | 
            
  | 17:17 | <ben> | synchronized wmf-config/db.php  'slaving is broken on db13. removing it from rotation in cluster s2.' | [production] | 
            
  | 16:38 | <notpeter> | returning srv273 and srv274 to apaches pool | [production] | 
            
  | 16:05 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Gendered namespaces for test2wiki' | [production] | 
            
  | 16:00 | <notpeter> | putting srv275 back into apaches pool and removing srv273 for upgrade to lucid | [production] | 
            
  | 15:42 | <notpeter> | returning srv276 to apaches and removing srv274 for upgrade to lucid | [production] | 
            
  | 15:30 | <notpeter> | removing srv275 from apaches pool for upgrade to lucid | [production] | 
            
  | 15:06 | <reedy> | synchronized php-1.18/includes/Linker.php  'Error handling for null given rather than a Title' | [production] | 
            
  | 15:05 | <notpeter> | removing srv276 from apaches pool for upgrade to lucid | [production] | 
            
  | 15:05 | <Reedy> | Who left the live hack in php-1.18 Linker.php type coercing public static function makeMediaLinkObj( Title $title, $text = '', $time = false ) { | [production] | 
            
  | 15:01 | <reedy> | synchronized php-1.18/includes/Linker.php  'Sync please?' | [production] | 
            
  | 15:00 | <reedy> | synchronized php-1.18/includes/Linker.php  'Testing for PHP fatal error in /home/wikipedia/common/php-1.18/includes/Linker.php line 831' | [production] | 
            
  | 14:58 | <reedy> | synchronized php-1.18/includes/Linker.php  'Testing for PHP fatal error in /home/wikipedia/common/php-1.18/includes/Linker.php line 831' | [production] | 
            
  | 14:57 | <reedy> | synchronized php-1.18/includes/Linker.php  'Testing for PHP fatal error in /home/wikipedia/common/php-1.18/includes/Linker.php line 831' | [production] | 
            
  | 12:01 | <mutante> | lvs1003's disk died - see RT #1549 | [production] | 
            
  | 11:40 | <mutante> | power cycling lvs1003 | [production] | 
            
  | 07:41 | <awjr> | restarting jenkins on grosley | [production] | 
            
  | 07:27 | <aaron> | synchronized php-1.18/extensions/SpamBlacklist/SpamBlacklist_body.php  'disabled wfDebugLog() call; UDP packets were way too large' | [production] | 
            
  | 06:31 | <tstarling> | synchronizing Wikimedia installation... : | [production] | 
            
  | 06:20 | <tstarling> | synchronized wmf-config/InitialiseSettings.php  'remove debug log group since the bug is isolated now' | [production] | 
            
  | 06:18 | <tstarling> | synchronized php-1.18/includes/User.php  'revert debugging hack' | [production] | 
            
  | 05:11 | <aaron> | synchronized wmf-config/flaggedrevs.php  'added extension namespace to $wgFlaggedRevsNamespaces for mediawikiwiki' | [production] | 
            
  | 04:58 | <tstarling> | synchronized php-1.18/includes/User.php  'debugging for getGroupPermissions warning' | [production] | 
            
  | 04:26 | <tstarling> | synchronized php-1.18/includes/User.php  'debugging patch for getGroupPermissions warning' | [production] | 
            
  | 04:26 | <tstarling> | synchronized wmf-config/InitialiseSettings.php  'debugging for getGroupPermissions' | [production] | 
            
  | 04:14 | <tstarling> | synchronized php-1.18/includes/User.php  'debugging patch for getGroupPermissions warning' | [production] | 
            
  | 04:13 | <aaron> | synchronized php-1.18/includes/Linker.php | [production] | 
            
  | 04:06 | <aaron> | synchronized php-1.18/includes/Linker.php | [production] | 
            
  | 04:02 | <aaron> | synchronized php-1.18/includes/User.php | [production] | 
            
  | 03:59 | <aaron> | synchronized php-1.18/includes/api/ApiQuerySiteinfo.php | [production] |