| 2010-08-17
      
      § | 
    
  | 16:41 | <rainman-sr> | all of search up, still fiddling with search11 to see why it gave strange I/O spikes during the batch2 migration | [production] | 
            
  | 16:37 | <RobH> | investigating srv230. | [production] | 
            
  | 16:32 | <RobH> | srv230 back online with memory replacement, synced and back in cluster | [production] | 
            
  | 16:31 | <robh> | synchronized php-1.5/wmf-config/lucene.php | [production] | 
            
  | 16:29 | <robh> | synchronized php-1.5/wmf-config/lucene.php  'Returning all search values to normal, should restore full search functionality.' | [production] | 
            
  | 16:22 | <rainman-sr> | bringing up search5,12, 13-20 | [production] | 
            
  | 16:21 | <RobH> | shutting down srv230 to swap out bad memory | [production] | 
            
  | 15:50 | <RobH> | search13-search20 relocated to b3-sdtpa.  All servers are online, working to bring search back to full deployment. | [production] | 
            
  | 14:36 | <rainman-sr> | search5,12 will also show offline because they run parts of the services that are temporarly disabled | [production] | 
            
  | 14:30 | <RobH> | search13-search20 will show offline during their relocation, approx until 16:00 if all things go well | [production] | 
            
  | 14:09 | <robh> | synchronized php-1.5/wmf-config/lucene.php  'Disabling wgEnableLucenePrefixSearch on projects while search servers are relocated' | [production] | 
            
  | 14:04 | <rainman-sr> | starting relocation of second batch of search servers | [production] | 
            
  | 13:05 | <andrew> | synchronized php-1.5/extensions/LiquidThreads_alpha/i18n/Lqt.namespaces.php | [production] | 
            
  | 13:05 | <andrew> | synchronized php-1.5/extensions/LiquidThreads/i18n/Lqt.namespaces.php | [production] | 
            
  | 12:19 | <andrew> | synchronizing Wikimedia installation... Revision: 71190 | [production] | 
            
  | 12:19 | <Andrew> | running scap to update LiquidThreads localisation | [production] | 
            
  | 11:01 | <andrew> | synchronized php-1.5/wmf-config/CommonSettings.php  'Add LiquidThreads to ptwikibooks' | [production] | 
            
  | 10:34 | <andrew> | synchronized php-1.5/extensions/LiquidThreads_alpha/LiquidThreads.php | [production] | 
            
  | 10:34 | <andrew> | synchronized php-1.5/extensions/LiquidThreads_alpha/i18n/Lqt.namespaces.php | [production] | 
            
  | 10:33 | <andrew> | synchronized php-1.5/extensions/LiquidThreads_alpha/i18n/Lqt.i18n.php | [production] | 
            
  | 10:33 | <andrew> | synchronized php-1.5/extensions/LiquidThreads/LiquidThreads.php | [production] | 
            
  | 10:33 | <andrew> | synchronized php-1.5/extensions/LiquidThreads/i18n/Lqt.namespaces.php | [production] | 
            
  | 10:33 | <andrew> | synchronized php-1.5/extensions/LiquidThreads/i18n/Lqt.i18n.php | [production] | 
            
  | 05:56 | <jeluf> | synchronized php-1.5/wmf-config/CommonSettings.php  '22109 -  add fourth level subdomains of wikimedia to ' | [production] | 
            
  | 05:42 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24808 - Localize Wikipedia name for newiki' | [production] | 
            
  | 05:34 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24355 - Namespace changes - si.wikipedia' | [production] | 
            
  | 05:31 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24255 - Change meta namespace names on siwikibooks' | [production] | 
            
  | 05:24 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24579 - Reconfigure Turkish Wikibooks' | [production] | 
            
  | 05:07 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24822 - XML-import for de-wiktionary' | [production] | 
            
  | 05:05 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24823 - alias for de-wiktionary' | [production] | 
            
  | 04:09 | <Tim> | wikimedia-task-appserver on srv224 was half-configured because parentless convert processes stuck in a deadlock were holding the *:80 listen filehandle open, preventing apache from starting. Killed the convert processes and reinstalled. | [production] | 
            
  | 04:05 | <Tim> | deployment of the new wikimedia-task-appserver led to nagios alerts about half-configured status on 16 servers due to a warning in rsync sync-common about file permissions. Fixed by deleting Mark's vim swapfile and reinstalling. | [production] | 
            
  | 03:36 | <tstarling> | synchronized php-1.5/wmf-config/CommonSettings.php  'disabling PagedTiffHandler, no identify present on non-scalers so upload fails' | [production] | 
            
  | 03:26 | <tstarling> | synchronized php-1.5/wmf-config/CommonSettings.php  'enabling PagedTiffHandler' | [production] | 
            
  | 03:23 | <tstarling> | synchronizing Wikimedia installation... Revision: 71190 | [production] | 
            
  | 03:22 | <Tim> | doing another scap to update ExtensionMessages.php | [production] | 
            
  | 03:16 | <root> | synchronizing Wikimedia installation... Revision: 71190 | [production] | 
            
  | 03:14 | <Tim> | doing svn up/scap to r71188 for PagedTiffHandler and supporting changes | [production] | 
            
  
    | 2010-08-16
      
      § | 
    
  | 21:44 | <RobH> | dns update for login vmhost on tesla, rt#108 | [production] | 
            
  | 20:38 | <Fred> | rebooting srv217 ->oom | [production] | 
            
  | 19:15 | <mark> | synchronized php-1.5/extensions/MWSearch/MWSearch_body.php  'Reenabling the disabled section, as rainman has disabled interwiki results in the lucene backend' | [production] | 
            
  | 18:51 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24304 - Reconfigure English Wikibooks' | [production] | 
            
  | 18:50 | <jeluf> | synchronized php-1.5/wmf-config/flaggedrevs.php  '24304 -  Reconfigure English Wikibooks' | [production] | 
            
  | 18:49 | <rainman-sr> | restarting search1,3,4,9 | [production] | 
            
  | 18:46 | <root> | synchronized php-1.5/extensions/MWSearch/MWSearch_body.php  'Temporarily disable part of MWSearch' | [production] | 
            
  | 18:45 | <JeLuF> | archived admin logs, moved January-June 2010 to Archive 15 | [production] | 
            
  | 18:45 | <root> | synchronized php-1.5/extensions/MWSearch/MWSearch_body.php  'Temporarily disable part of MWSearch' | [production] | 
            
  | 18:43 | <root> | synchronized php-1.5/extensions/MWSearch/MWSearch_body.php  'Temporarily disable part of MWSearch' | [production] | 
            
  | 18:38 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '17627 - Allow autoconfirmed users to patrol on ar Wikisource' | [production] | 
            
  | 18:08 | <Fred> | MWSearch extension is having issues... | [production] |