| 2013-02-19
      
      § | 
    
  | 23:04 | <maxsem> | Finished syncing Wikimedia installation... : Weekly mobile deployment | [production] | 
            
  | 23:03 | <maxsem> | synchronized php-1.21wmf9/extensions/MobileFrontend/ | [production] | 
            
  | 22:49 | <maxsem> | Started syncing Wikimedia installation... : Weekly mobile deployment | [production] | 
            
  | 22:41 | <preilly> | synchronized php-1.21wmf9/extensions/ArticleFeedback  'fix MAX call with GREATEST' | [production] | 
            
  | 22:29 | <RobH> | sq41 offline, rt 4550 | [production] | 
            
  | 22:20 | <RobH> | sq41 puppet freshness bad, wont accept ssh connections, serial redirection results in no output.  will need to reboot it | [production] | 
            
  | 22:17 | <RobH> | sq41 not allowing ssh, drac resetting | [production] | 
            
  | 22:15 | <sbernardin> | taking down ms-be12 for hardware replacement per: https://rt.wikimedia.org/Ticket/Display.html?id=4546 | [production] | 
            
  | 22:15 | <RobH> | sq79/80 puppet in locked state, manually restarted puppet, removed lock file, ran manually | [production] | 
            
  | 22:11 | <preilly> | synchronized php-1.21wmf9/extensions/ArticleFeedback  'fix mysql 5.5 / ansi sql incompatibility' | [production] | 
            
  | 21:54 | <RobH> | mw48 was set to puppet agent disabled, re-enabled and fired puppet run | [production] | 
            
  | 20:04 | <mflaschen> | synchronized .  'Syncing GuidedTour submodule omitted when new branch cut was first cloned.' | [production] | 
            
  | 19:59 | <Ryan_Lane> | disabling project storage volumes for all labs projects with no instances | [production] | 
            
  | 15:02 | <hashar> | restarting jenkins to enable some startup setting | [production] | 
            
  | 14:58 | <hashar> | manganese CPU spike ongoing. Started roughly at 14:40 UTC | [production] | 
            
  | 13:29 | <mutante> | wikimedia.jp.net now redirects to wikimedia.org | wikimediacommons.jp.net now redirects to commons.wm | [production] | 
            
  | 13:27 | <mutante> | DNS update - adding wikimedia.jp.net | [production] | 
            
  | 13:25 | <dzahn> | gracefulled all apaches | [production] | 
            
  | 13:20 | <mutante> | gracefulling eqiad Apaches via dsh to push redirect change for .jp.net domains | [production] | 
            
  | 12:39 | <apergos> | powercycling ms-be7 (unresponsive to ping or from console) | [production] | 
            
  | 11:29 | <mutante> | installing package upgrads on neon/icinga | [production] | 
            
  | 02:49 | <LocalisationUpdate> | completed (1.21wmf10) at Tue Feb 19 02:49:32 UTC 2013 | [production] | 
            
  | 02:28 | <LocalisationUpdate> | completed (1.21wmf9) at Tue Feb 19 02:28:25 UTC 2013 | [production] | 
            
  | 01:12 | <anomie> | synchronized php-1.21wmf10/includes/api/ApiQuerySiteinfo.php  'Backport fix for ugly API bug' | [production] | 
            
  | 00:42 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: | [production] | 
            
  | 00:30 | <tstarling> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  
    | 2013-02-18
      
      § | 
    
  | 23:51 | <tstarling> | synchronized php-1.21wmf9/extensions/Scribunto | [production] | 
            
  | 23:51 | <tstarling> | synchronized php-1.21wmf10/extensions/Scribunto | [production] | 
            
  | 23:46 | <tstarling> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 23:25 | <Tim> | on snapshot1002 -- fixed incorrect owner on some MW files, killed broken apt-get and upgraded wikimedia-task-appserver | [production] | 
            
  | 23:24 | <Tim> | fixed rsyncd.conf on scap proxies to allow connections from new apaches | [production] | 
            
  | 23:14 | <tstarling> | Finished syncing Wikimedia installation... : | [production] | 
            
  | 23:11 | <tstarling> | Started syncing Wikimedia installation... : | [production] | 
            
  | 22:45 | <tstarling> | Finished syncing Wikimedia installation... : | [production] | 
            
  | 22:42 | <tstarling> | Started syncing Wikimedia installation... : | [production] | 
            
  | 22:37 | <tstarling> | synchronized php-1.21wmf9/extensions/Scribunto | [production] | 
            
  | 22:37 | <tstarling> | synchronized php-1.21wmf10/extensions/Scribunto | [production] | 
            
  | 22:20 | <reedy> | synchronized wmf-config/CommonSettings.php | [production] | 
            
  | 20:38 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: wikidatawiki to 1.21wmf10 | [production] | 
            
  | 20:37 | <reedy> | synchronized php-1.21wmf10/cache/l10n/  'Scap is a liar' | [production] | 
            
  | 20:31 | <reedy> | Finished syncing Wikimedia installation... : Rebuild l10n cache for wikidata related extensions | [production] | 
            
  | 20:15 | <reedy> | Started syncing Wikimedia installation... : Rebuild l10n cache for wikidata related extensions | [production] | 
            
  | 19:59 | <reedy> | synchronized php-1.21wmf10/extensions/Wikibase | [production] | 
            
  | 19:59 | <reedy> | synchronized php-1.21wmf10/extensions/Diff | [production] | 
            
  | 19:58 | <reedy> | synchronized php-1.21wmf10/extensions/DataValues | [production] | 
            
  | 19:58 | <reedy> | synchronized php-1.21wmf10/extensions/WikimediaMessages | [production] | 
            
  | 19:44 | <reedy> | synchronized php-1.21wmf10/cache/l10n/ | [production] | 
            
  | 19:19 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki and mediawikiwiki to 1.21wmf10 | [production] | 
            
  | 19:03 | <reedy> | Finished syncing Wikimedia installation... : Build 1.21wmf10 message cache, and push it this time | [production] | 
            
  | 18:45 | <reedy> | Started syncing Wikimedia installation... : Build 1.21wmf10 message cache, and push it this time | [production] |