| 
      
        2011-03-15
      
      §
     | 
  
    
  | 21:18 | 
  <hcatlin> | 
  restarted mobile app on mobile1 due to unexpected load increase beginning at 20:43 | 
  [production] | 
            
  | 20:36 | 
  <RoanKattouw> | 
  Clearing message blobs | 
  [production] | 
            
  | 20:36 | 
  <catrope> | 
  synchronized php-1.17/extensions/ArticleFeedback/modules/ext.articleFeedback/ext.articleFeedback.js  'r84052' | 
  [production] | 
            
  | 20:35 | 
  <catrope> | 
  synchronized php-1.17/extensions/ArticleFeedback/ArticleFeedback.i18n.php  'r84052' | 
  [production] | 
            
  | 19:28 | 
  <Ryan_Lane> | 
  added svg to allowed upload types for wikitech | 
  [production] | 
            
  | 17:00 | 
  <mark> | 
  Replaced root's authorized_keys file | 
  [production] | 
            
  | 06:47 | 
  <tstarling> | 
  synchronized php-1.17/wmf-config/CommonSettings.php  'pool counter deployment on testwiki only' | 
  [production] | 
            
  | 06:46 | 
  <tstarling> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'pool counter deployment on testwiki only' | 
  [production] | 
            
  | 06:42 | 
  <root> | 
  synchronizing Wikimedia installation... Revision: 83954:  | 
  [production] | 
            
  | 06:42 | 
  <tstarling> | 
  synchronizing Wikimedia installation... Revision: 83954:  | 
  [production] | 
            
  | 06:41 | 
  <Tim> | 
  running scap to get the PoolCounter extension | 
  [production] | 
            
  | 06:28 | 
  <Tim> | 
  installed poolcounter on tarin | 
  [production] | 
            
  | 06:08 | 
  <Tim> | 
  tarin upgraded to lucid | 
  [production] | 
            
  | 05:15 | 
  <Tim> | 
  tarin just has an ancient test wordpress install on it which hasn't been touched since April 2010. Removing the website and mysql instance, will attempt to upgrade the distro. | 
  [production] | 
            
  | 03:20 | 
  <tstarling> | 
  synchronized php-1.17/includes/HTMLCacheUpdate.php  'disable squid cache invalidation from HTMLCacheUpdate, to hopefully eliminate apache CPU spikes' | 
  [production] | 
            
  | 03:12 | 
  <Tim> | 
  started job runners | 
  [production] | 
            
  | 03:09 | 
  <Tim> | 
  apache CPU spike, stopped job runners | 
  [production] | 
            
  | 03:05 | 
  <tstarling> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'removed bug 27891 logging' | 
  [production] | 
            
  | 03:04 | 
  <tstarling> | 
  synchronized php-1.17/includes/parser/ParserCache.php  'removed bug 27891 logging' | 
  [production] | 
            
  
    | 
      
        2011-03-14
      
      §
     | 
  
    
  | 19:57 | 
  <catrope> | 
  synchronized php-1.17/extensions/ArticleFeedback/modules/jquery.articleFeedback/jquery.articleFeedback.css  'r83954' | 
  [production] | 
            
  | 19:33 | 
  <catrope> | 
  synchronized php-1.17/extensions/ClickTracking/modules/ext.clickTracking.js  'r83952' | 
  [production] | 
            
  | 19:33 | 
  <catrope> | 
  synchronized php-1.17/extensions/ClickTracking/modules/jquery.clickTracking.js  'r83952' | 
  [production] | 
            
  | 19:28 | 
  <catrope> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'Enable ArticleFeedback on enwiki' | 
  [production] | 
            
  | 19:23 | 
  <catrope> | 
  ran sync-common-all  | 
  [production] | 
            
  | 19:22 | 
  <RoanKattouw> | 
  Got sudo weirdness on scap again, running sync-common-all | 
  [production] | 
            
  | 19:21 | 
  <catrope> | 
  synchronizing Wikimedia installation... Revision: 83949:  | 
  [production] | 
            
  | 19:20 | 
  <catrope> | 
  synchronized php-1.17/extensions/Collection/Collection.suggest.php  'Syntax error' | 
  [production] | 
            
  | 19:19 | 
  <RoanKattouw> | 
  Running scap to deploy ArticleFeedback (only enabled on test for now, will enable on enwiki after this) | 
  [production] | 
            
  | 18:39 | 
  <Ryan_Lane> | 
  installing sudo schema on nova-controller.tesla and restarting opendj | 
  [production] | 
            
  | 18:14 | 
  <catrope> | 
  synchronized php-1.17/extensions/CodeReview/CodeReview.php  'Style version bump' | 
  [production] | 
            
  | 18:14 | 
  <catrope> | 
  ran sync-common-all  | 
  [production] | 
            
  | 18:12 | 
  <RoanKattouw> | 
  Running sync-common-all to deploy r83934 | 
  [production] | 
            
  | 09:55 | 
  <Tim> | 
  increased memcached read timeout from 100ms to 500ms. This reduced the read timeout rate from 51/s to 3.4/s. | 
  [production] | 
            
  | 09:50 | 
  <tstarling> | 
  synchronized php-1.17/wmf-config/mc.php  'increasing memcached timeout from 100ms to 500ms while monitoring timeout rate' | 
  [production] | 
            
  | 09:44 | 
  <tstarling> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'log memcached timeout errors' | 
  [production] | 
            
  | 09:40 | 
  <tstarling> | 
  synchronized php-1.17/includes/memcached-client.php  'log timeout errors' | 
  [production] | 
            
  | 06:26 | 
  <tstarling> | 
  synchronized php-1.17/includes/parser/ParserCache.php  'more specific logging hack' | 
  [production] | 
            
  | 04:32 | 
  <tstarling> | 
  synchronized php-1.17/includes/Article.php  'r83869' | 
  [production] | 
            
  | 03:42 | 
  <tstarling> | 
  synchronized php-1.17/includes/parser/ParserCache.php  'logging tweak' | 
  [production] | 
            
  | 03:36 | 
  <tstarling> | 
  synchronized php-1.17/includes/parser/ParserCache.php  'logging tweak' | 
  [production] | 
            
  | 03:32 | 
  <tstarling> | 
  synchronized php-1.17/includes/parser/ParserCache.php  'more logging' | 
  [production] | 
            
  | 03:23 | 
  <tstarling> | 
  synchronized php-1.17/includes/parser/ParserCache.php  | 
  [production] | 
            
  | 03:07 | 
  <tstarling> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'logging hack for bug 27891' | 
  [production] |