| 2011-04-28
      
      § | 
    
  | 23:46 | <Tim> | doing hard reset of alsted, it's not sending ssh version strings and not allowing logins by serial | [production] | 
            
  | 22:17 | <pdhanda> | synchronized php-1.17/wmf-config/InitialiseSettings.php  'Disabled option to allow blocked users to edit their talk pages on itwiki. See Bug 9073' | [production] | 
            
  | 21:02 | <Ryan_Lane> | created Ambassador-announce-l list | [production] | 
            
  | 20:52 | <richcole1> | replacing memory in srv281 | [production] | 
            
  | 20:49 | <RobH> | srv281 shutdown for memory replacement | [production] | 
            
  | 20:41 | <pdhanda> | synchronized php-1.17/wmf-config/InitialiseSettings.php  'Enable by default, option to enable blocked users to edit their own talk page' | [production] | 
            
  | 20:34 | <richcole1> | replacing bad drive in srv284 | [production] | 
            
  | 20:08 | <Ryan_Lane> | powercycling srv281 | [production] | 
            
  | 20:03 | <RobH> | ms4 is powered down, was already offline and unresponsive to console. | [production] | 
            
  | 18:19 | <demon> | synchronizing Wikimedia installation... Revision: 87091: r87090, r87091 | [production] | 
            
  | 18:02 | <mark> | Restarted apache on srv235, probably corrupted apc cache | [production] | 
            
  | 17:57 | <awjrichards> | synchronized php-1.17/wmf-config/InitialiseSettings.php  'Re-enabling EmailCapture on enwiki' | [production] | 
            
  | 17:54 | <awjrichards> | synchronizing Wikimedia installation... Revision: 87084: | [production] | 
            
  | 17:47 | <awjrichards> | synchronized php-1.17/wmf-config/CommonSettings.php  'Preparing conf for ArticleFeedback with way less click tracking to prevent cluster breakage' | [production] | 
            
  | 17:40 | <RobH> | etherpad.proxy to 000-etherpad.proxy on hooper apache2 enabled, should fix | [production] | 
            
  | 17:34 | <RobH> | hooper vhost for testblog back online | [production] | 
            
  | 17:17 | <RobH> | rolled back testblog vhost from hooper | [production] | 
            
  | 17:01 | <RobH> | testblog enabled for initial setup on hooper without disrupting etherpad service (on either of its url redirects) | [production] | 
            
  | 16:56 | <RobH> | updating etherpad.proxy apache config to include eiximenis serveralias and enabling the testblog vhost | [production] | 
            
  | 16:48 | <Reedy> | Manually pinged CR update script, had got stuck - 2 revisions. Let me know if it does it again | [production] | 
            
  | 16:21 | <catrope> | synchronizing Wikimedia installation... Revision: 87085: | [production] | 
            
  | 16:21 | <RoanKattouw> | Rolling back ArticleFeedback deployment | [production] | 
            
  | 16:11 | <RoanKattouw> | API cluster was briefly dead just now after enabling EmailCapture, disabling it seems to be bringing it back | [production] | 
            
  | 16:09 | <catrope> | synchronized php-1.17/wmf-config/InitialiseSettings.php  'Disable EmailCapture, maybe this will bring our API Apaches back' | [production] | 
            
  | 16:04 | <awjrichards> | synchronized php-1.17/wmf-config/InitialiseSettings.php  'Enabling EmailCapture on enwiki' | [production] | 
            
  | 16:02 | <RoanKattouw> | (that was for the ArticleFeedback deployment) | [production] | 
            
  | 15:58 | <awjrichards> | synchronizing Wikimedia installation... Revision: 87083: | [production] | 
            
  | 14:08 | <nelson> | Upped the max_connections for rsyncd from 2 to 5 across the test cluster. | [production] | 
            
  | 13:56 | <nelson> | Upped the concurrency of swift-object-replicator from default of 1 to 5 | [production] | 
            
  | 13:38 | <mark> | Restored I/O scheduler deadline for device /dev/sda on alsted | [production] | 
            
  | 13:34 | <mark> | Temporarily set I/O scheduler cfq for device /dev/sda on alsted | [production] | 
            
  | 12:29 | <mark> | powercycled harmon | [production] | 
            
  | 01:00 | <awjr> | upgraded dev.civicrm.wikimedia.org to CiviCRM 3.4.0 | [production] |