| 
      
        2011-11-14
      
      §
     | 
  
    
  | 23:01 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  | 
  [production] | 
            
  | 21:29 | 
  <^demon> | 
  killed my svn dump from earlier, ate up all the free space on formey | 
  [production] | 
            
  | 21:03 | 
  <reedy> | 
  synchronized wmf-config/CommonSettings.php  'Enable REL1_18 in ED' | 
  [production] | 
            
  | 21:00 | 
  <Ryan_Lane> | 
  upgraded raskin | 
  [production] | 
            
  | 20:21 | 
  <^demon> | 
  doing full svn dump in screen on formey, probably will take awhile. | 
  [production] | 
            
  | 18:32 | 
  <preilly> | 
  emergency mobile frontend fix for broken lists  | 
  [production] | 
            
  | 18:32 | 
  <preilly> | 
  synchronized php-1.18/extensions/MobileFrontend/views/layout/application.html.php  'weekly update to mobile frontend' | 
  [production] | 
            
  | 18:27 | 
  <nikerabbit> | 
  synchronized php-1.18/tests/phpunit/languages/LanguageMlTest.php  '[[rev:102998|r102998]] - i18n deploy' | 
  [production] | 
            
  | 18:27 | 
  <preilly> | 
  synchronized php-1.18/extensions/MobileFrontend/stylesheets/common.css  'weekly update to mobile frontend' | 
  [production] | 
            
  | 18:26 | 
  <nikerabbit> | 
  synchronized php-1.18/extensions/CategoryTree/  '[[rev:102998|r102998]] - i18n deploy' | 
  [production] | 
            
  | 18:25 | 
  <nikerabbit> | 
  synchronized php-1.18/extensions/Babel/  '[[rev:102998|r102998]] - i18n deploy' | 
  [production] | 
            
  | 18:25 | 
  <nikerabbit> | 
  synchronized php-1.18/languages/  '[[rev:102998|r102998]] - i18n deploy' | 
  [production] | 
            
  | 17:18 | 
  <reedy> | 
  synchronized php-1.18/extensions/Contest/specials/SpecialContest.php  '[[rev:102992|r102992]]' | 
  [production] | 
            
  | 17:14 | 
  <reedy> | 
  synchronized php-1.18/extensions/Contest/Contest.i18n.php  '[[rev:102992|r102992]]' | 
  [production] | 
            
  | 16:41 | 
  <mark> | 
  Shutting down sq69 and sq70 for relocation | 
  [production] | 
            
  | 14:56 | 
  <mark> | 
  Migrated sq70 to varnish3; running apt-get dist-upgrade && reboot | 
  [production] | 
            
  | 14:53 | 
  <mark> | 
  Migrated sq69 to varnish3; running apt-get dist-upgrade && reboot | 
  [production] | 
            
  | 14:39 | 
  <mark> | 
  Migrated sq68 to varnish3; running apt-get dist-upgrade && reboot | 
  [production] | 
            
  | 14:29 | 
  <mark> | 
  Migrated sq67 to varnish3; running apt-get dist-upgrade && reboot | 
  [production] | 
            
  | 14:08 | 
  <mark> | 
  Temporarily disabled pmtpa as esams bits backend - preparation for pmtpa varnish upgrade | 
  [production] | 
            
  | 14:04 | 
  <catrope> | 
  synchronized php-1.18/extensions/WikiEditor/modules/jquery.wikiEditor.toolbar.config.js  '[[rev:102982|r102982]]' | 
  [production] | 
            
  | 14:04 | 
  <RoanKattouw> | 
  Strike that, it is responding to SSH because neither the connection nor the setup time out. It's just terribly, terribly slow | 
  [production] | 
            
  | 14:02 | 
  <RoanKattouw> | 
  searchidx2 is dead, not responding to SSH. Ganglia shows elevated load | 
  [production] | 
            
  | 13:57 | 
  <mark> | 
  Set all esams text amssq (SSD) squids to CARP weight 30 | 
  [production] | 
            
  | 13:46 | 
  <mark> | 
  Depooled knsq1 and knsq2 from esams bits duty, preparing for decommissioning | 
  [production] | 
            
  | 13:26 | 
  <mark> | 
  Ran dist-upgrade and reboot on cp3001 and cp3002 | 
  [production] | 
            
  | 13:26 | 
  <mark> | 
  Setup interface aggregates on cp3001 and cp3002 | 
  [production] | 
            
  | 12:04 | 
  <mark> | 
  Deploying bits.eqiad | 
  [production] | 
            
  | 11:58 | 
  <mark> | 
  bits-lb.esams record addition wasn't picked up by pdns on reload during the previous update, quickly moving bits-lb back to bits.esams | 
  [production] | 
            
  | 11:53 | 
  <mark> | 
  Added bits-lb.esams.wikimedia.org DNS record, and updated geodns to use that instead of bits.esams | 
  [production] | 
            
  | 11:52 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 31459 - Gender in user namespaces for Portuguese Wikipedia' | 
  [production] | 
            
  | 11:39 | 
  <mark> | 
  synchronized wmf-config/CommonSettings.php  'remove old Squids' | 
  [production] | 
            
  | 11:30 | 
  <mark> | 
  Renamed ganglia clusters/dirs on streber | 
  [production] | 
            
  | 11:22 | 
  <mark> | 
  Renamed ganglia RRD dirs on spence | 
  [production] | 
            
  | 10:35 | 
  <catrope> | 
  synchronized php-1.18/extensions/CentralNotice/special/SpecialBannerController.php  '[[rev:102966|r102966]]' | 
  [production] | 
            
  | 10:34 | 
  <catrope> | 
  synchronized php-1.18/extensions/Vector/modules/ext.vector.collapsibleTabs.js  '[[rev:102966|r102966]]' | 
  [production] | 
            
  | 09:58 | 
  <mark> | 
  Fixing ganglia's confusedness about $site | 
  [production] | 
            
  | 08:39 | 
  <catrope> | 
  synchronized php-1.18/extensions/Collection/Collection.body.php  '[[rev:102950|r102950]]' | 
  [production] | 
            
  | 08:26 | 
  <RoanKattouw> | 
  I meant [[rev:102944|r102944]] | 
  [production] | 
            
  | 07:52 | 
  <catrope> | 
  synchronized php-1.18/extensions/Gadgets/Gadgets_body.php  '[[rev:102994|r102994]]' | 
  [production] | 
            
  | 05:09 | 
  <awjr> | 
  synchroinzing payments cluster to [[rev:102939|r102939]] | 
  [production] | 
            
  | 03:31 | 
  <awjr> | 
  synchronizing payments cluster to [[rev:102935|r102935]] | 
  [production] | 
            
  | 02:24 | 
  <LocalisationUpdate> | 
  completed (1.18) at Mon Nov 14 02:27:03 UTC 2011 | 
  [production] | 
            
  | 01:57 | 
  <LocalisationUpdate> | 
  failed | 
  [production] |