| 2010-09-01
      
      § | 
    
  | 18:56 | <RoanKattouw> | Switchover is all done, everyone 's free to touch stuff again as far as I'm concerned | [production] | 
            
  | 18:55 | <catrope> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Vector switchover, thumb size change. enable global opt-out' | [production] | 
            
  | 18:54 | <catrope> | synchronized php-1.5/wmf-config/CommonSettings.php  'Vector switchover, thumb size change. enable global opt-out' | [production] | 
            
  | 18:54 | <RoanKattouw> | Deploying config for Vector switchover now | [production] | 
            
  | 18:49 | <RobH> | all new api servers mgmt accessible, still need OS installation. | [production] | 
            
  | 18:48 | <RobH> | pushing dns change, there was a typo in the zonefile for srv290 mgmt | [production] | 
            
  | 18:37 | <RoanKattouw> | Put config update for Vector switch in wmf-config/ and synced do test. DO NOT SYNC THIS until I confirm it's working | [production] | 
            
  | 18:24 | <catrope> | synchronized php-1.5/includes/UserRightsProxy.php  'r72132' | [production] | 
            
  | 18:22 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/PrefSwitch/SpecialPrefSwitch.php  'r72127' | [production] | 
            
  | 18:21 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/PrefSwitch/PrefSwitch.i18n.php  'r72127' | [production] | 
            
  | 18:21 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/PrefSwitch/PrefSwitch.php  'r72127' | [production] | 
            
  | 18:20 | <catrope> | synchronized php-1.5/skins/Vector.php  'r72127' | [production] | 
            
  | 18:20 | <RoanKattouw> | Deploying r72127 and r72132 with individual syncs | [production] | 
            
  | 18:03 | <RoanKattouw> | Setting default skin to Vector on testwiki | [production] | 
            
  | 17:53 | <RoanKattouw> | Testing r72127 on testwiki | [production] | 
            
  | 17:46 | <RobH> | srv280-srv289 back online.  all power balancing in b5-sdtpa complete. | [production] | 
            
  | 17:22 | <RobH> | srv270-srv279 back online, srv280-srv289 is next (and the last batch) | [production] | 
            
  | 17:06 | <RobH> | also taking down srv279 in last batch | [production] | 
            
  | 17:03 | <RobH> | srv264-srv269 back online, now bringing down srv270-srv278 | [production] | 
            
  | 16:50 | <RobH> | morebots needs better parsing to read my mind. | [production] | 
            
  | 16:49 | <RobH> | srv258-srv263 back online.  Now bringing down srv264-srv269 | [production] | 
            
  | 16:31 | <RobH> | srv258-srv263 are being shutdown to have their power rebalanced within the rack.  They will be back online shortly. | [production] | 
            
  
    | 2010-08-31
      
      § | 
    
  | 23:40 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/SpecialNoticeTemplate.php  'Picking up all changes in r72057' | [production] | 
            
  | 23:40 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/SpecialCentralNotice.php  'Picking up all changes in r72057' | [production] | 
            
  | 23:39 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/CentralNotice.i18n.php  'Picking up all changes in r72057' | [production] | 
            
  | 23:39 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/CentralNotice.db.php  'Picking up all changes in r72057' | [production] | 
            
  | 23:04 | <Ryan_Lane> | added wikidev group to mchenry and sanger | [production] | 
            
  | 22:02 | <mark> | Puppetised jjones account with new ssh key | [production] | 
            
  | 22:01 | <Ryan_Lane> | modified ssh configuration on cluster to disable password login, except while inside of the cluster. | [production] | 
            
  | 20:14 | <catrope> | synchronizing Wikimedia installation... Revision: 72043 | [production] | 
            
  | 20:14 | <RoanKattouw> | Running scap to deploy Siebrand's recent commits to 1.16wmf4 (messages files and Names.php) | [production] | 
            
  | 19:37 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24902 -  tawikinews' | [production] | 
            
  | 19:13 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24957 - enable editinterface user group on hindi wiki and grant bureacrats to add/remove this group' | [production] | 
            
  | 19:11 | <Ryan_Lane> | added review.tesla.usability.wikimedia.org to forward and reverse DNS | [production] | 
            
  | 18:43 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24882 -  Make NS aliases in Korean Wikinews.' | [production] | 
            
  | 17:35 | <tstarling> | synchronized php-1.5/wmf-config/CommonSettings.php  're-enabling MobileRedirect.js' | [production] | 
            
  | 17:34 | <tstarling> | synchronized php-1.5/extensions/WikimediaMobile/MobileRedirect.js  're-enabling' | [production] | 
            
  | 17:15 | <tstarling> | synchronized php-1.5/wmf-config/CommonSettings.php  'bumping mobile JS version' | [production] | 
            
  | 17:12 | <Tim> | disabling mobile redirect since it's been down for a long time now | [production] | 
            
  | 17:11 | <tstarling> | synchronized php-1.5/extensions/WikimediaMobile/MobileRedirect.js | [production] | 
            
  | 15:21 | <Tim> | fixed mobile1 and mobile2 temporarily with killall -9 -u deploy && /etc/init.d/apache2 start | [production] | 
            
  | 15:01 | <hcatlin> | rolled back the lastest m. code release because of catostrophic cluster failure. hoping for recovery. | [production] | 
            
  | 14:52 | <Tim> | mobile mostly down, restored hcatlin's sudo access on mobile1 | [production] | 
            
  | 09:20 | <root> | synchronizing Wikimedia installation... Revision: 71885 | [production] | 
            
  | 09:19 | <tstarling> | synchronizing Wikimedia installation... Revision: 71885 | [production] | 
            
  | 09:11 | <Tim> | added --ignore-externals to /h/w/b/l10nupdate to avoid SSL certificate warning when it attempts to update geshi | [production] | 
            
  | 09:07 | <Tim> | running LocalisationUpdate/update.php | [production] | 
            
  | 04:56 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24956 - enable flood flag user on hindi wiki and grant bureacrats to add or remove this group' | [production] | 
            
  | 04:52 | <jeluf> | synchronized php-1.5/wmf-config/flaggedrevs.php  'Bug 24842 - Enable flagged revisions in category namespace on Hungarian Wikipedia' | [production] | 
            
  | 04:50 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24957 - enable editinterface user group on hindi wiki and grant bureacrats to add/remove this group' | [production] |