| 
      
        2013-03-05
      
      §
     | 
  
    
  | 23:00 | 
  <logmsgbot_> | 
  maxsem Finished syncing Wikimedia installation... : Weekly mobile deployment | 
  [production] | 
            
  | 22:40 | 
  <logmsgbot_> | 
  maxsem Started syncing Wikimedia installation... : Weekly mobile deployment | 
  [production] | 
            
  | 22:36 | 
  <logmsgbot_> | 
  py synchronized wmf-config/db-eqiad.php  'repooling db1009 at 25% weight' | 
  [production] | 
            
  | 22:18 | 
  <logmsgbot_> | 
  reedy synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 22:16 | 
  <logmsgbot_> | 
  reedy synchronized wmf-config/ | 
  [production] | 
            
  | 22:14 | 
  <LeslieCarr> | 
  restarting pybal on lvs3 | 
  [production] | 
            
  | 22:14 | 
  <LeslieCarr> | 
  restarting pybal on lvs4 | 
  [production] | 
            
  | 22:13 | 
  <LeslieCarr> | 
  restarting pybal | 
  [production] | 
            
  | 22:07 | 
  <logmsgbot_> | 
  reedy rebuilt wikiversions.cdb and synchronized wikiversions files: | 
  [production] | 
            
  | 22:05 | 
  <logmsgbot_> | 
  reedy synchronized wmf-config/ | 
  [production] | 
            
  | 21:57 | 
  <LeslieCarr> | 
  restarting pybal on lvs1003 | 
  [production] | 
            
  | 21:55 | 
  <LeslieCarr> | 
  restarting pybal on lvs1006 | 
  [production] | 
            
  | 21:44 | 
  <Coren> | 
  tools rejiggered the webproxy config to be smarter about paths not leading to specific tools | 
  [production] | 
            
  | 21:43 | 
  <LeslieCarr> | 
  adding parsoidcache service to lvs1006 | 
  [production] | 
            
  | 21:42 | 
  <mutante> | 
  gracefulling eqiad Apaches - add ombudsmen.wm (bug 45744) | 
  [production] | 
            
  | 21:41 | 
  <logmsgbot_> | 
  dzahn gracefulled all apaches | 
  [production] | 
            
  | 21:35 | 
  <mutante> | 
  restarting pdns on ns2 | 
  [production] | 
            
  | 21:34 | 
  <mutante> | 
  DNS update - add ombudsmen (bug 45744) | 
  [production] | 
            
  | 20:49 | 
  <logmsgbot_> | 
  reedy synchronized docroot | 
  [production] | 
            
  | 20:35 | 
  <mutante> | 
  installing package upgrades on hooper | 
  [production] | 
            
  | 20:26 | 
  <logmsgbot_> | 
  reedy synchronized wmf-config/ | 
  [production] | 
            
  | 20:22 | 
  <mutante> | 
  installing package upgrades on hume | 
  [production] | 
            
  | 20:00 | 
  <mutante> | 
  wikipedia.ee now redirects to et.wikipedia, wikimedia.ee now redirects to et.wikimedia | 
  [production] | 
            
  | 19:56 | 
  <mutante> | 
  gracefulling eqiad Apaches, push wiki(m|p)edia.ee redirects | 
  [production] | 
            
  | 19:56 | 
  <logmsgbot_> | 
  dzahn gracefulled all apaches | 
  [production] | 
            
  | 19:18 | 
  <K4-713> | 
  updated payments cluster to 9b063ee2e399a3b | 
  [production] | 
            
  | 18:42 | 
  <logmsgbot_> | 
  mlitn synchronized wmf-config/InitialiseSettings.php  'Disable AFTv4 (entirely) and AFTv5 (leaving only opt-in) on enwiki' | 
  [production] | 
            
  | 18:04 | 
  <cmjohnson1> | 
  restarting dhcpd3 service brewster | 
  [production] | 
            
  | 16:55 | 
  <cmjohnson1> | 
  mw1085 powering off to troubleshoot DIMM error | 
  [production] | 
            
  | 05:35 | 
  <ori-l> | 
  EventLogging db on db1047: ALTER DATABASE log CHARACTER SET utf8 COLLATE utf8_unicode_ci; (was: latin1) | 
  [production] | 
            
  | 02:49 | 
  <logmsgbot_> | 
  LocalisationUpdate completed (1.21wmf11) at Tue Mar  5 02:48:58 UTC 2013 | 
  [production] | 
            
  | 02:28 | 
  <logmsgbot_> | 
  LocalisationUpdate completed (1.21wmf10) at Tue Mar  5 02:28:16 UTC 2013 | 
  [production] | 
            
  | 02:03 | 
  <K4-713> | 
  updated payments cluster to 5361c795f4bf31980f | 
  [production] | 
            
  | 00:46 | 
  <mutante> | 
  fixing exim/mail permissions on neon to make Icinga paging work. missing +x on /var/log/exim4 dir | 
  [production] | 
            
  
    | 
      
        2013-03-04
      
      §
     | 
  
    
  | 23:12 | 
  <mutante> | 
  starting dhcp3 on brewster | 
  [production] | 
            
  | 23:10 | 
  <LeslieCarr> | 
  p.s. bonding on foundry sucks | 
  [production] | 
            
  | 23:10 | 
  <LeslieCarr> | 
  bringing down virt2 network to bond it | 
  [production] | 
            
  | 22:18 | 
  <mutante> | 
  gracefulling eqiad Apaches to push indiawikipedia redirect | 
  [production] | 
            
  | 22:17 | 
  <logmsgbot_> | 
  dzahn gracefulled all apaches | 
  [production] | 
            
  | 22:03 | 
  <logmsgbot_> | 
  py synchronized wmf-config/db-pmtpa.php  'shuffling around pmtpa enwiki slaves' | 
  [production] | 
            
  | 21:50 | 
  <LeslieCarr> | 
  messing around with the network of virt2 - expect possible pmtpa labs network issues | 
  [production] | 
            
  | 21:50 | 
  <ottomata> | 
  staring puppet back up on oxygen, packet loss is better after disabling a filter | 
  [production] | 
            
  | 21:30 | 
  <cmjohnson1> | 
  authdns update | 
  [production] | 
            
  | 21:17 | 
  <logmsgbot_> | 
  reedy rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki and mediawikiwiki to 1.21wmf11 | 
  [production] | 
            
  | 21:16 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf11 | 
  [production] | 
            
  | 21:15 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf10 | 
  [production] | 
            
  | 21:14 | 
  <RobH> | 
  shutting down professor for a fan swap, should be a short window | 
  [production] | 
            
  | 21:11 | 
  <cmjohnson1> | 
  authdns update | 
  [production] | 
            
  | 21:05 | 
  <ottomata> | 
  stopping puppet on oxygen to see if we can disable filters and reduce packet loss | 
  [production] | 
            
  | 20:55 | 
  <Ryan_Lane> | 
  restarted icinga | 
  [production] |