| 
      
        2010-06-30
      
      §
     | 
  
    
  | 17:19 | 
  <RoanKattouw> | 
  Messed up by not doing stuff in the right order; switchover complete now | 
  [production] | 
            
  | 17:18 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'Whitespace change' | 
  [production] | 
            
  | 17:12 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'Whitespace change' | 
  [production] | 
            
  | 17:11 | 
  <RoanKattouw> | 
  Syncing dblist to make switchover work | 
  [production] | 
            
  | 17:09 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'Switch all wikis in switchover-jun30.dblist to Vector, UsabilityInitiative, new thumb size and new logo' | 
  [production] | 
            
  | 17:09 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'Switch all wikis in switchover-jun30.dblist to Vector, UsabilityInitiative, new thumb size and new logo' | 
  [production] | 
            
  | 15:57 | 
  <domas> | 
  domas' oprofile killed db40's mysqld :) | 
  [production] | 
            
  | 15:56 | 
  <Rob> | 
  removing out of band connection from scs-ext and moving to mr1 | 
  [production] | 
            
  | 04:54 | 
  <jeluf> | 
  synchronizing Wikimedia installation... Revision: 68749 | 
  [production] | 
            
  
    | 
      
        2010-06-29
      
      §
     | 
  
    
  | 22:22 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'Bump style version appendix' | 
  [production] | 
            
  | 22:20 | 
  <catrope> | 
  synchronized php-1.5/skins/vector/main-rtl.css  'r68749' | 
  [production] | 
            
  | 22:13 | 
  <catrope> | 
  synchronized php-1.5/skins/vector/main-ltr.css  'r68749' | 
  [production] | 
            
  | 21:45 | 
  <catrope> | 
  synchronized php-1.5/extensions/UsabilityInitiative/UsabilityInitiative.hooks.php  'Bump UI CSS style version' | 
  [production] | 
            
  | 21:43 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'Bump style version appendix' | 
  [production] | 
            
  | 21:41 | 
  <catrope> | 
  synchronizing Wikimedia installation... Revision: 68745 | 
  [production] | 
            
  | 21:41 | 
  <RoanKattouw> | 
  Running scap to deploy UsabilityInitiative and Vector fixes, for real this time | 
  [production] | 
            
  | 21:27 | 
  <RoanKattouw> | 
  Strike that, putting the code on test first | 
  [production] | 
            
  | 21:26 | 
  <RoanKattouw> | 
  Running scap to deploy UsabilityInitiative and Vector fixes | 
  [production] | 
            
  | 20:46 | 
  <Fred> | 
  removed A record for volunteer.w.o since it was pointed out that it linked to a winamp playlist. | 
  [production] | 
            
  | 19:14 | 
  <Fred> | 
  removed Naoko from lists to prevent bounces :( | 
  [production] | 
            
  | 18:46 | 
  <Fred> | 
  removed cron entry from isidore for bz-reporter -> installed mail on Kaulen which will now handle the weekly reports. | 
  [production] | 
            
  | 17:54 | 
  <Rob> | 
  connected mr1-pmtpa port 0/1 to csw5 port 7/1 | 
  [production] | 
            
  | 17:48 | 
  <Rob> | 
  and msw1-pmtpa:prt24 to msw-b2 (port unimportant) | 
  [production] | 
            
  | 17:47 | 
  <Rob> | 
  new mgmt connections: mgmt-gateway: serial to scs-ca-pmtpa:port1; mgmt-gateway:0/0 to msw1-pmta:port23, msw1-pmtpa:mgmt to scs-c1-pmtpa port2 | 
  [production] | 
            
  | 16:23 | 
  <Rob> | 
  dataset1 bad disk swapped and array shows rebuilding | 
  [production] | 
            
  | 15:37 | 
  <Rob> | 
  srv89 resurrected and playing nicely with the other server kids | 
  [production] | 
            
  | 15:15 | 
  <Rob> | 
  srv146 updated (puppet,sync-docroot,sync-apache), working, and back in lvs cluster | 
  [production] | 
            
  | 15:09 | 
  <Rob> | 
  fixing srv146, removed from lvs until fixed | 
  [production] | 
            
  | 08:19 | 
  <midom> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  | 
  [production] | 
            
  | 08:14 | 
  <midom> | 
  synchronized php-1.5/includes/db/DatabaseMysql.php  '68713 live' | 
  [production] | 
            
  | 08:13 | 
  <midom> | 
  synchronized php-1.5/includes/DefaultSettings.php  '68713 live' | 
  [production] | 
            
  | 08:08 | 
  <midom> | 
  synchronized php-1.5/includes/GlobalFunctions.php  'removing msg profiling' | 
  [production] | 
            
  | 06:00 | 
  <apergos> | 
  restarted morebots on wikitech | 
  [production] | 
            
  
    | 
      
        2010-06-25
      
      §
     | 
  
    
  | 18:13 | 
  <Rob> | 
  srv104 disk failure, shutting it down (via ticket to the noc) | 
  [production] | 
            
  | 15:01 | 
  <Rob> | 
  srv284 has E1410 System Fatal Error, will place ticket later today | 
  [production] | 
            
  | 14:59 | 
  <Rob> | 
  db28 is borked, needs more replacement memory, reopened ticket, will remain offline for now. | 
  [production] | 
            
  | 14:59 | 
  <Rob> | 
  fixed srv281 issues, needs reinstallation later today | 
  [production] | 
            
  | 14:58 | 
  <Rob> | 
  tested db34, LOM is doa, calling in later today | 
  [production] | 
            
  | 14:03 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'Switch private wikis to Vector per Erik M' | 
  [production] | 
            
  | 13:53 | 
  <Rob> | 
  ignore db28 flapping, working on it. | 
  [production] | 
            
  | 09:43 | 
  <domas> | 
  updated password formats for wikiuser on s1 and s2 | 
  [production] | 
            
  | 08:20 | 
  <midom> | 
  synchronized php-1.5/wmf-config/mc.php  'swapping out srv104, putting in srv242' | 
  [production] | 
            
  | 07:38 | 
  <JeLuF> | 
  enabled PHP modules on fenari so that noc.wikimedia.org deliveres syntax highlighted config files. | 
  [production] | 
            
  | 00:10 | 
  <andrew> | 
  synchronized php-1.5/extensions/AbuseFilter/SpecialAbuseLog.php  | 
  [production] |