| 
      
        2009-03-13
      
      §
     | 
  
    
  | 23:11 | 
  <Rob> | 
  migrated survey software from isidore to singer | 
  [production] | 
            
  | 22:21 | 
  <Rob> | 
  restarted apache on singer after enabling all the mod rewrite stuffs | 
  [production] | 
            
  | 21:58 | 
  <Rob> | 
  redeployed ./sync all for squid for whygive migration | 
  [production] | 
            
  | 21:58 | 
  <Rob> | 
  disabled blog and whygive apache virtual hosts on isidore | 
  [production] | 
            
  | 21:58 | 
  <Rob> | 
  migrated old whygive.wikimedia.org from isidore to singer. | 
  [production] | 
            
  | 21:21 | 
  <Rob> | 
  http://techblog.wikimedia.org is online (although quite sad and empty) | 
  [production] | 
            
  | 21:14 | 
  <Rob> | 
  setup tech blog on singer with database residing on db9 | 
  [production] | 
            
  | 21:06 | 
  <Rob> | 
  updated dns for new techblog (not yet live) | 
  [production] | 
            
  | 21:06 | 
  <Rob> | 
  updated squids configuration for blog move | 
  [production] | 
            
  | 21:00 | 
  <Rob> | 
  moved blog.wikimedia.org from old server isidore to new server singer | 
  [production] | 
            
  | 20:18 | 
  <Tim> | 
  srv85 died, possible disk failure, no ssh or memcached, still has HTTP. Removed it from the memcached list, removed it from apache LVS. | 
  [production] | 
            
  | 20:15 | 
  <Rob> | 
  updated spamfree plugin on blog.wikimedia.org | 
  [production] | 
            
  | 20:04 | 
  <Rob> | 
  updated to newest version of wordpress on blog.wikimedia.org and whygive.wikimedia.org | 
  [production] | 
            
  | 17:27 | 
  <Rob> | 
  Testing | 
  [production] | 
            
  | 17:24 | 
  <Rob> | 
  updated flaggedrevs.php per bug 16365 | 
  [production] | 
            
  | 17:23 | 
  <Tim> | 
  removed password auth from nagios | 
  [production] | 
            
  | 15:28 | 
  <river> | 
  ms4 disk c5t5d0 failed | 
  [production] | 
            
  
    | 
      
        2009-03-10
      
      §
     | 
  
    
  | 23:31 | 
  <tomaszf> | 
  renabled central notice for all template generation | 
  [production] | 
            
  | 23:30 | 
  <Andrew> | 
  on testwiki, that is | 
  [production] | 
            
  | 23:30 | 
  <Andrew> | 
  updated AuseFilter to r48294 for debugging | 
  [production] | 
            
  | 22:58 | 
  <Andrew> | 
  updated AbuseFilter to r48288 on test. | 
  [production] | 
            
  | 22:32 | 
  <Tim> | 
  restarted memcached on srv63 | 
  [production] | 
            
  | 22:24 | 
  <brion> | 
  updating Collection extension to r48283; fixes | 
  [production] | 
            
  | 18:45 | 
  <hi> | 
  identi.ca folks! domas and mark just cleared #profiling data! | 
  [production] | 
            
  | 17:27 | 
  <brion> | 
  reenabling limited centralnotice update cronjob on hume, since the live notice ain't working | 
  [production] | 
            
  | 00:52 | 
  <brion> | 
  domas makes it all better. yay domas! | 
  [production] | 
            
  | 00:50 | 
  <brion> | 
  cluster21 master (srv161) in read-only; we're having write failure problems on multiple wikis | 
  [production] | 
            
  | 00:48 | 
  <brion> | 
  putting ES back into service on enwiki; srv160/cluster20 master has been fixed. slaves still running it, but this is safe for read/write since reads will fall back to master | 
  [production] |