| 
      
        2009-03-23
      
      §
     | 
  
    
  | 23:32 | 
  <tstarling> | 
  synchronized php-1.5/InitialiseSettings.php  'rate limit exemption for usability testing' | 
  [production] | 
            
  | 22:56 | 
  <domas> | 
  db1 will serve fr/ja soonish  | 
  [production] | 
            
  | 22:56 | 
  <midom> | 
  synchronized php-1.5/db.php  'removing db1' | 
  [production] | 
            
  | 18:58 | 
  <Rob> | 
  upgraded spam plugin on wikimedia blog (shows old stuff in dashboard due to caching, its updated now) | 
  [production] | 
            
  | 18:23 | 
  <aZaFred> | 
  Setup access on LDAP and NIS for fvassard. | 
  [production] | 
            
  | 13:36 | 
  <midom> | 
  synchronized php-1.5/db.php  'rename s2a into s2dewiki, add s2commons with single primary server ixia, enable ixia with commons-only dataset, *poof*' | 
  [production] | 
            
  | 00:34 | 
  <brion-weekend> | 
  customized style a bit on [[techblog.wikimedia.org]] | 
  [production] | 
            
  
    | 
      
        2009-03-22
      
      §
     | 
  
    
  | 23:14 | 
  <Tim> | 
  killed long-running tidy instance on srv108 | 
  [production] | 
            
  | 22:10 | 
  <Tim> | 
  restarted memcached on srv82 | 
  [production] | 
            
  | 12:15 | 
  <domas> | 
  dumping commonswiki from db30 with server still pooled in and 4 dumper threads | 
  [production] | 
            
  | 10:38 | 
  <domas> | 
  hey, I found remote code execution vulnerability, it seems! :) | 
  [production] | 
            
  | 10:37 | 
  <andrew> | 
  synchronized php-1.5/extensions/AbuseFilter/AbuseFilter.parser.php  '--message=Arbitrary execution vulnerability in AbuseFilter, exploitable only by admins' | 
  [production] | 
            
  | 06:00 | 
  <tstarling> | 
  synchronized php-1.5/db.php  'depooled ixia, is full' | 
  [production] | 
            
  | 05:54 | 
  <Tim> | 
  db5 was running out of disk space due to excessive relay logs. Ran RESET SLAVE. | 
  [production] | 
            
  | 03:02 | 
  <andrew> | 
  synchronized php-1.5/extensions/AbuseFilter/Views/AbuseFilterViewExamine.php  '18096 Special:AbuseFilter/examine doesn't list new account creation log entries' | 
  [production] | 
            
  | 02:47 | 
  <andrew> | 
  synchronized php-1.5/extensions/AbuseFilter/AbuseFilter.class.php  'Fixed bug in batch testing interface' | 
  [production] | 
            
  | 02:41 | 
  <andrew> | 
  synchronized php-1.5/extensions/AbuseFilter/AbuseFilter.parser.php  'Optimisation of rmdoubles, causes 20-fold performance improvement on large pages' | 
  [production] | 
            
  | 02:23 | 
  <andrew> | 
  synchronized php-1.5/extensions/AbuseFilter/Views/AbuseFilterViewDiff.php  'Prevent leaking of private filters through diff interface' | 
  [production] | 
            
  
    | 
      
        2009-03-21
      
      §
     | 
  
    
  | 17:30 | 
  <midom> | 
  synchronized php-1.5/db.php  'someone forgot to enable db8.. putting back to pool' | 
  [production] | 
            
  | 17:29 | 
  <midom> | 
  synchronized php-1.5/db.php  'bringing back db26' | 
  [production] | 
            
  | 10:46 | 
  <midom> | 
  synchronized php-1.5/db.php  'taking out db26 for kernel experiments (2.6.28.8 with some different build options)' | 
  [production] | 
            
  | 00:12 | 
  <brion> | 
  upload-by-URL enabled for sysops on testwiki (using [[khaldun]] as internal proxy) | 
  [production] | 
            
  | 00:07 | 
  <brion> | 
  synchronized php-1.5/InitialiseSettings.php  | 
  [production] | 
            
  | 00:05 | 
  <brion> | 
  synchronized php-1.5/extensions/MWSearch/MWSearch_body.php  | 
  [production] | 
            
  | 00:04 | 
  <brion> | 
  synchronized php-1.5/includes/specials/SpecialUpload.php  | 
  [production] | 
            
  | 00:03 | 
  <brion> | 
  live-merging r48648 to allow $wgHTTPProxy to work for uploads and not interfere with search | 
  [production] | 
            
  
    | 
      
        2009-03-20
      
      §
     | 
  
    
  | 23:58 | 
  <Tim> | 
  repooled srv126, part of cluster12, appears to be up and working | 
  [production] | 
            
  | 23:58 | 
  <tstarling> | 
  synchronized php-1.5/db.php  | 
  [production] | 
            
  | 23:54 | 
  <Tim> | 
  depooled srv125 from ES, has been down for 12 days. cluster12 is now down to 1 server | 
  [production] | 
            
  | 23:54 | 
  <tstarling> | 
  synchronized php-1.5/db.php  | 
  [production] | 
            
  | 23:52 | 
  <Tim> | 
  install ganglia-metrics on db24 | 
  [production] | 
            
  | 23:13 | 
  <Tim> | 
  set up logrotate on locke, using the same script that we use for MW debug logs | 
  [production] | 
            
  | 22:56 | 
  <domas> | 
  kickbanned security threat from #wikimedia-tech, was trying to install keylogger and steal our passwords | 
  [production] | 
            
  | 22:54 | 
  <Tim> | 
  removed old squid log stream going to iris. Set up a log stream going from all the squids to locke. | 
  [production] | 
            
  | 22:37 | 
  <Tim> | 
  depooled adler, it's down | 
  [production] | 
            
  | 22:36 | 
  <tstarling> | 
  synchronized php-1.5/db.php  'depooled adler' | 
  [production] | 
            
  | 21:01 | 
  <mark> | 
  removed vlan 5 on csw5-pmtpa that was accidently created/left behind by Tim | 
  [production] | 
            
  | 20:47 | 
  <domas> | 
  installed snaprotate on db26, enabled snapshots with 2x8h schedule, updated [[Database snapshots]] | 
  [production] | 
            
  | 20:44 | 
  <midom> | 
  synchronized php-1.5/includes/api/ApiQueryRevisions.php  'livemerging r48642' | 
  [production] | 
            
  | 20:43 | 
  <midom> | 
  synchronized php-1.5/includes/filerepo/ArchivedFile.php  'livemerging r48644' | 
  [production] | 
            
  | 20:27 | 
  <domas> | 
  someone who wrote ArchivedFile::load, needs some pain and torture applied (query doesn't use index... ;-) | 
  [production] | 
            
  | 20:25 | 
  <midom> | 
  synchronized php-1.5/db.php  'reduced load for db26 from 200 to 100, as it has reduced amount of RAM and increased amount of other work' | 
  [production] | 
            
  | 20:09 | 
  <domas> | 
  added db26 to s1 pool | 
  [production] | 
            
  | 20:08 | 
  <midom> | 
  synchronized php-1.5/db.php  | 
  [production] | 
            
  | 18:47 | 
  <mark> | 
  JeLuF> |log \\o/ | 
  [production] | 
            
  | 18:46 | 
  <domas> | 
  pooled db22 back in, as it caught up on replication! | 
  [production] | 
            
  | 18:46 | 
  <mark> | 
  raised karma of /h/w/b/reset-mysql-slave | 
  [production] | 
            
  | 18:46 | 
  <midom> | 
  synchronized php-1.5/db.php  | 
  [production] |