| 
      
        2008-12-17
      
      §
     | 
  
    
  | 22:36 | 
  <brion> | 
  applied fix for Android browser on mobile gateway (also did the pl language setup recently) | 
  [production] | 
            
  | 22:06 | 
  <RobH> | 
  more lucene.php changes | 
  [production] | 
            
  | 21:12 | 
  <RobH> | 
  additions to lucene.php per rainmain | 
  [production] | 
            
  | 20:39 | 
  <mark> | 
  Corrected LVS service IPs on search2, search10-12 | 
  [production] | 
            
  | 20:03 | 
  <brion> | 
  hacked mw-serve init script on yongle into shape. will commit it in a bit and update docs | 
  [production] | 
            
  | 19:38 | 
  <brion> | 
  pdf server seems to have eaten all temp space on yongle. clearing... | 
  [production] | 
            
  | 19:26 | 
  <mark> | 
  Set up search2, search8-12 | 
  [production] | 
            
  | 18:57 | 
  <RobH> | 
  pushing dns changes for new misc. servers management resolution | 
  [production] | 
            
  | 18:30 | 
  <RobH> | 
  updated lucene.php with rainman to do things that I really do not get but he knows about. | 
  [production] | 
            
  | 16:28 | 
  <RobH> | 
  new servers auth1, nfs2, streber and williams are racked, IP's allocated, DRAC working.  No DHCP entries or OS installed yet. | 
  [production] | 
            
  | 16:08 | 
  <mark> | 
  restarted lighttpd on zwinger | 
  [production] | 
            
  | 15:59 | 
  <RobH> | 
  added williams to dns records, updated dns | 
  [production] | 
            
  | 15:50 | 
  <TimStarling> | 
  removed some binlogs on ixia | 
  [production] | 
            
  | 01:17 | 
  <brion> | 
  scapping a couple more fixes to r44698 | 
  [production] | 
            
  | 00:36 | 
  <brion-codereview> | 
  srv126 is borked -- read-only filesystem | 
  [production] | 
            
  | 00:23 | 
  <brion-codereview> | 
  scapping to 44696 | 
  [production] | 
            
  | 00:23 | 
  <brion-codereview> | 
  scapping | 
  [production] | 
            
  | 00:15 | 
  <brion-codereview> | 
  svn up'ing on test... | 
  [production] | 
            
  
    | 
      
        2008-12-16
      
      §
     | 
  
    
  | 23:09 | 
  <brion-codereview> | 
  disabling FixedImage extension -- was used for old 2006 and 2007 fundraisers; images no longer exist and are not applicable to current fundraisers | 
  [production] | 
            
  | 20:34 | 
  <RobH> | 
  ariel is dead, will decommission later. | 
  [production] | 
            
  | 20:29 | 
  <RobH> | 
  ariel is fubar, rebooting and investigating. | 
  [production] | 
            
  | 20:25 | 
  <RobH> | 
  restarted services on sq13 | 
  [production] | 
            
  | 20:21 | 
  <RobH> | 
  took down sq13 to clean its cache | 
  [production] | 
            
  | 20:09 | 
  <RobH> | 
  replaced bad /c0/p0 in amane | 
  [production] | 
            
  | 18:47 | 
  <brion> | 
  applying temporary resource limit lift on enwiki for an IP for workshop in SF | 
  [production] | 
            
  | 17:40 | 
  <RobH> | 
  updated dns for misc. servers project. | 
  [production] | 
            
  | 01:08 | 
  <brion> | 
  deploying r44643 update to CodeReview subversion proxy (swapped encoding protocol to avoid bugs in json_decode with some diffs) | 
  [production] | 
            
  | 00:04 | 
  <brion> | 
  running cleanupTitles.php in bg on all wikis... | 
  [production] | 
            
  
    | 
      
        2008-12-15
      
      §
     | 
  
    
  | 23:20 | 
  <brion> | 
  going to test fixes for FiveUpgrade.inc to back cleanupTitles.php, cleanupImages.php etc | 
  [production] | 
            
  | 22:22 | 
  <RobH> | 
  changed settings on metawiki to allow banned users to edit their talk pages per https://bugzilla.wikimedia.org/show_bug.cgi?id=16621 | 
  [production] | 
            
  | 21:25 | 
  <brion> | 
  reenabling handheld skin setting, was turned off during overload emergencies on 11-17 | 
  [production] | 
            
  | 21:13 | 
  <brion> | 
  rsyncd appears to be running on srv56. does anything else need to be done for index updates? | 
  [production] | 
            
  | 20:10 | 
  <brion> | 
  yongle hanging again, restarting apache | 
  [production] | 
            
  | 18:58 | 
  <RobH> | 
  started rsync daemon on srv56 per rainman | 
  [production] | 
            
  | 18:35 | 
  <RobH> | 
  setup new planet per https://bugzilla.wikimedia.org/show_bug.cgi?id=16511. | 
  [production] | 
            
  | 01:39 | 
  <brion-weekend> | 
  applying API deletion log fix from r44541 ([[bugzilla:16626]]) | 
  [production] | 
            
  | 00:09 | 
  <rainman-sr> | 
  rsyncd is not running on srv56, updates for wikis served by old indexer halted since Oct7. Run rsync --daemon on srv56 | 
  [production] | 
            
  
    | 
      
        2008-12-12
      
      §
     | 
  
    
  | 22:47 | 
  <brion> | 
  patched Bugzilla so we can exclude CC-only mails from wikibugs-l ([bugzilla:15585]]) | 
  [production] | 
            
  | 21:52 | 
  <brion> | 
  scapping to r44509 | 
  [production] | 
            
  | 19:19 | 
  <brion> | 
  put all the themes and plugins and patches back on wordpress for blog.wm.o. whee | 
  [production] | 
            
  | 19:15 | 
  <brion> | 
  restarted apache on isidore while fiddling with php error logging settings and blog started magically working again. sigh. going back to tweak its config back to normal | 
  [production] | 
            
  | 18:04 | 
  <brion> | 
  we managed to fix the svn update conflict on blog.wm.o (to wordpress 2.7) but it's still showing main page as blank | 
  [production] | 
            
  | 17:42 | 
  <mark> | 
  Telia connection / BGP session was up for 20 hours; problem seems resolved. Removed route filters | 
  [production] | 
            
  | 00:29 | 
  <brion> | 
  bumping to r44485 for more NS fixes for ms, ast | 
  [production] | 
            
  | 00:12 | 
  <brion> | 
  scapping bump to r44484, fixing a few issues w/ hu | 
  [production] | 
            
  | 00:06 | 
  <brion> | 
  updated wikibugs irc script to r44483, fixes issues w/ users w/o real name setting | 
  [production] |