| 2008-12-17
      
      § | 
    
  | 23:18 | <RobH> | more lucene changes | [production] | 
            
  | 22:36 | <brion> | applied fix for Android browser on mobile gateway (also did the pl language setup recently) | [production] | 
            
  | 22:06 | <RobH> | more lucene.php changes | [production] | 
            
  | 21:12 | <RobH> | additions to lucene.php per rainmain | [production] | 
            
  | 20:39 | <mark> | Corrected LVS service IPs on search2, search10-12 | [production] | 
            
  | 20:03 | <brion> | hacked mw-serve init script on yongle into shape. will commit it in a bit and update docs | [production] | 
            
  | 19:38 | <brion> | pdf server seems to have eaten all temp space on yongle. clearing... | [production] | 
            
  | 19:26 | <mark> | Set up search2, search8-12 | [production] | 
            
  | 18:57 | <RobH> | pushing dns changes for new misc. servers management resolution | [production] | 
            
  | 18:30 | <RobH> | updated lucene.php with rainman to do things that I really do not get but he knows about. | [production] | 
            
  | 16:28 | <RobH> | new servers auth1, nfs2, streber and williams are racked, IP's allocated, DRAC working.  No DHCP entries or OS installed yet. | [production] | 
            
  | 16:08 | <mark> | restarted lighttpd on zwinger | [production] | 
            
  | 15:59 | <RobH> | added williams to dns records, updated dns | [production] | 
            
  | 15:50 | <TimStarling> | removed some binlogs on ixia | [production] | 
            
  | 01:17 | <brion> | scapping a couple more fixes to r44698 | [production] | 
            
  | 00:36 | <brion-codereview> | srv126 is borked -- read-only filesystem | [production] | 
            
  | 00:23 | <brion-codereview> | scapping to 44696 | [production] | 
            
  | 00:23 | <brion-codereview> | scapping | [production] | 
            
  | 00:15 | <brion-codereview> | svn up'ing on test... | [production] | 
            
  
    | 2008-12-16
      
      § | 
    
  | 23:09 | <brion-codereview> | disabling FixedImage extension -- was used for old 2006 and 2007 fundraisers; images no longer exist and are not applicable to current fundraisers | [production] | 
            
  | 20:34 | <RobH> | ariel is dead, will decommission later. | [production] | 
            
  | 20:29 | <RobH> | ariel is fubar, rebooting and investigating. | [production] | 
            
  | 20:25 | <RobH> | restarted services on sq13 | [production] | 
            
  | 20:21 | <RobH> | took down sq13 to clean its cache | [production] | 
            
  | 20:09 | <RobH> | replaced bad /c0/p0 in amane | [production] | 
            
  | 18:47 | <brion> | applying temporary resource limit lift on enwiki for an IP for workshop in SF | [production] | 
            
  | 17:40 | <RobH> | updated dns for misc. servers project. | [production] | 
            
  | 01:08 | <brion> | deploying r44643 update to CodeReview subversion proxy (swapped encoding protocol to avoid bugs in json_decode with some diffs) | [production] | 
            
  | 00:04 | <brion> | running cleanupTitles.php in bg on all wikis... | [production] | 
            
  
    | 2008-12-15
      
      § | 
    
  | 23:20 | <brion> | going to test fixes for FiveUpgrade.inc to back cleanupTitles.php, cleanupImages.php etc | [production] | 
            
  | 22:22 | <RobH> | changed settings on metawiki to allow banned users to edit their talk pages per https://bugzilla.wikimedia.org/show_bug.cgi?id=16621 | [production] | 
            
  | 21:25 | <brion> | reenabling handheld skin setting, was turned off during overload emergencies on 11-17 | [production] | 
            
  | 21:13 | <brion> | rsyncd appears to be running on srv56. does anything else need to be done for index updates? | [production] | 
            
  | 20:10 | <brion> | yongle hanging again, restarting apache | [production] | 
            
  | 18:58 | <RobH> | started rsync daemon on srv56 per rainman | [production] | 
            
  | 18:35 | <RobH> | setup new planet per https://bugzilla.wikimedia.org/show_bug.cgi?id=16511. | [production] | 
            
  | 01:39 | <brion-weekend> | applying API deletion log fix from r44541 ([[bugzilla:16626]]) | [production] | 
            
  | 00:09 | <rainman-sr> | rsyncd is not running on srv56, updates for wikis served by old indexer halted since Oct7. Run rsync --daemon on srv56 | [production] | 
            
  
    | 2008-12-12
      
      § | 
    
  | 22:47 | <brion> | patched Bugzilla so we can exclude CC-only mails from wikibugs-l ([bugzilla:15585]]) | [production] | 
            
  | 21:52 | <brion> | scapping to r44509 | [production] | 
            
  | 19:19 | <brion> | put all the themes and plugins and patches back on wordpress for blog.wm.o. whee | [production] | 
            
  | 19:15 | <brion> | restarted apache on isidore while fiddling with php error logging settings and blog started magically working again. sigh. going back to tweak its config back to normal | [production] | 
            
  | 18:04 | <brion> | we managed to fix the svn update conflict on blog.wm.o (to wordpress 2.7) but it's still showing main page as blank | [production] | 
            
  | 17:42 | <mark> | Telia connection / BGP session was up for 20 hours; problem seems resolved. Removed route filters | [production] | 
            
  | 00:29 | <brion> | bumping to r44485 for more NS fixes for ms, ast | [production] | 
            
  | 00:12 | <brion> | scapping bump to r44484, fixing a few issues w/ hu | [production] |