| 
      
        2009-02-20
      
      §
     | 
  
    
  | 18:17 | 
  <river> | 
  test | 
  [production] | 
            
  | 18:17 | 
  <river> | 
  test | 
  [production] | 
            
  | 18:16 | 
  <river> | 
  set zil_disable on ms1 to improve nfs write performance | 
  [production] | 
            
  | 18:15 | 
  <mark> | 
  Raised max-conns to 50 | 
  [production] | 
            
  | 18:03 | 
  <mark> | 
  Cut down max conns even more (25) for pmtpa upload backend squids | 
  [production] | 
            
  | 17:40 | 
  <mark> | 
  Limited maximum connections to backend (ms1) to 50 per squid on upload squids, 1000 per squid on text | 
  [production] | 
            
  | 16:19 | 
  <NicDumZ> | 
  nagios definitely needs a nagia | 
  [production] | 
            
  | 16:17 | 
  <domas> | 
  plenty of fedoras had futex deadlocks  | 
  [production] | 
            
  | 16:16 | 
  <Rob> | 
  upgraded kernel and squid on sq14 and sq15 | 
  [production] | 
            
  | 15:49 | 
  <Rob> | 
  updated squid and kernel on sq13, rebooted, back online | 
  [production] | 
            
  | 14:59 | 
  <Rob> | 
  upgraded squid and kernel on sq5, sq6, sq7, sq8 | 
  [production] | 
            
  | 14:51 | 
  <Rob> | 
  upgraded squid and kernel on sq2-sq4 | 
  [production] | 
            
  | 14:50 | 
  <Tim> | 
  updated ContactPage extension, will deploy it on nlwiki shortly | 
  [production] | 
            
  | 10:00 | 
  <mark> | 
  Started backend squid on sq26, it was gone | 
  [production] | 
            
  
    | 
      
        2009-02-19
      
      §
     | 
  
    
  | 23:54 | 
  <brion> | 
  updating AbuseFilter to r47523 :P | 
  [production] | 
            
  | 23:51 | 
  <brion> | 
  updating AbuseFilter to r47522 | 
  [production] | 
            
  | 23:40 | 
  <brion> | 
  updating FlaggedRevs to r47522 | 
  [production] | 
            
  | 23:39 | 
  <Andrew> | 
  Enabled Abuse Filter on MediaWiki.org | 
  [production] | 
            
  | 23:17 | 
  <mark> | 
  Stopped experimental varnish on sq1, please keep Squid off as well | 
  [production] | 
            
  | 22:52 | 
  <Andrew> | 
  Allowed bureaucrats to remove 'sysop' right on testwiki. | 
  [production] | 
            
  | 22:42 | 
  <brion> | 
  updating includes/api to r47522 to fix a couple regressions | 
  [production] | 
            
  | 22:15 | 
  <mark> | 
  Started an experimental varnish instance on sq1 port 80 | 
  [production] | 
            
  | 21:23 | 
  <mark> | 
  Stopped Squids on sq1 | 
  [production] | 
            
  | 14:23 | 
  <Tim> | 
  removing memcached from srv154,srv155,srv157,srv158,srv169,srv170 | 
  [production] | 
            
  | 14:18 | 
  <Tim> | 
  started memcached on srv190-199 | 
  [production] | 
            
  | 14:06 | 
  <mark> | 
  Added "vport=80" to the http_host directive on all backend squids, to force Squid to use the default HTTP port, 80 | 
  [production] | 
            
  | 10:53 | 
  <domas> | 
  livemerged r47483 (backlinks cache read explicit order, :( ) | 
  [production] | 
            
  
    | 
      
        2009-02-18
      
      §
     | 
  
    
  | 23:30 | 
  <river> | 
  test | 
  [production] | 
            
  | 23:12 | 
  <domas> | 
  did anyone notice that our GFDL licensed log messages are syndicated as CC-BY? | 
  [production] | 
            
  | 23:09 | 
  <domas> | 
  @mark, go fix our servers! :) | 
  [production] | 
            
  | 23:09 | 
  <mark> | 
  it's good to know that some other @mark will be responsible now | 
  [production] | 
            
  | 23:06 | 
  <domas> | 
  #tag #clouds #are #awesome | 
  [production] | 
            
  | 23:04 | 
  <domas> | 
  @brion, will we have to log in #web20 formats from now on? #roundedcorners and such? | 
  [production] | 
            
  | 22:59 | 
  <brion> | 
  poke poke | 
  [production] | 
            
  | 22:44 | 
  <paperweight> | 
  changed root password on all servers to 'web20rox' | 
  [production] | 
            
  | 22:43 | 
  <idiotsinthehouse> | 
  hey all identi.ca idiots! | 
  [production] | 
            
  | 22:41 | 
  <Andrew> | 
  morebots back up, now logs to identi.ca with the name wikimediatech | 
  [production] | 
            
  | 22:30 | 
  <Andrew> | 
  test | 
  [production] | 
            
  | 22:28 | 
  <Andrew> | 
  test | 
  [production] | 
            
  | 22:27 | 
  <Andrew> | 
  test | 
  [production] | 
            
  | 22:20 | 
  <Andrew> | 
  This is a test; a test is this | 
  [production] | 
            
  | 21:59 | 
  <Rob> | 
  PDF creation moved to pdf1 | 
  [production] | 
            
  | 21:58 | 
  <Rob> | 
  changed pdf generation from eruzumi to pdf1, testing. | 
  [production] | 
            
  | 19:21 | 
  <Rob> | 
  srv255 changed to pdf1 and moved, drac setup along with dns resolution | 
  [production] | 
            
  | 19:19 | 
  <brion> | 
  scapping | 
  [production] | 
            
  | 19:18 | 
  <brion> | 
  svn up'ing test to r47457 | 
  [production] | 
            
  | 18:37 | 
  <Rob> | 
  reinstalling srv209 due to dhcp misconfiguration making it think it was srv208 | 
  [production] | 
            
  | 15:13 | 
  <mark> | 
  Restarted all upload frontend squids to get rid of the memleaking | 
  [production] | 
            
  | 14:20 | 
  <mark> | 
  Blocked all non-GET/HEAD HTTP methods in requests to upload frontend squids | 
  [production] | 
            
  | 12:46 | 
  <Tim> | 
  put r47447 live for temporary proposed fix of bug 17552 | 
  [production] |