| 2009-02-18
      
      § | 
    
  | 23:09 | <mark> | it's good to know that some other @mark will be responsible now | [production] | 
            
  | 23:06 | <domas> | #tag #clouds #are #awesome | [production] | 
            
  | 23:04 | <domas> | @brion, will we have to log in #web20 formats from now on? #roundedcorners and such? | [production] | 
            
  | 22:59 | <brion> | poke poke | [production] | 
            
  | 22:44 | <paperweight> | changed root password on all servers to 'web20rox' | [production] | 
            
  | 22:43 | <idiotsinthehouse> | hey all identi.ca idiots! | [production] | 
            
  | 22:41 | <Andrew> | morebots back up, now logs to identi.ca with the name wikimediatech | [production] | 
            
  | 22:30 | <Andrew> | test | [production] | 
            
  | 22:28 | <Andrew> | test | [production] | 
            
  | 22:27 | <Andrew> | test | [production] | 
            
  | 22:20 | <Andrew> | This is a test; a test is this | [production] | 
            
  | 21:59 | <Rob> | PDF creation moved to pdf1 | [production] | 
            
  | 21:58 | <Rob> | changed pdf generation from eruzumi to pdf1, testing. | [production] | 
            
  | 19:21 | <Rob> | srv255 changed to pdf1 and moved, drac setup along with dns resolution | [production] | 
            
  | 19:19 | <brion> | scapping | [production] | 
            
  | 19:18 | <brion> | svn up'ing test to r47457 | [production] | 
            
  | 18:37 | <Rob> | reinstalling srv209 due to dhcp misconfiguration making it think it was srv208 | [production] | 
            
  | 15:13 | <mark> | Restarted all upload frontend squids to get rid of the memleaking | [production] | 
            
  | 14:20 | <mark> | Blocked all non-GET/HEAD HTTP methods in requests to upload frontend squids | [production] | 
            
  | 12:46 | <Tim> | put r47447 live for temporary proposed fix of bug 17552 | [production] | 
            
  | 08:38 | <Tim> | svn up r47434 to fix Special:BrokenRedirects | [production] | 
            
  | 08:04 | <Tim> | cleaned up binlogs on db2 | [production] | 
            
  | 06:33 | <brion> | note there's a live hack in api categorymembers query which may be breaking lookups | [production] | 
            
  | 05:54 | <Tim> | set up bugzilla attachment_base, pointing to the new domain http://bug-attachment.wikimedia.org/, and set allow_attachment_display=on | [production] | 
            
  | 05:51 | <brion> | disabling $wgTorTagChanges in CommonSettings after the ext gets loaded (needs fix for testwiki) | [production] | 
            
  | 05:46 | <brion> | syncing reverted expr.php w/o bc stuff | [production] | 
            
  | 05:25 | <brion> | syncing extensions/FlaggedRevs/specialpages/OldReviewedPages_body.php fix | [production] | 
            
  | 05:24 | <brion> | syncing fix to Expr.php for bcpow() error | [production] | 
            
  | 05:16 | <brion> | syncing fix to extensions/ParserFunctions/Expr.php | [production] | 
            
  | 04:59 | <brion> | starting scap process... | [production] | 
            
  | 04:52 | <brion> | svn up'ing test to r47418 | [production] | 
            
  | 04:45 | <brion> | svn up'd test to 47417 | [production] | 
            
  | 04:30 | <brion> | removing editor, reviewer from add/remove for all users in test. that ws an old test not needed anymore :D | [production] | 
            
  | 03:42 | <brion> | rc tags tables created sitewide; should be safe to scap and check for final problems if we're brave | [production] | 
            
  | 03:35 | <brion> | applying patch-change-tags to all wikis | [production] | 
            
  | 02:57 | <brion> | ran patch-change_tag.sql on testwiki | [production] | 
            
  | 02:52 | <brion> | full svn up'ing for test wiki | [production] | 
            
  | 02:06 | <brion> | worked around breakage with pager base class incompat with latest codereview :P | [production] | 
            
  | 01:52 | <brion> | svn up'ing CodeReview to aid in completing code review ;) | [production] | 
            
  
    | 2009-02-17
      
      § | 
    
  | 23:58 | <Rob> | srv217-srv223 installed and online as apache servers.  Updated dsh groups and nagios, as well as pybal | [production] | 
            
  | 23:24 | <Rob> | installed OS on srv217-srv223, moving on to package installation. | [production] | 
            
  | 21:12 | <Rob> | reinstalling srv209, which thought it was srv208.  silly server.  srv208 has not been installed, gave to tomasz to check against setup checklist. | [production] | 
            
  | 21:05 | <Rob> | actually, srv209 installed as 208, bad dhcp entry.  Fixing | [production] | 
            
  | 21:04 | <Rob> | pulling srv208 and srv209 for quick reboots, their drac ips are wrong. | [production] | 
            
  | 21:04 | <Rob> | racked srv217-223 (also racked srv224/225 but no power yet) | [production] | 
            
  | 18:30 | <brion> | starting a batch run of update-special-pages-small just to ensure it actually works | [production] | 
            
  | 18:25 | <brion> | fixed hardcoded /usr/local path for PHP and use of obsolete /etc/cluster in update-special-pages and  update-special-pages-small; removing misleading log files ([[bugzilla:17534]]) | [production] | 
            
  | 01:49 | <Tim> | deleting all enotif jobs from the job queue, there is still a huge backlog | [production] |