| 2012-01-18
      
      § | 
    
  | 18:36 | <mutante> | fixed DHCP config for mw1108 on brewster, had the string "Failed to connect to 10.65.1.108." where the MAC address should have been. | [production] | 
            
  | 18:27 | <RobH> | searchidx1001 memory replaced per rt 2208 | [production] | 
            
  | 18:20 | <mutante> | tried to PXE boot mw1108 but no DHCP offers received | [production] | 
            
  | 18:15 | <RobH> | searchidx1001 memory being replaced | [production] | 
            
  | 18:14 | <LeslieCarr> | re-preffing tele2 routes | [production] | 
            
  | 18:11 | <RobH> | db1004 hard disk replaced per rt#2140, rebuilding | [production] | 
            
  | 17:40 | <LeslieCarr> | Draining HE to perform maintenance on the physical port | [production] | 
            
  | 16:57 | <reedy> | synchronized php-1.18/extensions/CongressLookup/SpecialCongressLookup.php  '[[rev:109395|r109395]]' | [production] | 
            
  | 14:45 | <mark> | Changed service IP addresses of lists.wikimedia.org in DNS to US prefixes | [production] | 
            
  | 14:40 | <mark> | Disabled hold_domains on sodium and lily | [production] | 
            
  | 14:28 | <mark> | Setup lily to route lists.wikimedia.org mails to sodium | [production] | 
            
  | 14:21 | <mark> | rsync complete. Running dpkg-reconfigure mailman on sodium | [production] | 
            
  | 13:43 | <demon> | synchronized php-1.18/extensions/CongressLookup/SpecialCongressLookup.php  '[[rev:109362|r109362]]' | [production] | 
            
  | 13:38 | <mark> | Started rsync of selected mailman directories under /var/lib/mailman from lily to sodium | [production] | 
            
  | 13:37 | <mark> | Removed all test messages on the exim4 queue on sodium | [production] | 
            
  | 13:37 | <mark> | Created /var LVM snapshot on lily | [production] | 
            
  | 13:35 | <mark> | Stopped lighttpd on lily | [production] | 
            
  | 13:34 | <mark> | Stopped mailman on lily and sodium | [production] | 
            
  | 13:30 | <mark> | Set hold_domains = lists.wikimedia.org on lily, to hold new lists mails on the queue | [production] | 
            
  | 13:30 | <mark> | Starting mailman migration | [production] | 
            
  | 13:03 | <mutante> | restarted pdns on ns0 | [production] | 
            
  | 08:49 | <neilk> | synchronizing Wikimedia installation... : | [production] | 
            
  | 07:45 | <awjrichards> | synchronized php/extensions/CongressLookup/SpecialCongressLookup.php  '[[rev:109336|r109336]]' | [production] | 
            
  | 06:28 | <awjrichards> | synchronized php/extensions/CongressLookup/SpecialCongressLookup.php  '[[rev:109324|r109324]]' | [production] | 
            
  | 06:24 | <awjrichards> | synchronized php/extensions/CongressLookup/SpecialCongressLookup.php  '[[rev:109322|r109322]]' | [production] | 
            
  | 06:08 | <awjrichards> | synchronized php/extensions/CongressLookup/SpecialCongressLookup.php  '[[rev:109319|r109319]]' | [production] | 
            
  | 05:32 | <laner> | synchronized wmf-config/InitialiseSettings.php  'Disabling CentralNotice for simplewiki' | [production] | 
            
  | 05:25 | <laner> | synchronized wmf-config/InitialiseSettings.php  'Disabling moodbar on enwiki' | [production] | 
            
  | 04:59 | <laner> | synchronized wmf-config/InitialiseSettings.php  'Disabling all editing for enwiki for SOPA blackout' | [production] | 
            
  | 04:50 | <Ryan_Lane> | queuing up changes for totally disabling edits. DO NOT SCAP! DO NOT SYNC InitialiseSettings! | [production] | 
            
  | 04:47 | <Ryan_Lane> | Editing enwiki's MediaWiki:Robots.txt to disallow BannerController for SOPA blackout | [production] | 
            
  | 04:45 | <laner> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 04:44 | <Ryan_Lane> | Disabling anon editing and page creation by users on enwiki for SOPA blackout | [production] | 
            
  | 04:33 | <neilk> | synchronized wmf-config/InitialiseSettings.php  'enable CongressLookup on enwiki' | [production] | 
            
  | 04:01 | <preilly> | synchronized php-1.18/extensions/MobileFrontend/ApplicationTemplate.php  'update version' | [production] | 
            
  | 03:56 | <neilk> | synchronizing Wikimedia installation... : deploying CongressLookup (for i18n reasons, not deploying to enwiki) | [production] | 
            
  | 03:28 | <laner> | synchronized wmf-config/InitialiseSettings.php  'Removing restriction of display title for SOPA landing pages' | [production] | 
            
  | 02:35 | <binasher> | cp1039-40 are now in service for mobile wikipedia | [production] | 
            
  | 02:04 | <LocalisationUpdate> | completed (1.18) at Wed Jan 18 02:04:57 UTC 2012 | [production] | 
            
  | 01:35 | <RobH> | cp1040 and cp1036 ready for use | [production] | 
            
  | 01:33 | <RobH> | cp1037, cp1038, cp1039 os installed, varnish partitions mounted, and puppet run | [production] | 
            
  
    | 2012-01-17
      
      § | 
    
  | 22:47 | <binasher> | ram only varnish instance now running on marmontel in front of apache/wordpress | [production] | 
            
  | 22:07 | <Ryan_Lane> | installing memcache on marmontel | [production] | 
            
  | 20:54 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Bug 33769 - Allow bureaucrats to remove sysop rights at Bashkir Wikipedia' | [production] | 
            
  | 20:13 | <mutante> | en.planet updates were stuck. reason was corrupted cache causing "bsddb.db.DBPageNotFoundError" which broke update script. solution was to kill stuck updates, delete files in cache dir and run update manually | [production] | 
            
  | 19:59 | <reedy> | synchronized php-1.18/includes/Feed.php  '[[rev:109197|r109197]]' | [production] | 
            
  | 19:36 | <Ryan_Lane> | added Cite extension to labscosnole | [production] | 
            
  | 19:29 | <reedy> | synchronized php-1.18/extensions/ArticleFeedbackv5/modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js  '[[rev:109186|r109186]]' | [production] | 
            
  | 18:05 | <RobH> | blog is instantly faster | [production] | 
            
  | 18:05 | <RobH> | theme updated on blog along with settting limit back to 20 comments per page | [production] |