| 2011-08-04
      
      § | 
    
  | 15:40 | <mark> | Mounted nas1001-a:/vol/images on ms1002:/vol/images for testing, started rsync copy from ms1002:/export/upload to /vol/images | [production] | 
            
  | 15:20 | <mark> | Created 12-disk aggregate test0 on nas1001-a | [production] | 
            
  | 15:01 | <RoanKattouw> | Staging some HTTPS support / prot rel URL code on testwiki, see today's 1.17wmf1 merges. DO NOT DEPLOY SITEWIDE until tomorrow | [production] | 
            
  | 13:19 | <RobH> | updated wikibugs on mchenry about 10 minutes ago and forgot to admin log | [production] | 
            
  | 13:18 | <mark> | Created new 64 bit aggregate and root volume on nas1001-a, copied over root data, destroyed old root volume, moving new aggregate to first disks | [production] | 
            
  | 13:00 | <RobH> | disabled line numbers in vimrc.local on puppet because i hate line numbers (it already says the line number yer on in the corner) | [production] | 
            
  | 12:21 | <mark> | Started disk replace operation of 0a.10.0 by 0a.10.3 and 0a.10.1 by 0a.10.4 on nas1-b | [production] | 
            
  | 12:12 | <mark> | Started disk replace operation of 0b.00.0 by 0a.01.1, and 0b.00.1 by 0a.01.2 on nas1-a | [production] | 
            
  | 12:08 | <mutante> | ragweed - package updates, restart required | [production] | 
            
  | 11:05 | <mutante> | amssq46 - power cycle, clean squid, dist-upgrade/kernel, reboot | [production] | 
            
  | 02:18 | <LocalisationUpdate> | completed at Thu Aug  4 02:20:48 UTC 2011 | [production] | 
            
  
    | 2011-08-03
      
      § | 
    
  | 21:22 | <binasher> | previous log line should have read $ppsize + 50 | [production] | 
            
  | 21:21 | <binasher> | dropping apache maxclients on mobile servers to ($ppsize - 50) | [production] | 
            
  | 18:24 | <aaron> | synchronized multiversion/MWMultiVersion.php  'deployed [[rev:93845|r93845]]' | [production] | 
            
  | 18:23 | <aaron> | synchronized multiversion/MWScript.php  'deployed [[rev:93845|r93845]]' | [production] | 
            
  | 16:19 | <tfinc> | synchronized wmf-config/InitialiseSettings.php  'testing collectinos on hewiki' | [production] | 
            
  | 16:13 | <mutante> | installed Net::Stomp Perl module on spence via cpan shell which fixed RT #703 | [production] | 
            
  | 14:32 | <RobH> | updating dns for magnesium and zinc eqiad misc servers to public IPs RT#1245 | [production] | 
            
  | 13:24 | <tstarling> | synchronized php-1.17/extensions/CodeReview/ui/WordCloud.php | [production] | 
            
  | 13:24 | <tstarling> | synchronized php-1.17/extensions/CodeReview/CodeReview.php | [production] | 
            
  | 12:21 | <mark> | Rebooting both nas1001 controllers with corrected network config | [production] | 
            
  | 10:02 | <root> | synchronized wmf-config/mc.php  'swapped out all down servers returning 0 with ones returning 100, cleaned up spares and down list to be accurate' | [production] | 
            
  | 09:44 | <RobH> | swapping out down memcached servers iwth tested up ones | [production] | 
            
  | 09:44 | <RobH> | incorrect, turns out there are some down, they reply but do not return full weight | [production] | 
            
  | 09:44 | <robh> | synchronized wmf-config/mc.php  'replacing down servers' | [production] | 
            
  | 09:40 | <RobH> | nagios shows spence memcached error, yet manual testing shows all memcached are pooled fine. | [production] | 
            
  | 09:01 | <Ryan_Lane> | installing ssl1001-4 | [production] | 
            
  | 08:09 | <apergos> | live hack to /usr/share/etherpad/etherpad/bin/run-local.sh on hooper to force maxthreads to 512 for etherpad.  restarted etherpad, this may or may not help connection issues (maybe we'll just oom instead?) | [production] | 
            
  | 07:40 | <domas> | stupid bot and its quotes, http://p.defau.lt/?UUberPb2hRtPxOzG7Q6OwQ is the url | [production] | 
            
  | 07:39 | <midom> | synchronized php/includes/db/DatabaseMysql.php  'livehacking out sql_mode and set charset code - http://p.defau.lt/?UUberPb2hRtPxOzG7Q6OwQ' | [production] | 
            
  | 02:20 | <LocalisationUpdate> | completed at Wed Aug  3 02:22:48 UTC 2011 | [production] | 
            
  
    | 2011-08-02
      
      § | 
    
  | 23:25 | <binasher> | done with that epic test | [production] | 
            
  | 23:04 | <binasher> | redeploying revised mobile acls with the redirector only matching /wiki/ urls for brief (15min) test | [production] | 
            
  | 22:19 | <aaron> | synchronized wmf-config/CommonSettings.php  'added $wgExtendedVersionNumber (Het Deploy), should have no effect now' | [production] | 
            
  | 19:19 | <binasher> | reverted mobile redirect acls | [production] | 
            
  | 18:58 | <binasher> | deploying mobile redirect acls/helper to all frontend text squids (--slow) | [production] | 
            
  | 18:16 | <jeluf> | synchronized wmf-config/InitialiseSettings.php  '30186 - Install WikiLove on the Hebrew Wikipedia' | [production] | 
            
  | 18:03 | <notpeter> | restarting torrus again. cuz it's my favorite thing to do... | [production] | 
            
  | 17:16 | <Reedy> | As part of fatal coming from OAIRepository decreased by 10% from 50 to 45. No fatals in logs for nearly a week though | [production] | 
            
  | 17:15 | <reedy> | synchronized wmf-config/CommonSettings.php  '[[bugzilla:29304|bug 29304]] Set to 45' | [production] | 
            
  | 17:04 | <reedy> | synchronized php-1.17/extensions/OAI/OAIRepo_body.php  '[[rev:93770|r93770]]' | [production] | 
            
  | 17:03 | <reedy> | synchronized php-1.17/extensions/OAI/OAIRepo.php  '[[rev:93770|r93770]]' | [production] | 
            
  | 17:02 | <reedy> | synchronized php-1.17/includes/parser/Preprocessor_DOM.php  '[[rev:93763|r93763]]' | [production] | 
            
  | 15:28 | <catrope> | synchronized php/includes/media/SVG.php  '[[rev:93750|r93750]]' | [production] | 
            
  | 14:23 | <laner> | synchronized wmf-config/InitialiseSettings.php  'Changing main namespace for ladwiki' | [production] | 
            
  | 12:49 | <midom> | synchronized wmf-config/InitialiseSettings.php  'reenabling disk cache' | [production] | 
            
  | 12:42 | <domas> | synced http://p.defau.lt/?Jze2xAwLV4Fvh8JC2B_KqQ to fix memcached side of pcache | [production] | 
            
  | 12:40 | <midom> | synchronized php/includes/objectcache/MemcachedClient.php | [production] | 
            
  | 12:07 | <midom> | synchronized wmf-config/InitialiseSettings.php  'disabling disk cache' | [production] | 
            
  | 12:00 | <Ryan_Lane> | changing management dns entry for phosphorus to ssl1004 | [production] |