| 2013-06-13
      
      § | 
    
  | 22:08 | <LeslieCarr> | restarted squid on cp1004 due to "Jun 13 21:40:34 cp1004 squid[1961]: commBind: Cannot bind socket FD 23 to *:0: (98) Address already in use" | [production] | 
            
  | 22:03 | <LocalisationUpdate> | failed: git pull of extensions failed | [production] | 
            
  | 21:54 | <LocalisationUpdate> | failed: git pull of extensions failed | [production] | 
            
  | 21:14 | <spage> | synchronized php-1.22wmf6/extensions/GuidedTour  'latest GuidedTour' | [production] | 
            
  | 21:13 | <spage> | synchronized php-1.22wmf6/extensions/EventLogging  'latest EventLogging' | [production] | 
            
  | 20:51 | <LeslieCarr> | mw1173 has bad memory, rt 5294 | [production] | 
            
  | 20:48 | <spage> | synchronized php-1.22wmf7/extensions/GuidedTour  'latest GuidedTour' | [production] | 
            
  | 20:47 | <LeslieCarr> | repuppetizing mw1020 after a reinstall | [production] | 
            
  | 20:40 | <Jeff_Green> | authdns update | [production] | 
            
  | 20:40 | <spage> | synchronized php-1.22wmf7/extensions/EventLogging  'latest EventLogging' | [production] | 
            
  | 20:18 | <olivneh> | synchronized php-1.22wmf7/extensions/UploadWizard/resources/mw.LanguageUpWiz.js  'Ia09ace5dc: Fix UploadWizard's config variables (2/2)' | [production] | 
            
  | 20:17 | <olivneh> | synchronized php-1.22wmf7/extensions/UploadWizard/includes/specials/SpecialUploadWizard.php  'Ia09ace5dc: Fix UploadWizard's config variables' | [production] | 
            
  | 20:15 | <kaldari> | synchronized php-1.22wmf6/extensions/Thanks/modules/ext.thanks.thank.js  'fix IE Thanks bug' | [production] | 
            
  | 20:15 | <reedy> | synchronized php-1.22wmf5/cache/ | [production] | 
            
  | 20:06 | <gwicke> | updated Parsoid to a33980cf | [production] | 
            
  | 19:44 | <binasher> | rebooted mc1009 as a firedrill, chaos monkey ahoy! | [production] | 
            
  | 19:35 | <cmjohnson1> | dns update | [production] | 
            
  | 19:05 | <kaldari> | synchronized php-1.22wmf7/extensions/Thanks/modules/ext.thanks.thank.js  'fix IE Thanks bug' | [production] | 
            
  | 18:58 | <py> | synchronized wmf-config/lucene-production.php  'moving last of search traffic to pmtpa' | [production] | 
            
  | 18:49 | <py> | synchronized wmf-config/lucene-production.php  'moving more search traffic to pmtpa' | [production] | 
            
  | 18:32 | <py> | synchronized wmf-config/lucene-production.php  'moving search traffic to pmtpa for en, de, fr, and ja' | [production] | 
            
  | 18:12 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: test2wiki, testwikidatawiki and mediawikiwiki to 1.22wmf7 | [production] | 
            
  | 18:06 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: wikiversity and wikipedia to 1.22wmf6 | [production] | 
            
  | 16:28 | <reedy> | Finished syncing Wikimedia installation... : testwiki to 1.22wmf7 and build l10n cache | [production] | 
            
  | 16:11 | <reedy> | Started syncing Wikimedia installation... : testwiki to 1.22wmf7 and build l10n cache | [production] | 
            
  | 16:03 | <reedy> | synchronized w | [production] | 
            
  | 16:01 | <reedy> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 15:46 | <reedy> | synchronized docroot | [production] | 
            
  | 15:44 | <reedy> | synchronized php-1.22wmf7  'Initial file sync' | [production] | 
            
  | 07:32 | <apergos> | the last of the c2100s in the swift pool has been replaced (and new rings pushed), finally. now we "only" have the H310 controllers to replace... | [production] | 
            
  | 02:28 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Thu Jun 13 02:28:27 UTC 2013 | [production] | 
            
  | 02:15 | <LocalisationUpdate> | completed (1.22wmf5) at Thu Jun 13 02:14:58 UTC 2013 | [production] | 
            
  | 02:08 | <LocalisationUpdate> | completed (1.22wmf6) at Thu Jun 13 02:08:12 UTC 2013 | [production] | 
            
  | 00:29 | <asher> | synchronized wmf-config/mc-eqiad.php  'disable persistent conns to twmprxy' | [production] | 
            
  | 00:23 | <paravoid> | ceph readding osd.81, osd.125, osd.137, osd.141 after disk replacement (#5202, #5228, #5248, #5263) | [production] | 
            
  | 00:21 | <asher> | synchronized wmf-config/mc-eqiad.php  'trying twemproxy on all eqiad hosts' | [production] | 
            
  | 00:13 | <asher> | synchronized wmf-config/mc-eqiad.php  'large server_failure_limit, small retry_timeout for twemproxy test hosts' | [production] | 
            
  
    | 2013-06-12
      
      § | 
    
  | 23:56 | <asher> | synchronized wmf-config  'revert twemproxy' | [production] | 
            
  | 23:47 | <asher> | restarted twemproxy on all servers | [production] | 
            
  | 23:45 | <asher> | synchronized wmf-config/twemproxy-pmtpa.yaml  'pull 10.0.12.15' | [production] | 
            
  | 23:42 | <aaron> | cleared profiling data | [production] | 
            
  | 23:34 | <asher> | synchronized wmf-config  'use twemproxy for memcached access from all hosts' | [production] | 
            
  | 23:32 | <tstarling> | cleared profiling data | [production] | 
            
  | 23:10 | <paravoid> | apt: updating libmemcached/php5-memcached (#5268) | [production] | 
            
  | 22:59 | <asher> | synchronized wmf-config/mc-eqiad.php  'set (server_failure_limit, retry_timeout) = 0 for twemproxy test hosts' | [production] | 
            
  | 22:25 | <aaron> | synchronized php-1.22wmf6/includes/objectcache/MemcachedPeclBagOStuff.php  '7f091934ceae0280b44292e5ce426b24c5a579a2' | [production] | 
            
  | 22:23 | <aaron> | synchronized php-1.22wmf5/includes/objectcache/MemcachedPeclBagOStuff.php  'f6287b2e2faba20aea7379dae84ee7c5dd84983f' | [production] | 
            
  | 21:34 | <mutante> | deploying new favicon on wikitech | [production] | 
            
  | 21:22 | <LeslieCarr> | restarting caesium | [production] | 
            
  | 20:55 | <mutante> | PXE booting analytics1020 | [production] |