| 2013-05-03
      
      § | 
    
  | 22:19 | <py> | synchronized wmf-config/db-pmtpa.php  'adding new pmtpa slave' | [production] | 
            
  | 21:09 | <cmjohnson1> | db10 is decommissioned...powering down for the final time | [production] | 
            
  | 20:03 | <maxsem> | synchronized php-1.22wmf2/extensions/MobileFrontend/includes/MobileFrontend.hooks.php | [production] | 
            
  | 19:54 | <maxsem> | synchronized php-1.22wmf3/extensions/MobileFrontend/includes/MobileFrontend.hooks.php | [production] | 
            
  | 07:51 | <MaxSem> | OMG logmsgbot is a terminator! | [production] | 
            
  | 06:16 | <tstarling> | synchronized wmf-config/missing.php | [production] | 
            
  | 04:03 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Fri May  3 04:03:12 UTC 2013 | [production] | 
            
  | 02:26 | <LocalisationUpdate> | completed (1.22wmf2) at Fri May  3 02:26:02 UTC 2013 | [production] | 
            
  | 02:15 | <LocalisationUpdate> | completed (1.22wmf3) at Fri May  3 02:15:45 UTC 2013 | [production] | 
            
  | 01:25 | <hashar> | Zuul processed all its events, seems to be fine now. *sleep* | [production] | 
            
  | 00:32 | <dzahn> | synchronized docroot | [production] | 
            
  | 00:32 | <mutante> | sync-docroot for iegcom wiki | [production] | 
            
  | 00:25 | <aaron> | synchronized multiversion/MWMultiVersion.php  '6512451451ac6617bedbfce7be17045bed794d70' | [production] | 
            
  | 00:15 | <mutante> | DNS update - add iegcom | [production] | 
            
  
    | 2013-05-02
      
      § | 
    
  | 23:20 | <mutante> | restarting lucene on search1021, search1022, search1017, search1018 (with some waiting in between) | [production] | 
            
  | 23:17 | <hashar> | yeah after hours and hours of fighting, Jenkins is finally working again. | [production] | 
            
  | 23:10 | <aaron> | synchronized wmf-config/jobqueue-eqiad.php | [production] | 
            
  | 22:58 | <hashar> | restarting jenkins… it got a few threads blocked and the main process is at 100% usage for now reason | [production] | 
            
  | 22:58 | <mutante> | repooling search1016 | [production] | 
            
  | 22:40 | <mutante> | gallium, run puppet, graceful Apache to deploy split log files | [production] | 
            
  | 22:34 | <asher> | synchronized wmf-config/db-eqiad.php  'pulling db1020 for upgrade' | [production] | 
            
  | 22:31 | <mutante> | depooling search1016, restarting lucene, etc.. (Search#Adding_new_wikis) | [production] | 
            
  | 22:29 | <mutante> | repooling search1015 | [production] | 
            
  | 22:22 | <hashar> | Zuul restarted.  The bug about slowness is {{bug|48025}} | [production] | 
            
  | 22:15 | <hashar> | restarting Zuul | [production] | 
            
  | 22:13 | <hashar> | Deploying a workaround on Zuul to make it stop querying the Jenkins API when it just want to check whether a job exist. {{gerrit|62095}} | [production] | 
            
  | 22:04 | <mutante> | depooling search1015 in pybal | [production] | 
            
  | 21:43 | <mflaschen> | synchronized php-1.22wmf3/extensions/GuidedTour/  'Sync GuidedTour to 1.22wmf3 for E3 deployment' | [production] | 
            
  | 21:41 | <mflaschen> | synchronized php-1.22wmf2/extensions/GuidedTour/  'Sync GuidedTour to 1.22wmf2 for E3 deployment' | [production] | 
            
  | 21:38 | <mutante> | importing wikimania2014wiki into search indexers | [production] | 
            
  | 21:34 | <mutante> | restarting search indexers on searchidx2, searchidx1001 to make sure the indexer knows about new wiki | [production] | 
            
  | 21:27 | <mflaschen> | synchronized php-1.22wmf3/skins/common/shared.css  'Sync font-size change for edit section links' | [production] | 
            
  | 21:15 | <spage> | synchronized php-1.22wmf2/extensions/ConfirmEdit  'update 1.22wmf2 to wmf3 version of ConfirmEdit' | [production] | 
            
  | 21:13 | <LeslieCarr> | adding GEANT via fiberring to avoid-paths | [production] | 
            
  | 20:20 | <reedy> | synchronized php-1.22wmf3/includes/ | [production] | 
            
  | 20:15 | <reedy> | synchronized docroot/bits/WikipediaMobileFirefoxOS/ | [production] | 
            
  | 19:51 | <Jeff_Green> | authdns-update to move db1025 to frack.eqiad.wmnet | [production] | 
            
  | 19:46 | <reedy> | synchronized wmf-config/interwiki.cdb  'Updating interwiki cache' | [production] | 
            
  | 19:39 | <ottomata> | rebooting oxygen | [production] | 
            
  | 19:32 | <ottomata> | re-enabling puppet on cp1031, it was administratively disabled.  running puppet there. | [production] | 
            
  | 19:23 | <RobH> | professor was already powered down (why?)  starting it back up now | [production] | 
            
  | 19:22 | <ottomata> | all webrequest udp2log loggers (squid and varnish) now send to gadolinium for socat unicast -> multicast relay | [production] | 
            
  | 19:22 | <RobH> | rebooting professor | [production] | 
            
  | 19:16 | <Jeff_Green> | moving db1025 into frack-fundraising1-c-eqiad | [production] | 
            
  | 19:13 | <kaldari> | synchronized php-1.22wmf2/extensions/Echo  'sync Echo ext for en.wiki' | [production] | 
            
  | 19:13 | <ottomata> | deployed squid frontend.conf.php changes to remove locke and send logs directly to gadolinium for multicast relay | [production] | 
            
  | 19:11 | <kaldari> | synchronized php-1.22wmf3/extensions/Echo  'sync Echo ext' | [production] | 
            
  | 19:08 | <kaldari> | synchronized php-1.22wmf3/extensions/Echo  'sync Echo ext' | [production] |