| 2014-03-29
      
      § | 
    
  | 17:31 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'touch' | [production] | 
            
  | 17:30 | <reedy> | synchronized database lists files: Update size related dblists | [production] | 
            
  | 17:28 | <reedy> | updated /a/common to {{Gerrit|I7e9595ce0}}: Prepare GeoData config for Elasticsearch switchover | [production] | 
            
  | 16:54 | <Reedy> | afl_namespace tinyint -> int on all medium wikis | [production] | 
            
  | 16:06 | <hoo> | 800+ connections open on db1021, db1037, db1045 (s5) | [production] | 
            
  | 15:49 | <Reedy> | afl_namespace tinyint -> int on all small wikis | [production] | 
            
  | 15:10 | <Reedy> | afl_namespace tinyint -> int on mediawikiwiki | [production] | 
            
  | 05:59 | <csteipp> | deployed patch for bug63251 for wmf19 and wmf20 | [production] | 
            
  | 03:16 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Sat Mar 29 03:16:24 UTC 2014 (duration 16m 23s) | [production] | 
            
  | 02:36 | <LocalisationUpdate> | completed (1.23wmf20) at 2014-03-29 02:36:39+00:00 | [production] | 
            
  | 02:16 | <LocalisationUpdate> | completed (1.23wmf19) at 2014-03-29 02:16:33+00:00 | [production] | 
            
  | 00:47 | <paravoid> | kill updateSpecial MWScript on terbium & pt-kill updateSpecial frwiki query running for 15m | [production] | 
            
  
    | 2014-03-28
      
      § | 
    
  | 18:37 | <ori> | added jgonera to wmf-deployment; was already a deployer but not in relevant gerrit group | [production] | 
            
  | 17:50 | <demon> | synchronized wmf-config/CommonSettings.php  'Prepping GeoData config for Elasticsearch support. No-op' | [production] | 
            
  | 17:49 | <demon> | synchronized wmf-config/InitialiseSettings-labs.php  'Prepping GeoData config for Elasticsearch support. No-op' | [production] | 
            
  | 17:49 | <demon> | synchronized wmf-config/InitialiseSettings.php  'Prepping GeoData config for Elasticsearch support. No-op' | [production] | 
            
  | 17:03 | <mutante> | DNS update - remove db40,db62 | [production] | 
            
  | 16:56 | <mutante> | shut down db40 | [production] | 
            
  | 15:43 | <hashar> | Jenkins removing pbuilder images from gallium.wikimedia.org and lanthanum.eqiad.wmnet production slaves. Packages should now be build on the labs slaves. | [production] | 
            
  | 14:11 | <mutante> | shut down db62 | [production] | 
            
  | 13:59 | <mutante> | db40 - revoke puppet cert,salt key,remove from monitoring | [production] | 
            
  | 13:57 | <mutante> | db40 - stopping mysql | [production] | 
            
  | 13:56 | <mutante> | db62 - revoke puppet cert,salt key,remove from monitoring | [production] | 
            
  | 12:26 | <hashar> | Jenkins: migrating both labs slaves puppet master to integration-puppetmaster.eqiad.wmflabs | [production] | 
            
  | 10:56 | <hashar> | Jenkins: depooling ntegration-debian-builder pmtpa slave. Jobs should be build on the eqiad slaves now.  debian-glue jobs might be broken for a while until I figure out how to get cow builder setup properly. | [production] | 
            
  | 09:43 | <paravoid> | removing language subdomains for wikidata | [production] | 
            
  | 09:30 | <reedy> | synchronized wmf-config/PrivateSettings.php  'Uncomment AdminSettings.php' | [production] | 
            
  | 04:26 | <ori> | sync-l10nupdate ran to completion. messages that were previously missing now render. | [production] | 
            
  | 04:01 | <RoanKattouw> | Running sync-l10nupdate to sync l10n cache | [production] | 
            
  | 03:56 | <Krinkle> | Rebuilding i18n using mw-update-l10n, syncing ExtensionMessages-1.23wmf20.php doesn't actually regenerate the cache | [production] | 
            
  | 03:55 | <RoanKattouw> | Running mw-update-l10n. Skipped wmf19, is now rebuilding l10ncache for wmf20 because I updated ExtensionMessages-1.23wmf20.php | [production] | 
            
  | 03:54 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Fri Mar 28 03:54:22 UTC 2014 (duration 46m 33s) | [production] | 
            
  | 03:49 | <catrope> | synchronized wmf-config/ExtensionMessages-1.23wmf20.php  'Regenerated to add new JSON dirs that were missing' | [production] | 
            
  | 03:49 | <RoanKattouw> | Maybe the new scap scripts regenerate ExtensionMessages-$version.php at the wrong time (i.e. do things in the wrong order)? | [production] | 
            
  | 03:48 | <Krinkle> | So scap/mw-update-l10n (at branch cut) or LocalisationUpdate must've run mergeMessagesFileList.php  wrongly somehow | [production] | 
            
  | 03:46 | <RoanKattouw> | ExtensionMessages-1.23wmf20.php is out of date, regenerated | [production] | 
            
  | 03:33 | <LocalisationUpdate> | completed (1.23wmf20) at 2014-03-28 03:33:49+00:00 | [production] | 
            
  | 03:27 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Fri Mar 28 03:27:41 UTC 2014 (duration 27m 40s) | [production] | 
            
  | 03:19 | <LocalisationUpdate> | completed (1.23wmf19) at 2014-03-28 03:19:47+00:00 | [production] | 
            
  | 03:08 | <RoanKattouw> | Rerunning LocalisationUpdate in an attempt to figure out what's going on | [production] |