| 2014-05-21
      
      § | 
    
  | 03:42 | <springle> | resume xtrabackup db1049 to db1068, throttled | [production] | 
            
  | 03:13 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Wed May 21 03:12:31 UTC 2014 (duration 12m 30s) | [production] | 
            
  | 03:13 | <mutante> | merging Change-Id: I2827d1ef347 and starting icinga fixed it | [production] | 
            
  | 03:09 | <springle> | killed db1068 xtrabackup, saturating db1064 network | [production] | 
            
  | 03:01 | <mutante> | icinga broken on neon due to missing servicegroup 'analytics_eqiad' | [production] | 
            
  | 02:30 | <LocalisationUpdate> | completed (1.24wmf5) at 2014-05-21 02:29:03+00:00 | [production] | 
            
  | 02:19 | <springle> | xtrabackup clone db1049 to db1068 | [production] | 
            
  | 02:18 | <springle> | synchronized wmf-config/db-eqiad.php  's4 reduce db1049 load while cloning' | [production] | 
            
  | 02:16 | <LocalisationUpdate> | completed (1.24wmf4) at 2014-05-21 02:15:36+00:00 | [production] | 
            
  | 02:04 | <springle> | synchronized wmf-config/db-eqiad.php  's4 raise db1056 to normal load, depool db1011' | [production] | 
            
  
    | 2014-05-20
      
      § | 
    
  | 23:53 | <maxsem> | synchronized php-1.24wmf4/extensions/MobileFrontend/  'https://gerrit.wikimedia.org/r/#/c/134504/' | [production] | 
            
  | 23:52 | <maxsem> | synchronized php-1.24wmf5/extensions/MobileFrontend/  'https://gerrit.wikimedia.org/r/#/c/134504/' | [production] | 
            
  | 23:40 | <maxsem> | synchronized php-1.24wmf5/extensions/MobileFrontend/  'https://gerrit.wikimedia.org/r/134517' | [production] | 
            
  | 23:39 | <maxsem> | synchronized php-1.24wmf4/extensions/MobileFrontend/  'https://gerrit.wikimedia.org/r/134517' | [production] | 
            
  | 23:14 | <maxsem> | synchronized php-1.24wmf4/extensions/MobileFrontend  'https://gerrit.wikimedia.org/r/#/c/134405/' | [production] | 
            
  | 23:12 | <maxsem> | synchronized php-1.24wmf5/extensions/MobileFrontend  'https://gerrit.wikimedia.org/r/#/c/134405/' | [production] | 
            
  | 22:00 | <aaron> | synchronized wmf-config/filebackend.php  '69201b4caf703ef1ab52b38be29c80b4e939fdc2 - no-op' | [production] | 
            
  | 21:57 | <aaron> | synchronized wmf-config/filebackend.php  'Removed old tampa config' | [production] | 
            
  | 21:41 | <bsitu> | synchronized wmf-config/InitialiseSettings.php  'Re-enable flow on mediawiki:Talk:Design' | [production] | 
            
  | 21:39 | <bsitu> | updated /a/common to {{Gerrit|I037cd0a42}}: Re-enable flow on Talk:Design ( Removed LQT code ) | [production] | 
            
  | 21:30 | <bsitu> | synchronized wmf-config/InitialiseSettings.php  'Disable flow on mediawiki:Talk:Design' | [production] | 
            
  | 21:28 | <bsitu> | updated /a/common to {{Gerrit|Idde23abd3}}: Undo "enable flow on Talk:Design" | [production] | 
            
  | 21:11 | <bsitu> | synchronized wmf-config/InitialiseSettings.php  'Enable Flow on 3 mediawiki talk pages' | [production] | 
            
  | 21:08 | <bsitu> | updated /a/common to {{Gerrit|I549967ca2}}: Group1 wikis to 1.24wmf5 | [production] | 
            
  | 20:36 | <Krinkle> | Reload zuul to deploy I80496db747a8668be | [production] | 
            
  | 19:14 | <hoo> | fixed Wikidata for php-1.24wmf5 on mw1138 by manually removing it and then running sync-common | [production] | 
            
  | 19:09 | <bd808> | ran sync-common on mw1138 | [production] | 
            
  | 19:02 | <bd808> | Updated scap to 7b6fc47 | [production] | 
            
  | 18:44 | <aude> | synchronized php-1.24wmf5/extensions/Wikidata  'Fix jquery error tooltip issue' | [production] | 
            
  | 18:33 | <bd808|deploy> | Gave up on updating scap with trebuchet | [production] | 
            
  | 18:32 | <bd808> | rebuilt wikiversions.cdb and synchronized wikiversions files: group1 to 1.24wmf5 | [production] | 
            
  | 18:22 | <RobH> | restarting salt minions on mw servers | [production] | 
            
  | 18:12 | <bd808|deploy> | `git deploy sync` for scap ended with "0/230 minions completed fetch" | [production] | 
            
  | 15:15 | <andrewbogott> | running salt "ms-be*" cmd.run "kill $(ps aux | grep 'find / -user' | awk '{print $2}')" to kill runaway 'finds' on swifts | [production] | 
            
  | 14:47 | <andrewbogott> | running 'find' commands on many hosts to chown files for users with new UIDs. | [production] | 
            
  | 13:20 | <Krinkle> | git-deploy: Deploying integration/slave-scripts I4a4e2a4c90fb6 | [production] | 
            
  | 03:38 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Tue May 20 03:37:32 UTC 2014 (duration 37m 31s) | [production] | 
            
  | 02:34 | <LocalisationUpdate> | completed (1.24wmf5) at 2014-05-20 02:33:31+00:00 | [production] | 
            
  | 02:20 | <LocalisationUpdate> | completed (1.24wmf4) at 2014-05-20 02:19:33+00:00 | [production] | 
            
  
    | 2014-05-19
      
      § | 
    
  | 23:37 | <catrope> | synchronized wmf-config/InitialiseSettings.php  'autopatrolled group on dewikivoyage' | [production] | 
            
  | 23:26 | <catrope> | synchronized php-1.24wmf5/extensions/VisualEditor | [production] | 
            
  | 23:25 | <catrope> | synchronized php-1.24wmf5/extensions/MobileFrontend | [production] | 
            
  | 23:24 | <catrope> | synchronized php-1.24wmf5/extensions/Wikidata | [production] | 
            
  | 22:19 | <andrewbogott> | Coren restarted opendj and I restarted pdns on virt1000.  Opendj was refusing connections for unclear reasons | [production] | 
            
  | 21:34 | <Krinkle> | Running deleteEqualMessages.php on urwiki (bug 43917) | [production] | 
            
  | 21:33 | <Krinkle> | Running deleteEqualMessages.php on commonswiki (bug 43917) | [production] | 
            
  | 20:53 | <superm401> | sync-l10nupdate-1 1.24wmf4 had one error on mw1218 | [production] | 
            
  | 20:27 | <gwicke> | updated Parsoid to 3ac048d7c4b | [production] | 
            
  | 20:05 | <csteipp> | fix deployed for bug 65501 | [production] | 
            
  | 20:00 | <Krinkle> | Running deleteEqualMessages.php on fowiki (bug 43917) | [production] |