| 2014-02-04
      
      § | 
    
  | 18:36 | <ottomata> | disabled puppet on cp3019 again, trying batch.num.messages there (just want these flappy alerts to be quiet!) | [production] | 
            
  | 17:43 | <demon> | synchronized php-1.23wmf11/extensions/LiquidThreads/pages/TalkpageView.php | [production] | 
            
  | 17:43 | <demon> | synchronized php-1.23wmf12/extensions/LiquidThreads/pages/TalkpageView.php | [production] | 
            
  | 17:32 | <ottomata> | reenabling puppet on cp3021, this reverts the batch.num.messages varnishkafka.conf change, let's see if cp3021 starts dropping again tomorrow | [production] | 
            
  | 17:29 | <ottomata> | reenabling puppet on cp3019 | [production] | 
            
  | 17:05 | <hashar> | Jenkins: updated bin/multigit.sh script to point to zuul.eqiad.wmnet instead of non working integration.wikimedia.org | [production] | 
            
  | 13:51 | <reedy> | updated /a/common to {{Gerrit|I8a1341149}}: increase db1059 load (96G ram compared to 64G for s4 siblings) | [production] | 
            
  | 11:17 | <springle> | synchronized wmf-config/db-eqiad.php  'db1059 LB increase' | [production] | 
            
  | 11:08 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1027 in s3, warm up' | [production] | 
            
  | 03:41 | <LocalisationUpdate> | ResourceLoader cache refresh completed at 2014-02-04 03:41:31+00:00 | [production] | 
            
  | 02:46 | <LocalisationUpdate> | completed (1.23wmf12) at 2014-02-04 02:46:12+00:00 | [production] | 
            
  | 02:25 | <LocalisationUpdate> | completed (1.23wmf11) at 2014-02-04 02:25:34+00:00 | [production] | 
            
  | 02:01 | <krinkle> | synchronized php-1.23wmf12/extensions/VisualEditor/modules/ve-mw/ui/inspectors/ve.ui.MWGalleryInspector.js  'touch for scap i18n race condition breakage' | [production] | 
            
  | 01:58 | <krinkle> | synchronized php-1.23wmf11/extensions/VisualEditor/modules/ve-mw/ui/inspectors/ve.ui.MWGalleryInspector.js  'touch for scap i18n race condition breakage' | [production] | 
            
  | 01:47 | <springle> | synchronized wmf-config/db-eqiad.php  'in s1: db1049 to full steam, repool db1050, depool db1043 for schema changes' | [production] | 
            
  | 01:45 | <krinkle> | synchronized php-1.23wmf11/extensions/VisualEditor/modules/ve-mw/ui/inspectors/ve.ui.MWExtensionInspector.js  'touch for scap i18n race condition breakage' | [production] | 
            
  | 01:40 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1059, warm up' | [production] | 
            
  | 01:03 | <mwalker> | finished scap: Krinkle, Mwalker and superm401 LD super scap (duration: 26m 24s) | [production] | 
            
  | 00:41 | <mwalker> | started scap: Krinkle, Mwalker and superm401 LD super scap | [production] | 
            
  | 00:03 | <jgonera> | finished scap (duration: 23m 55s) | [production] | 
            
  
    | 2014-02-03
      
      § | 
    
  | 23:44 | <jgonera> | started scap | [production] | 
            
  | 23:37 | <jgonera> | synchronized php-1.23wmf11/ | [production] | 
            
  | 23:32 | <aaron> | synchronized php-1.23wmf12/includes/filebackend/FileOpBatch.php  'aa3cb6330324ff18ce35fbb68f67b42035de3496' | [production] | 
            
  | 23:29 | <maxsem> | synchronized wmf-config/mobile.php  'https://gerrit.wikimedia.org/r/110087' | [production] | 
            
  | 23:27 | <aaron> | synchronized php-1.23wmf12/includes/filebackend/SwiftFileBackend.php  '1eb5600fe6518071b4c3d2eaa980f70143451be0' | [production] | 
            
  | 23:09 | <springle> | xtrabackup clone db1035 to db1027 | [production] | 
            
  | 22:41 | <springle> | xtrabackup clone db1042 to db1059 | [production] | 
            
  | 22:26 | <bd808> | Forced puppet run on logstash1001 so I could watch the logs as it restarted | [production] | 
            
  | 22:24 | <bd808> | puppetd --enable on logstash1001 | [production] | 
            
  | 21:34 | <bsitu> | synchronized wmf-config/InitialiseSettings.php  'Enable Flow on two enwiki WikiProject pages' | [production] | 
            
  | 21:29 | <ottomata> | stopping puppet on cp3021 to test troubleshoot varnishkafka delivery errors with a different change than cp3019 | [production] | 
            
  | 21:27 | <bsitu> | that was not our change, it's previous commit. Ours is "Enable Flow on two enwiki WikiProject pages" | [production] | 
            
  | 21:25 | <bsitu> | updated /a/common to {{Gerrit|I365611ca3}}: Bump wgCacheEpoch for Wikidata | [production] | 
            
  | 21:06 | <ottomata> | stopping puppet on cp3019 to investigate varnishkafka errors | [production] | 
            
  | 20:46 | <bd808> | Disabled puppet on logstash1001 to work around bug 60772 with local config hack | [production] | 
            
  | 20:41 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 20:30 | <reedy> | synchronized php-1.23wmf12/extensions/Wikidata/ | [production] | 
            
  | 20:28 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1049, warm up' | [production] | 
            
  | 20:15 | <awight> | synchronized php-1.23wmf11/extensions/EducationProgram | [production] | 
            
  | 20:11 | <Reedy> | mw1163 seems to be down. No ping/ssh response | [production] | 
            
  | 20:06 | <awight> | synchronized php-1.23wmf12/extensions/EducationProgram | [production] | 
            
  | 20:06 | <awight> | synchronized php-1.23wmf12/extensions/EducationProgram | [production] | 
            
  | 19:51 | <Reedy> | fixed user_last_timestamp column on wikis where it was not "varbinary(14) NULL default NULL" - bug 49196 | [production] | 
            
  | 19:38 | <awight> | crm updated from 95665543297b4f6016e443bcd2194fa5e0f0c96f to ec1e1fc75193ac8c0e6237fc0b80f58b3f8d55fc | [production] | 
            
  | 19:34 | <manybubbles1> | removed elastic1007 from the elasticsearch cluster because it crashed again unexpectedly | [production] | 
            
  | 19:33 | <gwicke> | deployed parsoid deploy 855eeb06 with source 4d5d9a0a | [production] | 
            
  | 19:11 | <Reedy> | Added org_last_active_date column to ep_orgs on all wikis running EducationProgram - bug 60775 | [production] | 
            
  | 06:56 | <springle> | synchronized wmf-config/db-eqiad.php  'depool db1049 for schema changes' | [production] | 
            
  | 05:44 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1051, warm up' | [production] | 
            
  | 03:40 | <springle> | synchronized wmf-config/db-eqiad.php  'depool db1051 for schema changes' | [production] |