| 
      
        2013-12-06
      
      §
     | 
  
    
  | 10:16 | 
  <hashar> | 
  synchronized php-1.23wmf5/extensions/ProofreadPage  'ProofreadPage bug fixes {{gerrit|99522}}' | 
  [production] | 
            
  | 10:02 | 
  <apergos> | 
  shot and restarted gmond processes on hooft | 
  [production] | 
            
  | 04:27 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  'repool es1001 after upgrade, max_connections lowered during warm up' | 
  [production] | 
            
  | 03:33 | 
  <reedy> | 
  synchronized multiversion/MWMultiVersion.php | 
  [production] | 
            
  | 02:51 | 
  <springle> | 
  set es1004 max_connections back to default (should have been reset previously, hence dberror log noise) | 
  [production] | 
            
  | 02:46 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  'depool es1001 for upgrade' | 
  [production] | 
            
  | 02:29 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  'direct vslow query group to LB=0 snapshot slaves' | 
  [production] | 
            
  | 02:27 | 
  <tstarling> | 
  synchronized wmf-config/trusted-xff.cdb | 
  [production] | 
            
  | 02:22 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Fri Dec  6 02:22:06 UTC 2013 | 
  [production] | 
            
  | 02:15 | 
  <LocalisationUpdate> | 
  completed (1.23wmf6) at Fri Dec  6 02:15:21 UTC 2013 | 
  [production] | 
            
  | 02:02 | 
  <LocalisationUpdate> | 
  completed (1.23wmf5) at Fri Dec  6 02:02:14 UTC 2013 | 
  [production] | 
            
  | 00:27 | 
  <mwalker> | 
  updating payments from 2afd91f91622abb369e05174d1dd244a2ff3adbd for amazon issues | 
  [production] | 
            
  | 00:18 | 
  <maxsem> | 
  synchronized php-1.23wmf6/extensions/DataValues  'touch' | 
  [production] | 
            
  | 00:04 | 
  <maxsem> | 
  synchronized php-1.23wmf6/extensions/MobileFrontend  'https://gerrit.wikimedia.org/r/99551' | 
  [production] | 
            
  
    | 
      
        2013-12-05
      
      §
     | 
  
    
  | 23:37 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Thu Dec  5 23:37:18 UTC 2013 | 
  [production] | 
            
  | 23:22 | 
  <aaron> | 
  synchronized wmf-config/jobqueue-eqiad.php  'Tweaked $wgJobQueueAggregator to use redis job servers and have failover' | 
  [production] | 
            
  | 23:22 | 
  <aaron> | 
  synchronized wmf-config/jobqueue-pmtpa.php  'Tweaked $wgJobQueueAggregator to use redis job servers and have failover' | 
  [production] | 
            
  | 22:55 | 
  <LocalisationUpdate> | 
  completed (1.23wmf6) at Thu Dec  5 22:54:59 UTC 2013 | 
  [production] | 
            
  | 22:21 | 
  <LocalisationUpdate> | 
  completed (1.23wmf5) at Thu Dec  5 22:21:52 UTC 2013 | 
  [production] | 
            
  | 21:22 | 
  <catrope> | 
  synchronized php-1.23wmf5/extensions/VisualEditor/modules/oojs-ui/oojs-ui.js  'touch' | 
  [production] | 
            
  | 21:19 | 
  <ottomata> | 
  stopping puppet on cp1046 to troubleshoot some ganglia stuff | 
  [production] | 
            
  | 20:58 | 
  <reedy> | 
  synchronized docroot and w | 
  [production] | 
            
  | 20:45 | 
  <apergos> | 
  over the last while, restarted swift-object-server on ms-be100* except for 1008, left that for poking at | 
  [production] | 
            
  | 20:44 | 
  <reedy> | 
  updated /a/common to {{Gerrit|Ifae924950}}: Remofve 1.22wmf15 through 1.22wmf19 | 
  [production] | 
            
  | 20:44 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: phase1 wikis to 1.23wmf6 | 
  [production] | 
            
  | 20:39 | 
  <reedy> | 
  synchronized php-1.23wmf6 | 
  [production] | 
            
  | 20:18 | 
  <reedy> | 
  synchronized docroot and w | 
  [production] | 
            
  | 20:16 | 
  <reedy> | 
  updated /a/common to {{Gerrit|Ida4a0d980}}: Everything else to 1.23wmf5 | 
  [production] | 
            
  | 20:09 | 
  <ori-l> | 
  increase in load coincided with sync of wmf6 to apaches and subsided on roll-back, but wmf6 was not enabled anywhere at the time of syncing | 
  [production] | 
            
  | 20:06 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: everything else to 1.23wmf5 | 
  [production] | 
            
  | 20:04 | 
  <reedy> | 
  updated /a/common to {{Gerrit|Ifda85f2ce}}: Add/update symlinks | 
  [production] | 
            
  | 19:41 | 
  <Coren> | 
  Quick bounce of labstore1001 (kernel tweak) | 
  [production] | 
            
  | 19:38 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki back to 1.23wmf5 for now | 
  [production] | 
            
  | 19:37 | 
  <reedy> | 
  finished scap: testwiki to 1.23wmf6, build l10n cache and rebuild for 1.23wmf5 | 
  [production] | 
            
  | 19:14 | 
  <reedy> | 
  started scap: testwiki to 1.23wmf6, build l10n cache and rebuild for 1.23wmf5 | 
  [production] | 
            
  | 19:08 | 
  <reedy> | 
  updated /a/common to {{Gerrit|I18d8a12b1}}: repool es1002 after upgrade | 
  [production] | 
            
  | 18:56 | 
  <reedy> | 
  synchronized php-1.23wmf6  'Staging' | 
  [production] | 
            
  | 17:38 | 
  <catrope> | 
  synchronized php-1.23wmf5/resources/ | 
  [production] | 
            
  | 17:38 | 
  <catrope> | 
  synchronized php-1.23wmf5/resources/oojs/oojs.js  'I synced you before, now start existing' | 
  [production] | 
            
  | 17:30 | 
  <catrope> | 
  synchronized php-1.23wmf5/extensions/MultimediaViewer  'oojs fixes' | 
  [production] | 
            
  | 17:30 | 
  <catrope> | 
  synchronized php-1.23wmf5/extensions/VisualEditor  'oojs fixes' | 
  [production] | 
            
  | 17:29 | 
  <catrope> | 
  synchronized php-1.23wmf5/resources/Resources.php  'Add oojs' | 
  [production] | 
            
  | 17:29 | 
  <catrope> | 
  synchronized php-1.23wmf5/resources/oojs/oojs.js  'Add oojs' | 
  [production] | 
            
  | 16:54 | 
  <ottomata> | 
  deployed varnishkafka 1.0.0-1 to cp1046,cp3011,cp4011 | 
  [production] | 
            
  | 16:07 | 
  <ottomata> | 
  added varnishkafka 1.0.0-1 to apt | 
  [production] | 
            
  | 15:43 | 
  <ottomata> | 
  added librdkafka 0.8.1-1~precise1 to apt | 
  [production] | 
            
  | 14:41 | 
  <bblack> | 
  mobile esams caches restarted again on new packages w/ new jemalloc, puppet's enabled there and should be stable | 
  [production] | 
            
  | 14:17 | 
  <Coren> | 
  bouncing labstore1001 (kernel upgrade) | 
  [production] | 
            
  | 13:28 | 
  <hashar> | 
  synchronized php-1.23wmf5/extensions/ProofreadPage  'Update Proofreadpage {{gerrit|99042}}' | 
  [production] | 
            
  | 13:25 | 
  <bblack> | 
  cp3012 running test varnish pkg w/ jemalloc 3.4.1 | 
  [production] |