| 
      
        2014-11-06
      
      §
     | 
  
    
  | 22:19 | 
  <subbu> | 
  updated parsoid to d23d2be6 (+ a hotfix to the production localsettings config file) | 
  [production] | 
            
  | 21:26 | 
  <ottomata> | 
  added Range header field to varnishkafka webrequest logs | 
  [production] | 
            
  | 19:45 | 
  <andrewbogott> | 
  restarted ntp on labstore1001 | 
  [production] | 
            
  | 19:33 | 
  <manybubbles> | 
  manybubbles is done with SWAT | 
  [production] | 
            
  | 16:11 | 
  <manybubbles> | 
  Synchronized php-1.25wmf7/extensions/MultimediaViewer/: SWAT revert layout changes (duration: 00m 06s) | 
  [production] | 
            
  | 16:02 | 
  <manybubbles> | 
  Synchronized wmf-config/: SWAT deploy some beta configs.  Should be noop. (duration: 00m 04s) | 
  [production] | 
            
  | 15:49 | 
  <_joe_> | 
  load-testing hhvm, in particular the servers with the new package | 
  [production] | 
            
  | 15:43 | 
  <_joe_> | 
  upgrading  mw1031,mw1032 to the new package, no crashes seeen since reinstall | 
  [production] | 
            
  | 15:24 | 
  <manybubbles> | 
  finished with performance testing for cirrus - new servers look like way way more than enough power | 
  [production] | 
            
  | 15:01 | 
  <manybubbles> | 
  dewiki is fine.  trying enwiki. | 
  [production] | 
            
  | 14:57 | 
  <manybubbles> | 
  performance test for zhwiki was good.  trying dewiki | 
  [production] | 
            
  | 14:55 | 
  <manybubbles> | 
  running performance test for Cirrus taking zhwiki | 
  [production] | 
            
  | 12:09 | 
  <akosiaris> | 
  Depool wtp1001, wtp1003-1006 for trusty upgrade | 
  [production] | 
            
  | 10:07 | 
  <_joe_> | 
  temporary raising weight of mw1018 and 1030 in pybal to load-test them and check for crashes | 
  [production] | 
            
  | 09:53 | 
  <_joe_> | 
  installing the new hhvm package on mw1030 and mw1018 in order to test for stability | 
  [production] | 
            
  | 02:05 | 
  <awight> | 
  CRM: drush vset maintenance_mode 1 | 
  [production] | 
            
  | 01:21 | 
  <Tim> | 
  restarted gmond on mw1018 and mw1031 | 
  [production] | 
            
  | 01:06 | 
  <mutante> | 
  git-sync-upstream on deployment-salt for beta puppetmaster  | 
  [production] | 
            
  | 00:56 | 
  <awight> | 
  disabling all queue consumers. | 
  [production] | 
            
  | 00:28 | 
  <maxsem> | 
  Synchronized php-1.25wmf6/extensions/MobileFrontend/: (no message) (duration: 00m 04s) | 
  [production] | 
            
  | 00:24 | 
  <maxsem> | 
  Synchronized php-1.25wmf6/extensions/MobileFrontend/: (no message) (duration: 00m 07s) | 
  [production] | 
            
  | 00:20 | 
  <maxsem> | 
  Synchronized php-1.25wmf7/extensions/VisualEditor/: SWAT (duration: 00m 07s) | 
  [production] | 
            
  | 00:18 | 
  <maxsem> | 
  Synchronized php-1.25wmf6/extensions/MobileFrontend/: SWAT (duration: 00m 04s) | 
  [production] | 
            
  | 00:18 | 
  <maxsem> | 
  Synchronized php-1.25wmf6/extensions/Flow/: SWAT (duration: 00m 05s) | 
  [production] | 
            
  | 00:13 | 
  <andrewbogott> | 
  ocg1001 is depressingly tiny and will probably keeping complaining about disk space until it's rebuilt | 
  [production] | 
            
  | 00:09 | 
  <andrewbogott> | 
  cleaned up some log files on ocg1001 and reduced logrotations to 7. | 
  [production] | 
            
  
    | 
      
        2014-11-05
      
      §
     | 
  
    
  | 22:02 | 
  <ejegg> | 
  updated fraud scoring | 
  [production] | 
            
  | 21:24 | 
  <subbu> | 
  redployed parsoid deploy sha 66befe47 (with the right bunyan log level that unbreaks VE) | 
  [production] | 
            
  | 21:13 | 
  <subbu> | 
  deployed parsoid version 978623eb  | 
  [production] | 
            
  | 21:05 | 
  <Reedy> | 
  Running foreachwikiindblist wikidataclient.dblist extensions/Wikidata/extensions/Wikibase/lib/maintenance/populateSitesTable.php --strip-protocols | 
  [production] | 
            
  | 21:02 | 
  <reedy> | 
  Synchronized wmf-config/interwiki.cdb: Updating interwiki cache (duration: 00m 15s) | 
  [production] | 
            
  | 21:00 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: maiwiki | 
  [production] | 
            
  | 21:00 | 
  <reedy> | 
  Synchronized wmf-config/InitialiseSettings.php: maiwiki (duration: 00m 14s) | 
  [production] | 
            
  | 20:59 | 
  <reedy> | 
  Synchronized database lists: maiwiki (duration: 00m 14s) | 
  [production] | 
            
  | 20:57 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: maiwiki | 
  [production] | 
            
  | 20:56 | 
  <reedy> | 
  Synchronized wmf-config/InitialiseSettings.php: maiwiki (duration: 00m 15s) | 
  [production] | 
            
  | 20:55 | 
  <reedy> | 
  Synchronized database lists: maiwiki (duration: 00m 18s) | 
  [production] | 
            
  | 20:49 | 
  <awight> | 
  turning off CiviMail activity record for each TY | 
  [production] | 
            
  | 20:45 | 
  <reedy> | 
  Synchronized wmf-config/: (no message) (duration: 00m 17s) | 
  [production] | 
            
  | 20:39 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.25wmf7 | 
  [production] | 
            
  | 20:37 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.25wmf6 | 
  [production] | 
            
  | 20:34 | 
  <reedy> | 
  Finished scap: testwiki to 1.25wmf7, build l10n cache (duration: 45m 03s) | 
  [production] | 
            
  | 19:49 | 
  <reedy> | 
  Started scap: testwiki to 1.25wmf7, build l10n cache | 
  [production] | 
            
  | 19:47 | 
  <reedy> | 
  Started scap: testwiki to 1.25wmf7, build l10n cache | 
  [production] | 
            
  | 19:46 | 
  <reedy> | 
  Started scap: testwiki to 1.25wmf7, build l10n cache | 
  [production] | 
            
  | 19:17 | 
  <andrewbogott> | 
  removed libvips-dev and libvips-tools from our custom repo for Trusty.  The default packages seem to work fine. | 
  [production] | 
            
  | 18:30 | 
  <andrewbogott> | 
  restarting icinga on neon | 
  [production] | 
            
  | 18:10 | 
  <awight> | 
  disabled TY job | 
  [production] | 
            
  | 18:02 | 
  <^d> | 
  elastic1022 unbanned from allocation since it has a network cable again | 
  [production] | 
            
  | 17:14 | 
  <demon> | 
  Synchronized wmf-config/PoolCounterSettings-eqiad.php: (no message) (duration: 00m 06s) | 
  [production] |