| 
      
        2014-11-07
      
      §
     | 
  
    
  | 20:53 | 
  <ebernhardson> | 
  Synchronized php-1.25wmf7/extensions/Flow: Bump flow submodule for bug 71858 (duration: 00m 08s) | 
  [production] | 
            
  | 20:33 | 
  <ori> | 
  Synchronized php-1.25wmf6/includes/WebResponse.php: I569b2ebbc: Add WebResponse::getHeader() (duration: 00m 09s) | 
  [production] | 
            
  | 20:13 | 
  <ori> | 
  Synchronized php-1.25wmf7/includes/WebResponse.php: I569b2ebbc: Add WebResponse::getHeader() (duration: 00m 07s) | 
  [production] | 
            
  | 20:03 | 
  <YuviPanda> | 
  restarted gitblit on antimony | 
  [production] | 
            
  | 19:36 | 
  <YuviPanda> | 
  upgraded php5-fss to 1.0-2 on virt1000 to prevent cronspam | 
  [production] | 
            
  | 19:36 | 
  <demon> | 
  Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 04s) | 
  [production] | 
            
  | 16:27 | 
  <godog> | 
  shut db1017 briefly for cmjohnson to look | 
  [production] | 
            
  | 14:46 | 
  <_joe_> | 
  installing hhvm package built with full debug symbols on mw1114 | 
  [production] | 
            
  | 09:00 | 
  <oblivian> | 
  Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 07s) | 
  [production] | 
            
  | 08:00 | 
  <_joe_> | 
  powercycled mw1169, console unresponsive, not responding to pings | 
  [production] | 
            
  | 07:52 | 
  <_joe_> | 
  killed the master apache process on mw1191, stuck in a futex wait, restarted apache | 
  [production] | 
            
  | 07:44 | 
  <_joe_> | 
  upgrading the hhvm appservers to the new package version, it seems stable enough | 
  [production] | 
            
  | 01:01 | 
  <reedy> | 
  Synchronized php-1.25wmf7/extensions/Flow/: (no message) (duration: 00m 16s) | 
  [production] | 
            
  | 00:49 | 
  <reedy> | 
  Synchronized php-1.25wmf6/includes/api/: (no message) (duration: 00m 14s) | 
  [production] | 
            
  | 00:36 | 
  <reedy> | 
  Synchronized php-1.25wmf7/extensions/MobileFrontend/: (no message) (duration: 00m 16s) | 
  [production] | 
            
  | 00:27 | 
  <reedy> | 
  Synchronized php-1.25wmf7/extensions/VisualEditor/: (no message) (duration: 00m 15s) | 
  [production] | 
            
  | 00:26 | 
  <reedy> | 
  Synchronized php-1.25wmf6/extensions/GeoData: (no message) (duration: 00m 14s) | 
  [production] | 
            
  | 00:03 | 
  <Reedy> | 
  running foreachwikiindblist wikidataclient.dblist extensions/Wikidata/extensions/Wikibase/lib/maintenance/populateSitesTable.php --strip-protocols | 
  [production] | 
            
  | 00:00 | 
  <reedy> | 
  Synchronized wmf-config/InitialiseSettings.php: touch (duration: 00m 14s) | 
  [production] | 
            
  | 00:00 | 
  <reedy> | 
  Synchronized langlist: mai (duration: 00m 14s) | 
  [production] | 
            
  
    | 
      
        2014-11-06
      
      §
     | 
  
    
  | 23:26 | 
  <bd808> | 
  deleted corrupt mediawki/core clone in workspace/mwext-MobileFrontend-qunit-mobile on gallium | 
  [production] | 
            
  | 23:24 | 
  <bd808> | 
  Killed 3 hung /usr/local/bin/logstash_optimize_index.sh processes on logstash1002 | 
  [production] | 
            
  | 23:22 | 
  <bd808> | 
  restarted logstash on logstash1002 | 
  [production] | 
            
  | 23:21 | 
  <bd808> | 
  restarted logstash on logstash1003 | 
  [production] | 
            
  | 23:02 | 
  <bd808> | 
  restarted logstash on logstash1001 for the usual reason (no events making it to elasticsearch) | 
  [production] | 
            
  | 22:19 | 
  <subbu> | 
  updated parsoid to d23d2be6 (+ a hotfix to the production localsettings config file) | 
  [production] | 
            
  | 21:26 | 
  <ottomata> | 
  added Range header field to varnishkafka webrequest logs | 
  [production] | 
            
  | 19:45 | 
  <andrewbogott> | 
  restarted ntp on labstore1001 | 
  [production] | 
            
  | 19:33 | 
  <manybubbles> | 
  manybubbles is done with SWAT | 
  [production] | 
            
  | 16:11 | 
  <manybubbles> | 
  Synchronized php-1.25wmf7/extensions/MultimediaViewer/: SWAT revert layout changes (duration: 00m 06s) | 
  [production] | 
            
  | 16:02 | 
  <manybubbles> | 
  Synchronized wmf-config/: SWAT deploy some beta configs.  Should be noop. (duration: 00m 04s) | 
  [production] | 
            
  | 15:49 | 
  <_joe_> | 
  load-testing hhvm, in particular the servers with the new package | 
  [production] | 
            
  | 15:43 | 
  <_joe_> | 
  upgrading  mw1031,mw1032 to the new package, no crashes seeen since reinstall | 
  [production] | 
            
  | 15:24 | 
  <manybubbles> | 
  finished with performance testing for cirrus - new servers look like way way more than enough power | 
  [production] | 
            
  | 15:01 | 
  <manybubbles> | 
  dewiki is fine.  trying enwiki. | 
  [production] | 
            
  | 14:57 | 
  <manybubbles> | 
  performance test for zhwiki was good.  trying dewiki | 
  [production] | 
            
  | 14:55 | 
  <manybubbles> | 
  running performance test for Cirrus taking zhwiki | 
  [production] | 
            
  | 12:09 | 
  <akosiaris> | 
  Depool wtp1001, wtp1003-1006 for trusty upgrade | 
  [production] | 
            
  | 10:07 | 
  <_joe_> | 
  temporary raising weight of mw1018 and 1030 in pybal to load-test them and check for crashes | 
  [production] | 
            
  | 09:53 | 
  <_joe_> | 
  installing the new hhvm package on mw1030 and mw1018 in order to test for stability | 
  [production] | 
            
  | 02:05 | 
  <awight> | 
  CRM: drush vset maintenance_mode 1 | 
  [production] | 
            
  | 01:21 | 
  <Tim> | 
  restarted gmond on mw1018 and mw1031 | 
  [production] | 
            
  | 01:06 | 
  <mutante> | 
  git-sync-upstream on deployment-salt for beta puppetmaster  | 
  [production] |