| 
      
        2016-07-11
      
      §
     | 
  
    
  | 14:05 | 
  <moritzm> | 
  rebooting achernar for kernel update | 
  [production] | 
            
  | 13:31 | 
  <moritzm> | 
  rebooting acamar for kernel update | 
  [production] | 
            
  | 13:07 | 
  <ema> | 
  upgrading esams cache_maps to varnish 4.1.3-1wm1 | 
  [production] | 
            
  | 12:51 | 
  <gehel> | 
  upgrading nodejs to 4.4.6 on maps2.* servers | 
  [production] | 
            
  | 12:24 | 
  <ema> | 
  upgrading ulsfo cache_maps to varnish 4.1.3-1wm1 | 
  [production] | 
            
  | 12:14 | 
  <elukey> | 
  restarted hhvm on mw1261 | 
  [production] | 
            
  | 12:03 | 
  <ema> | 
  upgrading codfw cache_maps to varnish 4.1.3-1wm1 | 
  [production] | 
            
  | 11:30 | 
  <ema> | 
  upgrading eqiad cache_maps to varnish 4.1.3-1wm1 | 
  [production] | 
            
  | 11:19 | 
  <moritzm> | 
  installing hhvm updates on canary app servers | 
  [production] | 
            
  | 10:25 | 
  <hashar> | 
  CI: upgraded Chromium from v49 to v51 (v50 caused qunit jobs to fail / timeout randomly) T136188 | 
  [production] | 
            
  | 10:00 | 
  <godog> | 
  swift codfw-prod: ms-be202[567] weight 2000 | 
  [production] | 
            
  | 09:47 | 
  <moritzm> | 
  installing GCC stable updates on trusty systems (also provides some runtime libs in addition to GCC itself) | 
  [production] | 
            
  | 09:40 | 
  <ema> | 
  upgrading cp1046 to varnish 4.1.3-1wm1 | 
  [production] | 
            
  | 07:29 | 
  <mobrovac> | 
  change-prop deploying 2b699a6 | 
  [production] | 
            
  | 07:26 | 
  <mobrovac> | 
  graphoid deploying 375d31fd | 
  [production] | 
            
  | 07:24 | 
  <mobrovac> | 
  mathoid deploying 669cfc0 | 
  [production] | 
            
  | 07:19 | 
  <mobrovac> | 
  cxserver deploying fd8eca47e | 
  [production] | 
            
  | 07:15 | 
  <mobrovac> | 
  citoid deploying 274c0231d | 
  [production] | 
            
  | 07:13 | 
  <mobrovac> | 
  mobileapps deploying 6e409f46 | 
  [production] | 
            
  | 06:34 | 
  <_joe_> | 
  restarted hhvm on mw1168 | 
  [production] | 
            
  | 06:13 | 
  <moritzm> | 
  restarted saltmaster on neodymium | 
  [production] | 
            
  | 02:26 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Mon Jul 11 02:26:55 UTC 2016 (duration 5m 41s) | 
  [production] | 
            
  | 02:21 | 
  <mwdeploy@tin> | 
  scap sync-l10n completed (1.28.0-wmf.9) (duration: 08m 41s) | 
  [production] | 
            
  
    | 
      
        2016-07-09
      
      §
     | 
  
    
  | 19:54 | 
  <bd808> | 
  restarted logstash on logstash1003 for de-dot plugin update (T136001) | 
  [production] | 
            
  | 19:52 | 
  <bd808> | 
  restarted logstash on logstash1002 for de-dot plugin update (T136001) | 
  [production] | 
            
  | 19:50 | 
  <bd808> | 
  restarted logstash on logstash1001 for de-dot plugin update (T136001) | 
  [production] | 
            
  | 19:46 | 
  <bd808> | 
  Updated logstash/plugins to 18b3f1f (Fix de_dot to process keys with falsey values) | 
  [production] | 
            
  | 02:29 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Sat Jul  9 02:29:53 UTC 2016 (duration 6m 7s) | 
  [production] | 
            
  | 02:23 | 
  <mwdeploy@tin> | 
  scap sync-l10n completed (1.28.0-wmf.9) (duration: 08m 21s) | 
  [production] | 
            
  | 00:28 | 
  <mutante> | 
  analytics1027 - icinga said puppet fail, just ran it, recovery, same on neon.. something kafka graphite checks  | 
  [production] | 
            
  | 00:06 | 
  <mutante> | 
  gerrit is back | 
  [production] | 
            
  | 00:05 | 
  <ostriches> | 
  gerrit: disabled puppet for a minute so I can unbreak gerrit so I can fix gerrit in puppet. | 
  [production] | 
            
  
    | 
      
        2016-07-08
      
      §
     | 
  
    
  | 23:57 | 
  <mutante> | 
  behold, gerrit might restart now for config change | 
  [production] | 
            
  | 23:52 | 
  <Amir1> | 
  manually restarting uwsgi-ores in scb1001  | 
  [production] | 
            
  | 23:48 | 
  <krinkle@tin> | 
  Synchronized docroot/foundation/: Remove unused docroot/foundation/index.html (duration: 00m 30s) | 
  [production] | 
            
  | 23:32 | 
  <Amir1> | 
  manually restarting uwsgi-ores on scb1002 | 
  [production] | 
            
  | 22:59 | 
  <urandom> | 
  Restarting restbase1015-b.eqiad.wmnet to cancel running streams : T139362 | 
  [production] | 
            
  | 22:26 | 
  <eileen> | 
  from bdf2afd417b70332c9542fd3ee4f14cb4e6f93cc to f477a42014dd1e6759849b347d5f73d710954d0b | 
  [production] | 
            
  | 22:13 | 
  <urandom> | 
  Restarting restbase1014-b.eqiad.wmnet to cancel running streams : T139362 | 
  [production] | 
            
  | 22:02 | 
  <legoktm@tin> | 
  Synchronized php-1.28.0-wmf.9/extensions/CentralAuth/: Fix job serializing (and status display on Special:GlobalRenameProgress) - T137973 (duration: 00m 32s) | 
  [production] | 
            
  | 21:59 | 
  <reedy@tin> | 
  Synchronized wmf-config/extension-list: Fix RestBaseUpdateJobs in extension-list (duration: 00m 36s) | 
  [production] | 
            
  | 21:51 | 
  <reedy@tin> | 
  Synchronized wmf-config/CommonSettings.php: Use Canonical entry point for RestBaseUpdateJobs (duration: 00m 33s) | 
  [production] | 
            
  | 21:50 | 
  <reedy@tin> | 
  Synchronized multiversion/MWWikiversions.php: Remove some else if spaces (duration: 00m 46s) | 
  [production] | 
            
  | 21:48 | 
  <urandom> | 
  Restarting restbase1009-b.eqiad.wmnet to cancel running streams : T139362 | 
  [production] | 
            
  | 21:31 | 
  <elukey> | 
  mw1146 powercycled (memory pressure, no ssh/root login) | 
  [production] | 
            
  | 21:17 | 
  <urandom> | 
  Restarting restbase1015-a.eqiad.wmnet to cancel running streams : T139362 | 
  [production] | 
            
  | 21:10 | 
  <urandom> | 
  Restarting restbase1014-a.eqiad.wmnet to cancel running streams : T139362 | 
  [production] | 
            
  | 21:04 | 
  <urandom> | 
  Restarting restbase1009-a.eqiad.wmnet to cancel running streams : T139362 | 
  [production] |