| 2020-04-08
      
      § | 
    
  | 21:20 | <jforrester@deploy1001> | Synchronized php-1.35.0-wmf.27/extensions/TemplateData/includes/TemplateDataHooks.php: Restore call to OutputPage::setupOOUI() (duration: 01m 07s) | [production] | 
            
  | 21:19 | <jforrester@deploy1001> | Synchronized php-1.35.0-wmf.26/extensions/TemplateData/includes/TemplateDataHooks.php: Restore call to OutputPage::setupOOUI() (duration: 01m 09s) | [production] | 
            
  | 20:09 | <otto@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-main' for release 'canary' . | [production] | 
            
  | 20:09 | <otto@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-main' for release 'production' . | [production] | 
            
  | 20:06 | <otto@deploy1001> | helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-main' for release 'canary' . | [production] | 
            
  | 20:06 | <otto@deploy1001> | helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-main' for release 'production' . | [production] | 
            
  | 20:04 | <otto@deploy1001> | helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-main' for release 'canary' . | [production] | 
            
  | 20:04 | <otto@deploy1001> | helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-main' for release 'production' . | [production] | 
            
  | 19:51 | <gehel> | restart wdqs-updater after deployment | [production] | 
            
  | 19:49 | <mstyles@deploy1001> | Finished deploy [wdqs/wdqs@c2995eb]: WDQS version 0.3.21 (duration: 14m 37s) | [production] | 
            
  | 19:44 | <dpifke@deploy1001> | Finished deploy [performance/navtiming@4acb04d]: Deploy new navtiming with First Input Delay metric https://phabricator.wikimedia.org/T238091 (duration: 00m 05s) | [production] | 
            
  | 19:44 | <dpifke@deploy1001> | Started deploy [performance/navtiming@4acb04d]: Deploy new navtiming with First Input Delay metric https://phabricator.wikimedia.org/T238091 | [production] | 
            
  | 19:35 | <mstyles@deploy1001> | Started deploy [wdqs/wdqs@c2995eb]: WDQS version 0.3.21 | [production] | 
            
  | 19:08 | <jhuneidi@deploy1001> | Synchronized php: group1 wikis to 1.35.0-wmf.27  refs T247774 (duration: 01m 06s) | [production] | 
            
  | 19:07 | <jhuneidi@deploy1001> | rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.27  refs T247774 | [production] | 
            
  | 19:02 | <longma> | deploying 1.35.0-wmf.27 to group1 | [production] | 
            
  | 18:37 | <jforrester@deploy1001> | Synchronized php-1.35.0-wmf.27/skins/Vector: T248761: Revert moving indicators in DOM (duration: 01m 07s) | [production] | 
            
  | 18:17 | <reedy@deploy1001> | Synchronized php-1.35.0-wmf.27/extensions/TemplateData/includes/TemplateDataHooks.php: T236809 (duration: 01m 06s) | [production] | 
            
  | 18:16 | <reedy@deploy1001> | Synchronized php-1.35.0-wmf.26/extensions/TemplateData/includes/TemplateDataHooks.php: T236809 (duration: 01m 10s) | [production] | 
            
  | 17:31 | <hnowlan@deploy1001> | helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . | [production] | 
            
  | 17:23 | <hnowlan@deploy1001> | helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . | [production] | 
            
  | 17:13 | <dzahn@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 17:13 | <dzahn@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 16:16 | <ema> | cache_upload: rolling varnish-fe restarts to bump transient storage limit T185968 | [production] | 
            
  | 15:21 | <dzahn@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 15:19 | <dzahn@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 15:11 | <ema> | cp3051: param.set shortlived=0 to try ease pressure on transient memory | [production] | 
            
  | 14:23 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Repool db1112 after schema change', diff saved to https://phabricator.wikimedia.org/P10947 and previous config saved to /var/cache/conftool/dbconfig/20200408-142341-marostegui.json | [production] | 
            
  | 14:14 | <jeh@deploy1001> | Finished deploy [horizon/deploy@0d18f67]: update horizon submodule to enable server groups (duration: 03m 30s) | [production] | 
            
  | 14:10 | <jeh@deploy1001> | Started deploy [horizon/deploy@0d18f67]: update horizon submodule to enable server groups | [production] | 
            
  | 13:40 | <mutante> | stopped and masked zuul-merger service on contint2001 via puppet (T224591) | [production] | 
            
  | 13:30 | <ema> | cp3050: stop vhtcpd, start purged T249583 | [production] | 
            
  | 13:22 | <vgutierrez> | enable inbound TLSv1.3 in text@ulsfo - T170567 | [production] | 
            
  | 13:05 | <ema> | purged 0.1 uploaded to buster-wikimedia T249583 | [production] | 
            
  | 12:31 | <tgr@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: re-sync (duration: 01m 07s) | [production] | 
            
  | 12:29 | <tgr@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:585219|Enable GrowthExperiments suggested edits on uk, hu, hy, eu wikipedias (T247308)]] (duration: 01m 08s) | [production] | 
            
  | 12:17 | <tgr@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:584135|Enable GrowthExperiments welcome survey on Ukrainian, Hungarian, Armenian Wikipedias (T238295) (duration: 01m 08s) | [production] | 
            
  | 12:09 | <tgr@deploy1001> | Synchronized wmf-config/: SWAT: [[gerrit:584183|Enable GrowthExperiments on French Wiktionary (T235964)]] (duration: 01m 06s) | [production] | 
            
  | 11:56 | <tgr@deploy1001> | Synchronized dblists/: SWAT: [[gerrit:584183|Enable GrowthExperiments on French Wiktionary (T235964)]] (duration: 01m 03s) | [production] | 
            
  | 11:48 | <mutante> | logstash1009 - restarted logstash | [production] | 
            
  | 11:43 | <tgr@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:585766|Enable WikibaseQualityConstraints on test commons (T248117)]] (duration: 01m 05s) | [production] | 
            
  | 11:43 | <marostegui> | Deploy schema change on db1112, this will generate lag on labs s3 | [production] | 
            
  | 11:43 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Depool db1112 for schema change', diff saved to https://phabricator.wikimedia.org/P10942 and previous config saved to /var/cache/conftool/dbconfig/20200408-114315-marostegui.json | [production] | 
            
  | 11:39 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Repool db1078 after schema change', diff saved to https://phabricator.wikimedia.org/P10941 and previous config saved to /var/cache/conftool/dbconfig/20200408-113901-marostegui.json | [production] | 
            
  | 11:29 | <tgr@deploy1001> | Synchronized wmf-config/: SWAT: [[gerrit:584133|Deploy GrowthExperiments on Serbian Wikipedia (T241181)]] (duration: 01m 06s) | [production] | 
            
  | 11:28 | <tgr@deploy1001> | Synchronized dblists/: SWAT: [[gerrit:584133|Deploy GrowthExperiments on Serbian Wikipedia (T241181)]] (duration: 01m 17s) | [production] |