| 
      
        2021-08-04
      
      ยง
     | 
  
    
  | 22:18 | 
  <ebernhardson@deploy1002> | 
  Finished deploy [wikimedia/discovery/analytics@34cd541]: gerrit:709835 and 709836 (duration: 06m 52s) | 
  [production] | 
            
  | 22:11 | 
  <ebernhardson@deploy1002> | 
  Started deploy [wikimedia/discovery/analytics@34cd541]: gerrit:709835 and 709836 | 
  [production] | 
            
  | 20:56 | 
  <legoktm@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 20:21 | 
  <legoktm@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 19:22 | 
  <dduvall> | 
  1.37.0-wmf.17 promoted to group1. no new errors or troubling error rates spotted (T281158) | 
  [production] | 
            
  | 19:16 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 19:12 | 
  <dduvall@deploy1002> | 
  Synchronized php: group1 wikis to 1.37.0-wmf.17 (duration: 01m 15s) | 
  [production] | 
            
  | 19:11 | 
  <dduvall@deploy1002> | 
  rebuilt and synchronized wikiversions files: group1 wikis to 1.37.0-wmf.17 | 
  [production] | 
            
  | 18:45 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 18:41 | 
  <urbanecm@deploy1002> | 
  Synchronized php-1.37.0-wmf.17/extensions/GrowthExperiments/: 5c3ac582335265287369e2d06332645ddbcba412: Fix array key handling for GEHelpPanelLinks in on-wiki config (T288023) (duration: 01m 08s) | 
  [production] | 
            
  | 18:34 | 
  <dzahn@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2380.codfw.wmnet | 
  [production] | 
            
  | 18:33 | 
  <dzahn@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2379.codfw.wmnet | 
  [production] | 
            
  | 18:30 | 
  <urbanecm@deploy1002> | 
  Synchronized php-1.37.0-wmf.16/extensions/GrowthExperiments/: 36a2b9f58148dad5434daa6d03b77f4c8b839314: Fix array key handling for GEHelpPanelLinks in on-wiki config (T288023) (duration: 01m 06s) | 
  [production] | 
            
  | 18:28 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 18:25 | 
  <mutante> | 
  mw2379, mw2380 - scap pull | 
  [production] | 
            
  | 18:16 | 
  <brennen> | 
  gitlab1001: upgrading to 13.12.9 | 
  [production] | 
            
  | 18:11 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 18:11 | 
  <brennen> | 
  gitlab2001: upgrading to 13.12.9 | 
  [production] | 
            
  | 18:10 | 
  <otto@deploy1002> | 
  Synchronized wmf-config/InitialiseSettings.php: Enable canary events by default - T287789 (duration: 01m 06s) | 
  [production] | 
            
  | 18:03 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mw2380.codfw.wmnet with reason: reimage | 
  [production] | 
            
  | 18:03 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 1:00:00 on mw2380.codfw.wmnet with reason: reimage | 
  [production] | 
            
  | 18:01 | 
  <dzahn@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on mw2380.codfw.wmnet with reason: REIMAGE | 
  [production] | 
            
  | 18:01 | 
  <dzahn@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 1:00:00 on mw2380.codfw.wmnet with reason: reimage | 
  [production] | 
            
  | 18:01 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 1:00:00 on mw2380.codfw.wmnet with reason: reimage | 
  [production] | 
            
  | 18:01 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mw2379.codfw.wmnet with reason: reimage | 
  [production] | 
            
  | 18:01 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 1:00:00 on mw2379.codfw.wmnet with reason: reimage | 
  [production] | 
            
  | 17:59 | 
  <dzahn@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on mw2379.codfw.wmnet with reason: REIMAGE | 
  [production] | 
            
  | 17:59 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 2:00:00 on mw2380.codfw.wmnet with reason: REIMAGE | 
  [production] | 
            
  | 17:57 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 2:00:00 on mw2379.codfw.wmnet with reason: REIMAGE | 
  [production] | 
            
  | 17:49 | 
  <dzahn@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2378.codfw.wmnet | 
  [production] | 
            
  | 17:49 | 
  <dzahn@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2377.codfw.wmnet | 
  [production] | 
            
  | 17:46 | 
  <dzahn@cumin1001> | 
  conftool action : set/weight=30; selector: name=mw2380.codfw.wmnet | 
  [production] | 
            
  | 17:46 | 
  <dzahn@cumin1001> | 
  conftool action : set/weight=30; selector: name=mw237[7-9].codfw.wmnet | 
  [production] | 
            
  | 17:45 | 
  <dzahn@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2357.codfw.wmnet | 
  [production] | 
            
  | 17:41 | 
  <dzahn@cumin1001> | 
  conftool action : set/weight=25; selector: name=mw2357.codfw.wmnet | 
  [production] | 
            
  | 17:40 | 
  <mutante> | 
  mw2357, mw2377, mw2378 - scap pull | 
  [production] | 
            
  | 17:40 | 
  <dzahn@cumin1001> | 
  conftool action : set/weight=30; selector: name=mw2357.codfw.wmnet | 
  [production] | 
            
  | 17:39 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 17:29 | 
  <dzahn@cumin1001> | 
  conftool action : set/weight=25; selector: name=mw238[1-2].codfw.wmnet | 
  [production] | 
            
  | 17:29 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 17:27 | 
  <ejegg> | 
  updated payments-wiki config to 360c8a1f08 | 
  [production] | 
            
  | 17:26 | 
  <dzahn@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2355.codfw.wmnet | 
  [production] | 
            
  | 17:25 | 
  <dzahn@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2353.codfw.wmnet | 
  [production] | 
            
  | 17:25 | 
  <dzahn@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2351.codfw.wmnet | 
  [production] | 
            
  | 17:25 | 
  <dzahn@cumin1001> | 
  conftool action : set/weight=25; selector: name=mw2355.codfw.wmnet | 
  [production] | 
            
  | 17:25 | 
  <dzahn@cumin1001> | 
  conftool action : set/weight=25; selector: name=mw2353.codfw.wmnet | 
  [production] | 
            
  | 17:25 | 
  <dzahn@cumin1001> | 
  conftool action : set/weight=25; selector: name=mw2351.codfw.wmnet | 
  [production] | 
            
  | 17:15 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2378.codfw.wmnet with reason: REIMAGE | 
  [production] | 
            
  | 17:13 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2377.codfw.wmnet with reason: REIMAGE | 
  [production] | 
            
  | 17:12 | 
  <urbanecm@deploy1002> | 
  Synchronized php-1.37.0-wmf.17/extensions/GrowthExperiments/maintenance/updateMenteeData.php: 66c2c7593322dfc575edc818aaff8d9b79466bdd: updateMenteeData: Output how long the script took (T287964) (duration: 01m 07s) | 
  [production] |