| 
      
        2020-02-11
      
      ยง
     | 
  
    
  | 19:36 | 
  <dduvall@deploy1001> | 
  Pruned MediaWiki: 1.35.0-wmf.11 (duration: 10m 53s) | 
  [production] | 
            
  | 19:35 | 
  <marxarelli> | 
  running `scap clean --delete` for old wmf branches wmf.11, wmf.14, wmf.15, wmf.16 (T233867) | 
  [production] | 
            
  | 19:03 | 
  <volans> | 
  uploaded spicerack_0.0.30-1_amd64.deb to apt.wikimedia.org stretch-wikimedia | 
  [production] | 
            
  | 19:00 | 
  <Urbanecm> | 
  Create User:Ammarpad on ngwikimedia and promote to sysop, bureaucrat (T240771) | 
  [production] | 
            
  | 18:48 | 
  <jforrester@deploy1001> | 
  rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.18 | 
  [production] | 
            
  | 18:43 | 
  <twentyafterfour> | 
  getting ready to deploy wmf.18 refs  T233866 | 
  [production] | 
            
  | 18:42 | 
  <greg-g> | 
  restarting stashbot | 
  [production] | 
            
  | 18:35 | 
  <bblack> | 
  ns1.wikimedia.org - changing static route destination on cr[12]-codfw from authdns2001 to dns2002 - T242017 | 
  [production] | 
            
  | 18:33 | 
  <Urbanecm> | 
  Create ngwikimedia is done (T240771) | 
  [production] | 
            
  | 18:26 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Create ngwikimedia (T240771) (duration: 01m 03s) | 
  [production] | 
            
  | 18:24 | 
  <urbanecm@deploy1001> | 
  Synchronized multiversion/MWMultiVersion.php: Create ngwikimedia (T240771) (duration: 01m 06s) | 
  [production] | 
            
  | 18:21 | 
  <urbanecm@deploy1001> | 
  rebuilt and synchronized wikiversions files: Create ngwikimedia (T240771) | 
  [production] | 
            
  | 18:20 | 
  <dpifke@deploy1001> | 
  Finished deploy [performance/navtiming@b471b64]: (no justification provided) (duration: 00m 05s) | 
  [production] | 
            
  | 18:20 | 
  <dpifke@deploy1001> | 
  Started deploy [performance/navtiming@b471b64]: (no justification provided) | 
  [production] | 
            
  | 18:19 | 
  <urbanecm@deploy1001> | 
  Synchronized dblists/: Create ngwikimedia (T240771) (duration: 01m 06s) | 
  [production] | 
            
  | 17:57 | 
  <bblack> | 
  reboot dns2002 post-reimaging | 
  [production] | 
            
  | 17:13 | 
  <vgutierrez> | 
  Disable KA on cp4031 - T244464 | 
  [production] | 
            
  | 16:49 | 
  <vgutierrez> | 
  pool cp3055 running buster - T242093 | 
  [production] | 
            
  | 16:43 | 
  <vgutierrez> | 
  repooling cp4031 | 
  [production] | 
            
  | 16:38 | 
  <vgutierrez> | 
  depooling cp4031 for some KA tests | 
  [production] | 
            
  | 16:25 | 
  <vgutierrez> | 
  pool cp3056 running buster - T242093 | 
  [production] | 
            
  | 16:23 | 
  <bblack> | 
  dns2002 - shutting down for hardware work and reinstall - T242017 | 
  [production] | 
            
  | 16:21 | 
  <bblack> | 
  dns2002 - stopping bird adverts to depool service for T242017 | 
  [production] | 
            
  | 16:20 | 
  <bblack> | 
  dns2002 - downtimed in icinga for T242017 | 
  [production] | 
            
  | 16:07 | 
  <vgutierrez@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 16:05 | 
  <vgutierrez@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 15:38 | 
  <vgutierrez> | 
  depool cp3056 and reimage as buster - T242093 | 
  [production] | 
            
  | 15:36 | 
  <vgutierrez> | 
  pool cp3058 running buster - T242093 | 
  [production] | 
            
  | 15:29 | 
  <otto@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings-labs.php: Configuring test.event stream in beta, no-op in prod - T242122 (duration: 01m 08s) | 
  [production] | 
            
  | 15:24 | 
  <vgutierrez@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | 
  [production] | 
            
  | 15:24 | 
  <vgutierrez@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 15:14 | 
  <vgutierrez@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 15:12 | 
  <vgutierrez@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 14:58 | 
  <vgutierrez> | 
  depool cp3055 and reimage as buster - T242093 | 
  [production] | 
            
  | 14:56 | 
  <vgutierrez> | 
  pool cp3057 running buster - T242093 | 
  [production] | 
            
  | 14:52 | 
  <moritzm> | 
  pruning old CAS logs (predating the current logger config for /var/log/cas/*) from idp1001/idp2001 | 
  [production] | 
            
  | 14:38 | 
  <vgutierrez@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 14:35 | 
  <vgutierrez@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 14:21 | 
  <Amir1> | 
  ladsgroup@mwmaint1002:~$ mwscript createAndPromote.php --wiki=labswiki --force "Ladsgroup" --custom-groups checkuser | 
  [production] | 
            
  | 14:20 | 
  <vgutierrez> | 
  restart varnish-fe on cp4031 - T244464 | 
  [production] | 
            
  | 14:06 | 
  <vgutierrez> | 
  depool cp3057 and cp3058 and reimage as buster - T242093 | 
  [production] | 
            
  | 13:52 | 
  <vgutierrez> | 
  pool cp3059 and cp3060 running buster - T242093 | 
  [production] | 
            
  | 13:03 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Depool db1107 after 10.4 testing - T242702', diff saved to https://phabricator.wikimedia.org/P10382 and previous config saved to /var/cache/conftool/dbconfig/20200211-130343-marostegui.json | 
  [production] | 
            
  | 12:56 | 
  <vgutierrez@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 12:53 | 
  <vgutierrez@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 12:34 | 
  <Amir1> | 
  EU SWAT is done | 
  [production] | 
            
  | 12:31 | 
  <vgutierrez@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 12:29 | 
  <vgutierrez@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 12:28 | 
  <ladsgroup@deploy1001> | 
  Synchronized wmf-config/Wikibase.php: SWAT: [[gerrit:571339|Fix typo in the config name (T244697)]], take II, cache (duration: 01m 06s) | 
  [production] | 
            
  | 12:26 | 
  <ladsgroup@deploy1001> | 
  Synchronized wmf-config/Wikibase.php: SWAT: [[gerrit:571339|Fix typo in the config name (T244697)]] (duration: 01m 05s) | 
  [production] |