| 2020-03-10
      
      § | 
    
  | 23:53 | <volker-e@deploy1001> | Finished deploy [design/style-guide@8eb1daf]: Deploy design/style-guide:  (duration: 00m 05s) | [production] | 
            
  | 23:53 | <volker-e@deploy1001> | Started deploy [design/style-guide@8eb1daf]: Deploy design/style-guide: | [production] | 
            
  | 23:50 | <ejegg> | disabled recurring donation charge jobs for upgrade | [production] | 
            
  | 23:48 | <mutante> | mw2376 - systemctl start apache2 | [production] | 
            
  | 23:45 | <dzahn@cumin1001> | conftool action : set/weight=15; selector: name=mw2376.codfw.wmnet | [production] | 
            
  | 23:45 | <ebernhardson> | start in-place reindex procedure on kowiki against eqiad and codfw | [production] | 
            
  | 23:44 | <ebernhardson@deploy1001> | Synchronized php-1.35.0-wmf.22/extensions/CirrusSearch/includes/Maintenance/Reindexer.php: (no justification provided) (duration: 01m 07s) | [production] | 
            
  | 23:42 | <cmjohnson@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 23:39 | <cmjohnson@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 23:38 | <ebernhardson@deploy1001> | Synchronized php-1.35.0-wmf.23/extensions/CirrusSearch/includes/Maintenance/Reindexer.php: cirrus: Wait around after a refresh before counting docs (duration: 01m 08s) | [production] | 
            
  | 23:37 | <mutante> | mw2366 - systemctl start nutcracker | [production] | 
            
  | 23:11 | <dzahn@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 23:11 | <dzahn@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 23:07 | <dzahn@cumin1001> | conftool action : set/pooled=yes; selector: name=mw237[135].codfw.wmnet | [production] | 
            
  | 23:05 | <dzahn@cumin1001> | conftool action : set/pooled=yes; selector: name=mw236[579].codfw.wmnet | [production] | 
            
  | 23:05 | <krinkle@deploy1001> | Synchronized wmf-config/wgConf.php: Ib5473af6 (duration: 01m 07s) | [production] | 
            
  | 23:02 | <krinkle@deploy1001> | Synchronized multiversion/MWConfigCacheGenerator.php: Ib5473af6 (duration: 01m 07s) | [production] | 
            
  | 22:58 | <krinkle@deploy1001> | Synchronized multiversion/MWMultiVersion.php: Ib5473af6 (duration: 01m 07s) | [production] | 
            
  | 22:31 | <dzahn@cumin1001> | conftool action : set/pooled=yes; selector: name=mw236[0-5].codfw.wmnet | [production] | 
            
  | 22:28 | <dzahn@cumin1001> | conftool action : set/pooled=yes; selector: name=mw235[0-9].codfw.wmnet | [production] | 
            
  | 22:12 | <dzahn@cumin1001> | conftool action : set/weight=15; selector: name=mw237[0-4].codfw.wmnet | [production] | 
            
  | 22:12 | <dzahn@cumin1001> | conftool action : set/weight=15; selector: name=mw236[0-9].codfw.wmnet | [production] | 
            
  | 22:11 | <mutante> | mw2359 sudo systemctl start php7.2-fpm_check_restart | [production] | 
            
  | 22:09 | <dzahn@cumin1001> | conftool action : set/weight=15; selector: name=mw235[0-9].codfw.wmnet | [production] | 
            
  | 21:58 | <ebernhardson@deploy1001> | Finished deploy [search/mjolnir/deploy@dda3d28]: re-sync latest version to trigger scap scripts on new elastic nodes in codfw (duration: 02m 15s) | [production] | 
            
  | 21:56 | <ebernhardson@deploy1001> | Started deploy [search/mjolnir/deploy@dda3d28]: re-sync latest version to trigger scap scripts on new elastic nodes in codfw | [production] | 
            
  | 21:51 | <ebernhardson@deploy1001> | Finished deploy [search/mjolnir/deploy@dda3d28]: re-sync latest version to trigger scap scripts on new elastic nodes in codfw (duration: 00m 23s) | [production] | 
            
  | 21:51 | <ebernhardson@deploy1001> | Started deploy [search/mjolnir/deploy@dda3d28]: re-sync latest version to trigger scap scripts on new elastic nodes in codfw | [production] | 
            
  | 21:38 | <volker-e@deploy1001> | Finished deploy [design/style-guide@8eb1daf]: Deploy design/style-guide:  (duration: 00m 07s) | [production] | 
            
  | 21:38 | <volker-e@deploy1001> | Started deploy [design/style-guide@8eb1daf]: Deploy design/style-guide: | [production] | 
            
  | 21:29 | <dzahn@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 21:26 | <dzahn@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:49 | <brennen@deploy1001> | rebuilt and synchronized wikiversions files: group0 to 1.35.0-wmf.23 | [production] | 
            
  | 20:39 | <brennen@deploy1001> | Finished scap: testwiki to php-1.35.0-wmf.23 and rebuild l10n cache (duration: 163m 37s) | [production] | 
            
  | 20:29 | <dzahn@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 20:29 | <dzahn@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 20:11 | <eileen> | civicrm revision changed from 10506a9644 to 3de711ed49, config revision is 2d7b926c1d | [production] | 
            
  | 20:10 | <eileen> | process-control config revision is 2d7b926c1d | [production] | 
            
  | 19:41 | <otto@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-main' for release 'production' . | [production] | 
            
  | 19:40 | <otto@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-main' for release 'canary' . | [production] | 
            
  | 19:38 | <mutante> | gerrit1001 - /var/log/syslog empty and 2 rsyslogd procs running, killing one of them, stopping the other, letting puppet run | [production] | 
            
  | 19:37 | <otto@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-analytics-external' for release 'canary' . | [production] | 
            
  | 19:34 | <otto@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'canary' . | [production] | 
            
  | 19:34 | <volker-e@deploy1001> | Finished deploy [design/style-guide@62bf7c6]: Deploy design/style-guide:  (duration: 00m 06s) | [production] | 
            
  | 19:34 | <volker-e@deploy1001> | Started deploy [design/style-guide@62bf7c6]: Deploy design/style-guide: | [production] | 
            
  | 19:32 | <otto@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-analytics' for release 'production' . | [production] |