| 2019-03-26
      
      § | 
    
  | 23:37 | <SMalyshev> | repooled wdqs2003 | [production] | 
            
  | 23:12 | <ebernhardson@deploy1001> | Synchronized wmf-config/InitialiseSettings-labs.php: T216206 : sync noop labs config: Actually load WBCS-Lexeme extension before trying to use it  (duration: 00m 57s) | [production] | 
            
  | 22:12 | <gehel> | freezing and unfreezing writes to elasticsearch codfw | [production] | 
            
  | 21:47 | <SMalyshev> | depool wdq2003 to catch it up | [production] | 
            
  | 21:32 | <ebernhardson> | manually thaw search.svc.codfw.wmnet:9643 | [production] | 
            
  | 21:31 | <mholloway-shell@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enable WikimediaEditorTasks on testwikidatawiki (duration: 00m 57s) | [production] | 
            
  | 21:22 | <mdholloway> | created new db tables for WikimediaEditorTasks in x1 | [production] | 
            
  | 21:00 | <SMalyshev> | depooled wdqs1006 to see if it'd catch up better | [production] | 
            
  | 20:19 | <marxarelli> | correction: group0 to 1.33.0-wmf.23 | [production] | 
            
  | 20:15 | <dduvall@deploy1001> | rebuilt and synchronized wikiversions files: group0 to 1.33.0-wmf.0 | [production] | 
            
  | 20:08 | <ejegg> | updated payments-wiki from f42910460b to 6661655e37 | [production] | 
            
  | 19:58 | <dduvall@deploy1001> | Finished scap: testwiki to php-1.33.0-wmf.23 and rebuild l10n cache (duration: 37m 59s) | [production] | 
            
  | 19:20 | <dduvall@deploy1001> | Started scap: testwiki to php-1.33.0-wmf.23 and rebuild l10n cache | [production] | 
            
  | 19:18 | <marxarelli> | scap clean failure due to T218783. train is rolling without cleanup | [production] | 
            
  | 19:17 | <jynus> | reloading db2095 mariadb instances to reload and check filters | [production] | 
            
  | 19:13 | <jynus> | reloading db2094 mariadb instances to reload and check filters | [production] | 
            
  | 19:07 | <dduvall@deploy1001> | clean aborted: Pruned MediaWiki: 1.33.0-wmf.17 (duration: 00m 10s) | [production] | 
            
  | 19:04 | <jynus> | reloading db1125 mariadb instances to reload and check filters | [production] | 
            
  | 18:49 | <marxarelli> | branch 1.33.0-wmf.23 was cut successfully (T206677) | [production] | 
            
  | 18:24 | <jynus> | reloading db1124 mariadb instances to reload and check filters | [production] | 
            
  | 18:21 | <marxarelli> | starting branch cut for 1.33.0-wmf.23 (T206677) | [production] | 
            
  | 18:09 | <thcipriani> | gerrit back on version 2.15.12, upgrade complete. | [production] | 
            
  | 18:05 | <thcipriani> | restarting gerrit on cobalt for update to 2.15.12 | [production] | 
            
  | 18:05 | <gehel@cumin2001> | END (FAIL) - Cookbook sre.elasticsearch.e6-upgrade (exit_code=99) | [production] | 
            
  | 18:05 | <thcipriani@deploy1001> | Finished deploy [gerrit/gerrit@d3d2134]: Gerrit to 2.15.12 on cobalt (duration: 00m 15s) | [production] | 
            
  | 18:04 | <thcipriani@deploy1001> | Started deploy [gerrit/gerrit@d3d2134]: Gerrit to 2.15.12 on cobalt | [production] | 
            
  | 18:03 | <thcipriani@deploy1001> | Finished deploy [gerrit/gerrit@d3d2134]: Gerrit to 2.15.12 on gerrit2001 only (duration: 00m 11s) | [production] | 
            
  | 18:03 | <thcipriani@deploy1001> | Started deploy [gerrit/gerrit@d3d2134]: Gerrit to 2.15.12 on gerrit2001 only | [production] | 
            
  | 18:01 | <thcipriani> | starting gerrit 2.15.12 upgrade | [production] | 
            
  | 17:45 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 17:45 | <otto@deploy1001> | scap-helm eventgate-analytics cluster eqiad completed | [production] | 
            
  | 17:45 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade production -f eventgate-analytics-eqiad-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: eqiad] | [production] | 
            
  | 17:43 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 17:43 | <otto@deploy1001> | scap-helm eventgate-analytics cluster codfw completed | [production] | 
            
  | 17:43 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade production -f eventgate-analytics-codfw-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: codfw] | [production] | 
            
  | 17:41 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 17:41 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 17:41 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 17:39 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 17:39 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 17:39 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --dry-run --debug stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] |