| 2019-04-16
      
      ยง | 
    
  | 15:21 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 15:21 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 15:20 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging --version 0.0.28 -f eventgate-analytics-staging-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 15:19 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 15:19 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 15:19 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 15:18 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 15:18 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 15:18 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 15:16 | <elukey> | roll restart kafka on kafka-jumbo100[1-6] to pick up openjdk upgrades | [production] | 
            
  | 14:58 | <gehel> | manual data transfer from wdqs1008 to wdqs1009 - T220830 | [production] | 
            
  | 14:56 | <ema> | swift-fe-eqiad: nginx reload for new TLS certificate T204245 | [production] | 
            
  | 14:53 | <gehel@cumin1001> | END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) | [production] | 
            
  | 14:52 | <gehel@cumin1001> | START - Cookbook sre.wdqs.data-transfer | [production] | 
            
  | 14:51 | <ema@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=ms-fe1005.eqiad.wmnet | [production] | 
            
  | 14:45 | <ema> | test https://gerrit.wikimedia.org/r/504340 on ms-fe1005 T204245 | [production] | 
            
  | 14:30 | <ema> | swift-fe-codfw: nginx reload for new TLS certificate T204245 | [production] | 
            
  | 14:22 | <gehel@cumin1001> | END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) | [production] | 
            
  | 14:21 | <gehel@cumin1001> | START - Cookbook sre.wdqs.data-transfer | [production] | 
            
  | 14:20 | <elukey> | roll restart of all the druid daemons on druid100[1-6] to pick up new openjdk updates | [production] | 
            
  | 14:17 | <ema@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=ms-fe2005.codfw.wmnet | [production] | 
            
  | 14:07 | <jijiki> | Pooling thumbor1001 | [production] | 
            
  | 14:04 | <ema> | test https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/504331/ on ms-fe2005 T204245 | [production] | 
            
  | 14:01 | <ema@puppetmaster1001> | conftool action : set/pooled=no; selector: name=ms-fe2005.codfw.wmnet | [production] | 
            
  | 14:01 | <jijiki> | Depooling thumbor1001 | [production] | 
            
  | 13:58 | <jijiki> | Disable puppet on thumbor1001 for ~24h to serve traffic via haproxy - T187765 | [production] | 
            
  | 13:54 | <gehel@cumin1001> | END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) | [production] | 
            
  | 13:53 | <gehel@cumin1001> | START - Cookbook sre.wdqs.data-transfer | [production] | 
            
  | 13:52 | <jijiki> | Enable puppet on thumbor* | [production] | 
            
  | 13:42 | <gehel@cumin1001> | END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) | [production] | 
            
  | 13:41 | <gehel@cumin1001> | START - Cookbook sre.wdqs.data-transfer | [production] | 
            
  | 13:39 | <gehel> | restetting cookbooks repo on cumin1001 (local changes) | [production] | 
            
  | 13:34 | <jijiki> | Disabling puppet on thumbor* to merge 504284 | [production] | 
            
  | 13:13 | <ema> | cp-ats: upgrade fifo-log-demux to 0.2 and restart services | [production] | 
            
  | 13:10 | <ema> | fifo-log-demux 0.2 uploaded to stretch-wikimedia | [production] | 
            
  | 13:03 | <arturo> | T220095 renaming/reimaging labtestcontrol2003 as cloudcontrol2003-dev | [production] | 
            
  | 12:58 | <moritzm> | installing ghostscript update on thumbor1001 | [production] | 
            
  | 12:54 | <gehel> | cleanup redundant prometheus-elasticsearch units on elasticsearch servers | [production] | 
            
  | 12:52 | <godog> | swift eqiad-prod continue ms-be1013 decom - T220590 | [production] | 
            
  | 12:17 | <moritzm> | installing OpenSSL 1.0.2 updates on cp* Varnish hosts | [production] | 
            
  | 12:07 | <arturo> | rebooting cloudvirt200[123]-dev because deep changes in config | [production] | 
            
  | 11:18 | <hoo@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Add wgWikibaseMusicalNotationLineWidthInches to config (T218191) (duration: 00m 52s) | [production] | 
            
  | 11:10 | <hoo@deploy1001> | Synchronized wmf-config/Wikibase.php: Revert "WikibaseClient: Conditionally enable mapframe support" (T218051) (duration: 00m 51s) | [production] | 
            
  | 11:07 | <hoo@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enable signatures in 2019: NS (ID 128) for wikimaniawiki (T221062) (duration: 00m 52s) | [production] | 
            
  | 10:49 | <gilles> | T221065 eswiki purge finished | [production] | 
            
  | 10:45 | <moritzm> | installing libjs-bootstrap updates from Stretch point release | [production] | 
            
  | 10:21 | <gilles> | T221065 mwscript purgeList.php eswiki --all --verbose on mwmaint1002 | [production] | 
            
  | 10:21 | <moritzm> | installing xapian-core update from stretch point release | [production] | 
            
  | 10:18 | <gilles@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: T221065 Set up origin trials on Spanish Wikipedia mobile site (duration: 00m 52s) | [production] | 
            
  | 09:59 | <jijiki> | Enabling puppet again on on dbproxy* and thumbor* | [production] |