| 2019-04-16
      
      ยง | 
    
  | 20:04 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 20:04 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 20:04 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values --set debug_mode_enabled=true stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 19:59 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 19:59 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 19:59 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values --set debug_mode_enabled=true stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 19:56 | <gehel> | restarting cassandra on maps* for config change - T221055 | [production] | 
            
  | 19:49 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 19:49 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 19:49 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values --set debug_mode_enabled=true stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 19:48 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 19:48 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 19:48 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values --set main_app.debug_mode_enabled=true stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 19:11 | <twentyafterfour> | twentyafterfour@deploy1001:/srv/mediawiki-staging$ scap prep 1.34.0-wmf.1 | [production] | 
            
  | 19:07 | <bblack> | restarting varnish backend on cp1083 | [production] | 
            
  | 19:04 | <bblack> | restarting varnish backend on cp1085 | [production] | 
            
  | 18:55 | <cdanis> | cdanis@cp1085.eqiad.wmnet ~ % sudo -i depool | [production] | 
            
  | 18:53 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 18:53 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 18:53 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values --set main_app.profiling_enabled=true stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 18:46 | <twentyafterfour> | branching 1.34.0-wmf.1 refs T220726 | [production] | 
            
  | 18:25 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 18:25 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 18:25 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values --set wmfdebug_enabled=true stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 18:14 | <cmjohnson1> | powering off mw1280 to replace DIMM | [production] | 
            
  | 18:07 | <mutante> | restbase2007, restbase2008 - re-enabled puppet which was disabled with reason 'decom'ed' but actually needed to run to decom after they had moved to role::spare::system (T208087) | [production] | 
            
  | 17:56 | <reedy@deploy1001> | Synchronized php-1.33.0-wmf.25/extensions/WikimediaIncubator/: T220623 (duration: 00m 53s) | [production] | 
            
  | 17:47 | <herron> | beginning rolling ELK upgrade to 5.6.15 | [production] | 
            
  | 17:46 | <lucaswerkmeister-wmde@deploy1001> | Synchronized wmf-config/InitialiseSettings-labs.php: [[gerrit:504386|no-op preparatory change (T221107)]] (duration: 00m 52s) | [production] | 
            
  | 17:36 | <arturo> | toolforge k8s reallocation (from nova-network to neutron) is causing troubles with IRC bots, expect missing entries in the SAL | [production] | 
            
  | 17:28 | <aborrero@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 17:28 | <aborrero@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 17:27 | <andrewbogott> | restarting rabbitmq on cloudcontrol1003 | [production] | 
            
  | 17:26 | <cdanis@puppetmaster1001> | conftool action : set/pooled=no; selector: dc=eqiad,name=mw1280.eqiad.wmnet,cluster=api_appserver | [production] | 
            
  | 17:25 | <arturo> | rebooted cloudnet1003 | [production] | 
            
  | 17:24 | <gehel> | force initialization of unassigned shards on elasticsearch eqiad | [production] | 
            
  | 17:16 | <lucaswerkmeister-wmde@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: [[gerrit:504374|no-op preparatory change (T221108)]] (duration: 00m 52s) | [production] | 
            
  | 16:54 | <Lucas_WMDE> | lucaswerkmeister-wmde@mwmaint1002:~$ mwscript extensions/WikibaseQualityConstraints/maintenance/ImportConstraintEntities.php --wiki=testwikidatawiki --config-format=wgConf | tee T221108.php | [production] | 
            
  | 16:53 | <mutante> | bast2001 - shutdown -h now - decom'ed (T219492) | [production] | 
            
  | 16:48 | <mutante> | puppet node clean bast2001.wikimedia.org ; puppet node deactivate bast2001.wikimedia.org ; it showed up in Icinga again despite running decom cookbook (T219492) | [production] | 
            
  | 16:47 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 16:47 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 16:47 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values --set wmfdebug_enabled=true stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 16:44 | <otto@deploy1001> | scap-helm eventgate-analytics finished | [production] | 
            
  | 16:44 | <otto@deploy1001> | scap-helm eventgate-analytics cluster staging completed | [production] | 
            
  | 16:44 | <otto@deploy1001> | scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | [production] | 
            
  | 16:43 | <jynus> | upgrading and shutting down db1078 T219115 | [production] | 
            
  | 16:41 | <jynus> | disabling notifications on db1078 T219115 | [production] | 
            
  | 16:37 | <jynus@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1078 (duration: 00m 52s) | [production] | 
            
  | 15:36 | <arturo> | reimaging cloudnet2002-dev because role name change | [production] |