| 
      
        2016-09-16
      
      §
     | 
  
    
  | 21:03 | 
  <hashar> | 
  deployment-tin  did a git gc on /srv/deployment/ores   That freed up disk space and cleared an alarm on co master mira02 | 
  [releng] | 
            
  | 21:00 | 
  <hashar> | 
  deleted deployment-parsoid05  | 
  [releng] | 
            
  | 20:52 | 
  <hashar> | 
  fixed puppet on deployment-parsoid05 . Temporary instance will delete it later to clear out shinken.wmflabs.org | 
  [releng] | 
            
  | 20:27 | 
  <hashar> | 
  beta:  force running puppet in batches of 4 instances:  salt --batch 4 -v 'deployment-*' cmd.run 'puppet agent -tv' | 
  [releng] | 
            
  | 20:13 | 
  <hashar> | 
  beta: restarted puppetmaster  | 
  [releng] | 
            
  | 20:07 | 
  <hashar> | 
  beta: salt -v '*' cmd.run 'rm -fR /var/lib/puppet/client/ssl/' | 
  [releng] | 
            
  | 20:07 | 
  <hashar> | 
  beta: stopping puppetmaster,  rm -f /var/lib/puppet/server/ssl/ca/signed/* | 
  [releng] | 
            
  | 19:53 | 
  <hashar> | 
  beta created instance "deployment-parsoid05" Should be deleted later, that is merely to purge the hostname from Shinken ( http://shinken.wmflabs.org/host/deployment-parsoid05 ) | 
  [releng] | 
            
  | 11:42 | 
  <hashar> | 
  beta: apt-get upgrade on deployment-jobrunner01 | 
  [releng] | 
            
  | 11:36 | 
  <hashar> | 
  apt-get upgrade on deployment-tin , bring in a new hhvm version and others | 
  [releng] | 
            
  
    | 
      
        2016-09-15
      
      §
     | 
  
    
  | 22:29 | 
  <legoktm> | 
  sudo salt '*precise*' cmd.run 'service mysql start', all mysql's are down | 
  [releng] | 
            
  | 16:45 | 
  <godog> | 
  install xenial kernel on deployment-zotero01 and reboot T145793 | 
  [releng] | 
            
  | 16:18 | 
  <hashar> | 
  prometheus enabled on all beta cluster instance.  Does not support Precise hence puppet will fail on the last two Precise instances deployment-db1 and deployment-db2  until they are migrated to Jessie   T138778 | 
  [releng] | 
            
  | 15:53 | 
  <godog> | 
  add role::prometheus::node_exporter to classes in hiera:deployment-prep T144502 | 
  [releng] | 
            
  | 15:10 | 
  <hashar> | 
  beta: Applying puppet class role::prometheus::node_exporter to mira02 just like mira.  That is for godog | 
  [releng] | 
            
  | 15:08 | 
  <hashar> | 
  T144006 Disabled Jenkins job  beta-scap-eqiad.  On mira02  rm -fR /srv/*  .  Applying puppet for role::labs::lvm::srv | 
  [releng] | 
            
  | 15:05 | 
  <hashar> | 
  T144006  Applying class role::labs::lvm::srv to mira02  (it is out of disk space :D ) | 
  [releng] | 
            
  | 14:45 | 
  <hashar> | 
  T144006 sudo -u jenkins-deploy -H SSH_AUTH_SOCK=/run/keyholder/proxy.sock ssh mwdeploy@mira02.deployment-prep.eqiad.wmflabs | 
  [releng] | 
            
  | 14:44 | 
  <hashar> | 
  T144006 sudo -u jenkins-deploy -H SSH_AUTH_SOCK=/run/keyholder/proxy.sock ssh mwdeploy@deployment-mediawiki05.deployment-prep.eqiad.wmflabs | 
  [releng] | 
            
  | 12:33 | 
  <elukey> | 
  added base::firewall, beta::deployaccess, mediawiki::conftool, role::mediawiki::appserver to mediawiki05 | 
  [releng] | 
            
  | 12:20 | 
  <elukey> | 
  terminate mediawiki02 to create mediawiki05 | 
  [releng] | 
            
  | 10:48 | 
  <hashar> | 
  beta: cherry picking moritzm patch https://gerrit.wikimedia.org/r/#/c/310793/ "Also handle systemd in keyholder script" T144578 | 
  [releng] | 
            
  | 09:33 | 
  <hashar> | 
  T144006 sudo -u jenkins-deploy -H SSH_AUTH_SOCK=/run/keyholder/proxy.sock ssh mwdeploy@deployment-mediawiki06.deployment-prep.eqiad.wmflabs | 
  [releng] | 
            
  | 09:10 | 
  <elukey> | 
  executed git pull and then git rebase -i on deployment puppet master | 
  [releng] | 
            
  | 08:52 | 
  <elukey> | 
  terminated mediawiki03 and created mediawiki06 | 
  [releng] | 
            
  | 08:45 | 
  <elukey> | 
  removed mediawiki03 from puppet with https://gerrit.wikimedia.org/r/#/c/310749/ | 
  [releng] | 
            
  | 02:36 | 
  <Krinkle> | 
  Reloading Zuul to deploy https://gerrit.wikimedia.org/r/310701  | 
  [releng] |