| 
      
        2015-07-29
      
      §
     | 
  
    
  | 23:55 | 
  <marxarelli> | 
  clearing disk space on integrations-slave-trusty-1012 with `find /mnt/jenkins-workspace/workspace -mindepth 1 -maxdepth 1 -type d -mtime +15 -exec rm -rf {} \;` | 
  [releng] | 
            
  | 18:15 | 
  <bd808> | 
  upgraded nutcracker on deployment-jobrunner01 | 
  [releng] | 
            
  | 18:14 | 
  <bd808> | 
  upgraded nutcracker on deployment-videoscaler01 | 
  [releng] | 
            
  | 18:08 | 
  <bd808> | 
  rm deployment-fluorine:/a/mw-log/archive/*-201506* | 
  [releng] | 
            
  | 18:08 | 
  <bd808> | 
  rm deployment-fluorine:/a/mw-log/archive/*-201505* | 
  [releng] | 
            
  | 18:02 | 
  <bd808> | 
  rm deployment-videoscaler01:/var/log/atop.log.?* | 
  [releng] | 
            
  | 16:49 | 
  <thcipriani> | 
  lots of "Error connecting to 10.68.16.193: Can't connect to MySQL server on '10.68.16.193'" deployment-db1 seems up and functional :( | 
  [releng] | 
            
  | 16:27 | 
  <thcipriani> | 
  deployment-prep login timeouts, tried restarting apache, hhvm, and nutcracker on mediawiki{01..03} | 
  [releng] | 
            
  | 14:38 | 
  <bblack> | 
  cherry-picked https://gerrit.wikimedia.org/r/#/c/215624 (updated to PS8) into deployment-puppetmaster ops/puppet | 
  [releng] | 
            
  | 14:28 | 
  <bblack> | 
  cherry-picked https://gerrit.wikimedia.org/r/#/c/215624 into deployment-puppetmaster ops/puppet | 
  [releng] | 
            
  | 12:38 | 
  <hashar_> | 
  salt minions are back somehow | 
  [releng] | 
            
  | 12:36 | 
  <hashar_> | 
  salt on deployment-salt is missing most of the instances :-((( | 
  [releng] | 
            
  | 03:00 | 
  <ostriches> | 
  deployment-bastion: please please someone rebuild me to not have a stupid 2G /var partition | 
  [releng] | 
            
  | 02:59 | 
  <ostriches> | 
  deployment-bastion: purged a bunch of atop and pacct logs, and apt cache...clogging up /var again. | 
  [releng] | 
            
  | 02:34 | 
  <legoktm> | 
  deploying https://gerrit.wikimedia.org/r/227640 | 
  [releng] | 
            
  
    | 
      
        2015-07-28
      
      §
     | 
  
    
  | 23:43 | 
  <marxarelli> | 
  running `jenkins-jobs update config/ 'mwext-mw-selenium'` to deploy I7afa07e9f559bffeeebaf7454cc6b39a37e04063 | 
  [releng] | 
            
  | 21:05 | 
  <bd808> | 
  upgraded nutcracker on mediawiki03 | 
  [releng] | 
            
  | 21:04 | 
  <bd808> | 
  upgraded nutcracker on mediawiki02 | 
  [releng] | 
            
  | 21:01 | 
  <bd808> | 
  upgraded nutcracker on mediawiki01 | 
  [releng] | 
            
  | 19:49 | 
  <jzerebecki> | 
  reloading zuul b1b2cab..b02830e | 
  [releng] | 
            
  | 11:18 | 
  <hashar> | 
  Assigning label "BetaClusterBastion" to https://integration.wikimedia.org/ci/computer/deployment-bastion.eqiad/ | 
  [releng] | 
            
  | 11:12 | 
  <hashar> | 
  Jenkins jobs for the beta cluster ended up stuck again.  Found a workaround by removing the Jenkins label  on deployment-bastion node and reinstating it.  Seems to get rid of the deadlock ( ref: https://phabricator.wikimedia.org/T72597#1487801 ) | 
  [releng] | 
            
  | 09:50 | 
  <hashar> | 
  deployment-apertium01 is back!  The ferm rules were outdated / not maintained by puppet, dropped ferm entirely. | 
  [releng] | 
            
  | 09:40 | 
  <hashar> | 
  rebooting deployment-apertium01 to ensure its ferm rules are properly loaded on boot ( https://phabricator.wikimedia.org/T106658 ) | 
  [releng] | 
            
  | 00:46 | 
  <legoktm> | 
  deploying https://gerrit.wikimedia.org/r/227383 | 
  [releng] |