| 2015-01-07
      
      § | 
    
  | 10:25 | <hashar> | deleting /etc/varnish on deplloyment-cache-text02 and running puppet | [releng] | 
            
  | 10:24 | <hashar> | beta varnish text cache is broken. The vcl refuses to load because of undefined probes | [releng] | 
            
  | 10:01 | <hashar> | restarted deployment-cache-mobile03 and deployment-cache-text02 | [releng] | 
            
  | 09:50 | <hashar> | rebooting deployment-cache-bits01 | [releng] | 
            
  | 00:41 | <Krinkle> | rm -rf slave-scripts and re-cloning from integration/jenkins.git on all slaves (under sudo, just like puppet originally did) - git-status and jshint both work fine now | [releng] | 
            
  | 00:40 | <Krinkle> | Permissions of deployment/integration/slave-scripts on labs slave are all screwed up (git-status says files are dirty, but when run as root git-status is clean and jshint also works fine via sudo) | [releng] | 
            
  | 00:29 | <Krinkle> | Tried reconnecting Gearman, relaunching slave agents. Force-restarting Zuul now. | [releng] | 
            
  | 00:15 | <Krinkle> | Permissions in deployment/integration/slave-scripts on integration-slave1003 are screwed up as well | [releng] | 
            
  
    | 2014-12-19
      
      § | 
    
  | 23:01 | <greg-g> | Krinkle restarted Gearman, which got the jobs to flow again | [releng] | 
            
  | 20:51 | <Krinkle> | integration-slave1005 (new Ubuntu Trusty instance) is now pooled | [releng] | 
            
  | 18:51 | <Krinkle> | Re-created and provisioning integration-slave1005 (UbuntuTrusty) | [releng] | 
            
  | 18:23 | <bd808> | redis input to logstash stuck; restarted service | [releng] | 
            
  | 18:16 | <bd808> | ran `apt-get dist-upgrade` on logstash01 | [releng] | 
            
  | 18:02 | <bd808> | removed local mwdeploy user & group from videoscaler01 | [releng] | 
            
  | 18:01 | <bd808> | deployment-videoscaler01 has mysteriously aquired a local mwdeploy user instead of the ldap one | [releng] | 
            
  | 17:58 | <bd808> | forcing puppet run on deploymnet-videoscaler01 | [releng] | 
            
  | 07:24 | <Krinkle> | Restarting Gearman connection to Jenkins | [releng] | 
            
  | 07:24 | <Krinkle> | Attempt #5 at re-creating integration-slave1001. Completed provisioning per Setup instructions. Pooled. | [releng] | 
            
  | 05:33 | <Krinkle> | Rebasing integration-puppetmaster with latest operations/puppet upstream (5 patches) | [releng] | 
            
  | 00:06 | <bd808> | restored local commit with ssh keys for scap to deployment-salt | [releng] | 
            
  
    | 2014-12-18
      
      § | 
    
  | 23:57 | <bd808> | temporarily disabled jenkins scap job | [releng] | 
            
  | 23:56 | <bd808> | killed some ancient screen sessions on deployment-bastion | [releng] | 
            
  | 23:53 | <bd808> | Restarted udp2log-mw on deployment-bastion | [releng] | 
            
  | 23:53 | <bd808> | Restarted salt-minion on deployement-bastion | [releng] | 
            
  | 23:47 | <bd808> | Updated scap to latest HEAD version | [releng] | 
            
  | 21:57 | <Krinkle> | integration-slave1005 is not ready. It's incompletely setup due to https://phabricator.wikimedia.org/T84917 | [releng] | 
            
  | 19:29 | <marxarelli> | restarted puppetmaster on deployment-salt | [releng] | 
            
  | 19:29 | <marxarelli> | seeing "Could not evaluate: getaddrinfo: Temporary failure in name resolution" in the deployment-* puppet logs | [releng] | 
            
  | 14:17 | <hashar> | deleting instance deployment-parsoid04 and removing it from Jenkins | [releng] | 
            
  | 14:08 | <hashar> | restarted varnish backend on parsoidcache02 | [releng] | 
            
  | 14:00 | <hashar> | parsoid05 seems happy: curl http://localhost:8000/_version: <tt>{"name":"parsoid","version":"0.2.0-git","sha":"d16dd2db6b3ca56e73439e169d52258214f0aeb2"}</tt> | [releng] | 
            
  | 14:00 | <hashar> | parsoid05 seems happy: curl http://localhost:8000/_version<br/> | [releng] | 
            
  | 13:56 | <hashar> | applying latest changes of Parsoid on parsoid05 via: <tt>zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/services/parsoid --change 180671,2</tt> | [releng] | 
            
  | 13:56 | <hashar> | parsoid05: disabling puppet, stopping parsoid, rm -fR /srv/deployment/parsoid ; rerunning the Jenkins beta-parsoid-update-eqiad to hopefully recreate everything properly | [releng] | 
            
  | 13:52 | <hashar> | making parsoid05 a Jenkins slave to replace parsoid04 | [releng] | 
            
  | 13:24 | <hashar> | apt-get upgrade on parsoidcache02 and parsoid04 | [releng] | 
            
  | 13:23 | <hashar> | updated labs/private on puppet master to fix a puppet dependency cycle with sudo-ldap | [releng] | 
            
  | 13:19 | <hashar> | rebased puppetmaster repo | [releng] | 
            
  | 12:52 | <hashar> | deleting the workspace for the beta-parsoid-update-eqiad jenkins job on deployment-parsoid04 . Some file belong to root which prevent the job from processing | [releng] | 
            
  | 09:13 | <hashar> | enabled MediaWiki core 'structure' PHPUnit tests for all extensions.  Will require folks to fix their incorrect AutoLoader and  RessourceLoader entries. {{gerrit|180496}} {{bug|T78798}} | [releng] |