2751-2800 of 7896 results (16ms)
2016-09-16 §
21:03 <hashar> deployment-tin did a git gc on /srv/deployment/ores That freed up disk space and cleared an alarm on co master mira02 [releng]
21:00 <hashar> deleted deployment-parsoid05 [releng]
20:52 <hashar> fixed puppet on deployment-parsoid05 . Temporary instance will delete it later to clear out shinken.wmflabs.org [releng]
20:27 <hashar> beta: force running puppet in batches of 4 instances: salt --batch 4 -v 'deployment-*' cmd.run 'puppet agent -tv' [releng]
20:13 <hashar> beta: restarted puppetmaster [releng]
20:07 <hashar> beta: salt -v '*' cmd.run 'rm -fR /var/lib/puppet/client/ssl/' [releng]
20:07 <hashar> beta: stopping puppetmaster, rm -f /var/lib/puppet/server/ssl/ca/signed/* [releng]
19:53 <hashar> beta created instance "deployment-parsoid05" Should be deleted later, that is merely to purge the hostname from Shinken ( http://shinken.wmflabs.org/host/deployment-parsoid05 ) [releng]
11:42 <hashar> beta: apt-get upgrade on deployment-jobrunner01 [releng]
11:36 <hashar> apt-get upgrade on deployment-tin , bring in a new hhvm version and others [releng]
2016-09-15 §
22:29 <legoktm> sudo salt '*precise*' cmd.run 'service mysql start', all mysql's are down [releng]
16:45 <godog> install xenial kernel on deployment-zotero01 and reboot T145793 [releng]
16:18 <hashar> prometheus enabled on all beta cluster instance. Does not support Precise hence puppet will fail on the last two Precise instances deployment-db1 and deployment-db2 until they are migrated to Jessie T138778 [releng]
15:53 <godog> add role::prometheus::node_exporter to classes in hiera:deployment-prep T144502 [releng]
15:10 <hashar> beta: Applying puppet class role::prometheus::node_exporter to mira02 just like mira. That is for godog [releng]
15:08 <hashar> T144006 Disabled Jenkins job beta-scap-eqiad. On mira02 rm -fR /srv/* . Applying puppet for role::labs::lvm::srv [releng]
15:05 <hashar> T144006 Applying class role::labs::lvm::srv to mira02 (it is out of disk space :D ) [releng]
14:45 <hashar> T144006 sudo -u jenkins-deploy -H SSH_AUTH_SOCK=/run/keyholder/proxy.sock ssh mwdeploy@mira02.deployment-prep.eqiad.wmflabs [releng]
14:44 <hashar> T144006 sudo -u jenkins-deploy -H SSH_AUTH_SOCK=/run/keyholder/proxy.sock ssh mwdeploy@deployment-mediawiki05.deployment-prep.eqiad.wmflabs [releng]
12:33 <elukey> added base::firewall, beta::deployaccess, mediawiki::conftool, role::mediawiki::appserver to mediawiki05 [releng]
12:20 <elukey> terminate mediawiki02 to create mediawiki05 [releng]
10:48 <hashar> beta: cherry picking moritzm patch https://gerrit.wikimedia.org/r/#/c/310793/ "Also handle systemd in keyholder script" T144578 [releng]
09:33 <hashar> T144006 sudo -u jenkins-deploy -H SSH_AUTH_SOCK=/run/keyholder/proxy.sock ssh mwdeploy@deployment-mediawiki06.deployment-prep.eqiad.wmflabs [releng]
09:10 <elukey> executed git pull and then git rebase -i on deployment puppet master [releng]
08:52 <elukey> terminated mediawiki03 and created mediawiki06 [releng]
08:45 <elukey> removed mediawiki03 from puppet with https://gerrit.wikimedia.org/r/#/c/310749/ [releng]
02:36 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/310701 [releng]
2016-09-14 §
21:37 <hashar> integration: setting "ulimit -c 2097152" on all slaves due to Zend PHP segfaulting T142158 [releng]
15:21 <godog> cherry-pick https://gerrit.wikimedia.org/r/#/c/310557/ on puppet master [releng]
14:31 <hashar> Added otto to integration labs project [releng]
13:28 <gehel> upgrading deployment-logstash2 to elasticsearch 2.3.5 - T145404 [releng]
09:27 <hashar> Deleting deployment-mediawiki01 , replaced by deployment-mediawiki04 T144006 [releng]
07:19 <legoktm> sudo salt '*trusty*' cmd.run 'service mysql start', it was down on all trusty salves [releng]
07:17 <legoktm> mysql just died on a bunch of slaves (trusty-1013, 1012, 1001) [releng]
2016-09-13 §
20:47 <Krenair> Created SRV record _etcd._tcp.beta.wmflabs.org for etcd/confd [releng]
17:02 <marxarelli> re-enabling beta cluster jenkins jobs following maintenance window [releng]
16:59 <marxarelli> aborting beta cluster db migration due to time constraints and ops outage. will reschedule [releng]
15:34 <marxarelli> disabled beta jenkins builds while in maintenance mode [releng]
2016-09-12 §
14:41 <elukey> applied base::firewall, beta::deployaccess, mediawiki::conftool, role::mediawiki::appserver to deployment-mediawiki04.deployment-prep.eqiad.wmflabs (Debian jessie instance) - T144006 [releng]
12:50 <gehel> rolling back upgrading elasticsearch to 2.4.0 on deployment-elastic05 - T145058 [releng]
12:03 <gehel> upgrading elasticsearch to 2.4.0 on deployment-elastic0? - T145058 [releng]
12:01 <hashar> Gerrit: made analytics-wmde group to be owned by themselves [releng]
11:57 <hashar> Gerrit: added ldap/wmde as an included group of the 'wikidata' group. Asked by and demoed to addshore [releng]
2016-09-11 §
20:35 <Krenair> started cron service on deployment-salt02 again, seems it got killed Tue 2016-08-30 13:42:39 UTC - hopefully this will fix the puppet staleness alert [releng]
18:45 <legoktm> deploying https://gerrit.wikimedia.org/r/309829 [releng]
2016-09-09 §
20:53 <thcipriani> testing scap 3.2.5-1 on beta cluster [releng]
11:08 <hashar> Added git tag for latest versions of mediawiki/selenium and mediawiki/ruby/api [releng]
09:30 <legoktm> Image ci-jessie-wikimedia-1473412532 in wmflabs-eqiad is ready [releng]
08:53 <legoktm> added phpflavor-php70 label to integration-slave-jessie-100[1-5] [releng]
08:49 <legoktm> deploying https://gerrit.wikimedia.org/r/309048 [releng]