|
2016-02-09
§
|
| 22:18 |
<legoktm> |
re-enabling puppet on all CI slaves |
[releng] |
| 22:02 |
<legoktm> |
reloading zuul to see if it'll pickup the new composer-php53 job |
[releng] |
| 21:53 |
<legoktm> |
enabling puppet on just integration-slave-trusty-1012 |
[releng] |
| 21:52 |
<legoktm> |
cherry-picked https://gerrit.wikimedia.org/r/#/c/269370/ onto integration-puppetmaster |
[releng] |
| 21:50 |
<legoktm> |
disabling puppet on all trusty/precise CI slaves |
[releng] |
| 21:40 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/269533 |
[releng] |
| 17:49 |
<marxarelli> |
disabled/enabled gearman in jenkins, connection works this time |
[releng] |
| 17:49 |
<marxarelli> |
performed stop/start of zuul on gallium to restore zuul and gearman |
[releng] |
| 17:45 |
<marxarelli> |
"Failed: Unable to Connect" in jenkins when testing gearman connection |
[releng] |
| 17:40 |
<marxarelli> |
killed old zull process manually and restarted service |
[releng] |
| 17:39 |
<marxarelli> |
restart of zuul fails as well. old process cannot be killed |
[releng] |
| 17:38 |
<marxarelli> |
reloading zuul fails with "failed to kill 13660: Operation not permitted" |
[releng] |
| 16:06 |
<bd808> |
Deleted corrupt integration-slave-precise-1003:/mnt/jenkins-workspace/workspace/mediawiki-core-php53lint/.git |
[releng] |
| 15:11 |
<hashar> |
mira: /srv/mediawiki-staging/multiversion/checkoutMediaWiki 1.27.0-wmf.13 php-1.27.0-wmf.13 |
[releng] |
| 14:51 |
<hashar> |
./make-wmf-branch -n 1.27.0-wmf.13 -o master |
[releng] |
| 14:50 |
<hashar> |
pooling back integration-slave-precise1001 - 1004. Manually fetched git repos in workspace for mediawiki core php53 |
[releng] |
| 14:49 |
<hashar> |
make-wmf-branch instance: created a local ssh key pair and set the config to use User: hashar |
[releng] |
| 14:13 |
<hashar> |
pooling https://integration.wikimedia.org/ci/computer/integration-slave-precise-1012/ Mysql is back .. Blame puppet |
[releng] |
| 14:12 |
<hashar> |
de pooling https://integration.wikimedia.org/ci/computer/integration-slave-precise-1012/ Mysql is gone somehow |
[releng] |
| 14:04 |
<hashar> |
Manually git fetching mediawiki-core in /mnt/jenkins-workspace/workspace/mediawiki-core-php53lint of slaves precise 1001 to 1004 (git on Precise is remarkably too slow) |
[releng] |
| 13:28 |
<hashar> |
salt '*trusty*' cmd.run 'update-alternatives --set php /usr/bin/hhvm' |
[releng] |
| 13:28 |
<hashar> |
salt '*precise*' cmd.run 'update-alternatives --set php /usr/bin/php5' |
[releng] |
| 13:17 |
<hashar> |
salt -v --batch=3 '*slave*' cmd.run 'puppet agent -tv' |
[releng] |
| 13:15 |
<hashar> |
removing https://gerrit.wikimedia.org/r/#/c/269370/ from CI puppet master |
[releng] |
| 13:14 |
<hashar> |
slave recurse infinitely doing /bin/bash -eu /srv/deployment/integration/slave-scripts/bin/mw-install-mysql.sh then loop over /bin/bash /usr/bin/php maintenance/install.php --confpath /mnt/jenkins-workspace/workspace/mediawiki-core-qunit/src --dbtype=mysql --dbserver=127.0.0.1:3306 --dbuser=jenkins_u2 --dbpass=pw_jenkins_u2 --dbname=jenkins_u2_mw --pass testpass TestWiki WikiAdmin https://phabricator.wikimedia.org/T126327 |
[releng] |
| 12:46 |
<hashar> |
Mass testing php loop of death: salt -v '*slave*' cmd.run 'timeout 2s /srv/deployment/integration/slave-scripts/bin/php --version' |
[releng] |
| 12:40 |
<hashar> |
mass rebooting CI slaves from wikitech |
[releng] |
| 12:39 |
<hashar> |
salt -v '*' cmd.run "bash -c 'cd /srv/deployment/integration/slave-scripts; git pull'" |
[releng] |
| 12:33 |
<hashar> |
all slaves dieing due to PHP looping |
[releng] |
| 12:02 |
<legoktm> |
re-enabling puppet on all trusty/precise slaves |
[releng] |
| 11:20 |
<legoktm> |
cherry-picked https://gerrit.wikimedia.org/r/#/c/269370/ on integration-puppetmaster |
[releng] |
| 11:20 |
<legoktm> |
enabling puppet just on integration-slave-trusty-1012 |
[releng] |
| 11:13 |
<legoktm> |
disabling puppet on all *(trusty|precise)* slaves |
[releng] |
| 10:25 |
<hashar> |
pooling in integration-slave-trusty-1018 |
[releng] |
| 03:19 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/269359 |
[releng] |
| 02:53 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/238988 |
[releng] |
| 00:39 |
<hashar> |
gallium edited /usr/share/python/zuul/local/lib/python2.7/site-packages/zuul/trigger/gerrit.py and modified: replication_timeout = 300 -> replication_timeout = 10 |
[releng] |
| 00:37 |
<hashar> |
live hacking Zuul code to have it stop sleeping() on force merge |
[releng] |
| 00:36 |
<hashar> |
killing zuul |
[releng] |
|
2016-02-08
§
|
| 23:48 |
<legoktm> |
finally deploying https://gerrit.wikimedia.org/r/269327 |
[releng] |
| 23:14 |
<hashar> |
zuul promote --pipeline gate-and-submit --changes 269065,2 https://gerrit.wikimedia.org/r/#/c/269065/ |
[releng] |
| 23:10 |
<hashar> |
pooling integration-slave-precise-1001 1002 1004 |
[releng] |
| 22:47 |
<hashar> |
Err need to reboot newly provisioned instances before adding them to Jenkins (kernel upgrade,apache restart etc) |
[releng] |
| 22:45 |
<hashar> |
Pooled https://integration.wikimedia.org/ci/computer/integration-slave-precise-1003/ |
[releng] |
| 22:25 |
<hashar> |
integration-slave-precise-{1001-1004} applied role::ci::slave::labs, running puppet in slaves. I have added the instances as Jenkins slaves and put them offline. Whenever puppet is done, we can mark them online in Jenkins then monitor the jobs running on them are working properly |
[releng] |
| 22:15 |
<hashar> |
Provisioning integration-slave-precise-{1001-1004} https://phabricator.wikimedia.org/T126274 (need more php53 slots) |
[releng] |
| 22:13 |
<hashar> |
Deleted cache-rsync instance superseded by castor instance |
[releng] |
| 22:10 |
<hashar> |
Deleting pmcache.integration.eqiad.wmflabs (was to investigate various kind of central caches). |
[releng] |
| 20:14 |
<marxarelli> |
aborting pending mediawiki-extensions-php53 job for CheckUser |
[releng] |
| 20:08 |
<bd808> |
toggled "Enable Gearman" off and on in Jenkins to wake up deployment-bastion workers |
[releng] |