|
2015-07-28
§
|
| 23:43 |
<marxarelli> |
running `jenkins-jobs update config/ 'mwext-mw-selenium'` to deploy I7afa07e9f559bffeeebaf7454cc6b39a37e04063 |
[releng] |
| 21:05 |
<bd808> |
upgraded nutcracker on mediawiki03 |
[releng] |
| 21:04 |
<bd808> |
upgraded nutcracker on mediawiki02 |
[releng] |
| 21:01 |
<bd808> |
upgraded nutcracker on mediawiki01 |
[releng] |
| 19:49 |
<jzerebecki> |
reloading zuul b1b2cab..b02830e |
[releng] |
| 11:18 |
<hashar> |
Assigning label "BetaClusterBastion" to https://integration.wikimedia.org/ci/computer/deployment-bastion.eqiad/ |
[releng] |
| 11:12 |
<hashar> |
Jenkins jobs for the beta cluster ended up stuck again. Found a workaround by removing the Jenkins label on deployment-bastion node and reinstating it. Seems to get rid of the deadlock ( ref: https://phabricator.wikimedia.org/T72597#1487801 ) |
[releng] |
| 09:50 |
<hashar> |
deployment-apertium01 is back! The ferm rules were outdated / not maintained by puppet, dropped ferm entirely. |
[releng] |
| 09:40 |
<hashar> |
rebooting deployment-apertium01 to ensure its ferm rules are properly loaded on boot ( https://phabricator.wikimedia.org/T106658 ) |
[releng] |
| 00:46 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/227383 |
[releng] |
|
2015-07-24
§
|
| 17:35 |
<marxarelli> |
updating integration slave scripts from integration-saltmaster to deploy I6906fadede546ce2205797da1c6b267aed586e17 |
[releng] |
| 17:17 |
<marxarelli> |
running `jenkins-jobs update config/ 'mediawiki-selenium-integration' 'mwext-mw-selenium'` to deploy Ib289d784c7b3985bd4823d967fbc07d5759dc756 |
[releng] |
| 17:05 |
<marxarelli> |
running `jenkins-jobs update config/ 'mediawiki-selenium-integration'` to deploy and test Ib289d784c7b3985bd4823d967fbc07d5759dc756 |
[releng] |
| 17:04 |
<hashar> |
integration-saltmaster, in a '''screen''' : salt -b 1 '*slave*' cmd.run '/usr/local/sbin/puppet-run'|tee hashar-massrun.log |
[releng] |
| 17:04 |
<hashar> |
cancelled last command |
[releng] |
| 17:03 |
<hashar> |
integration-saltmaster : salt -b 1 '*slave*' cmd.run '/usr/local/sbin/puppet-run' & && disown && exit |
[releng] |
| 16:55 |
<hashar> |
Might have fixed the puppet/pip mess on CI slaves by creating a symlink from /usr/bin/pip to /usr/local/bin/pip ( https://gerrit.wikimedia.org/r/#/c/226729/1..2/modules/contint/manifests/packages/python.pp,unified ) |
[releng] |
| 16:36 |
<hashar> |
puppet on Jenkins slaves might have some intermittent issues due to pip installation https://gerrit.wikimedia.org/r/226729 |
[releng] |
| 15:29 |
<hashar> |
removing pip obsolete download-cache setting ( https://gerrit.wikimedia.org/r/#/c/226730/ ) |
[releng] |
| 15:27 |
<hashar> |
upgrading pip to 7.1.0 via pypi ( https://gerrit.wikimedia.org/r/#/c/226729/ ). Revert plan is to uncherry pick the patch on the puppetmaster and: pip uninstall pip |
[releng] |
| 12:46 |
<hashar> |
Jenkins: switching gearman plugin from our custom compiled 0.1.1-9-g08e9c42-change_192429_2 to upstream 0.1.2. They are actually the exact same versions. |
[releng] |
| 08:40 |
<hashar> |
upgrading zuul to zuul_2.0.0-327-g3ebedde-wmf3precise1 to fix a regression ( https://phabricator.wikimedia.org/T106531 ) |
[releng] |
| 08:39 |
<hashar> |
upgrading zuul |
[releng] |
|
2015-07-23
§
|
| 23:03 |
<marxarelli> |
running `jenkins-jobs update config/ 'browsertests-*'` to deploy I2d0f83d0c6a406d46627578cb8db0706d1b8655d |
[releng] |
| 16:38 |
<marxarelli> |
Reloading Zuul to deploy I96b6218a208f133209452c71bcf01a1088305aea |
[releng] |
| 15:40 |
<urandom> |
applied wip logstash & cassandra changes (https://gerrit.wikimedia.org/r/#/c/226025/) to deployment-prep |
[releng] |
| 13:24 |
<hashar> |
apt-get upgrade integration-puppetmaster and rebooting it |
[releng] |
| 13:23 |
<hashar> |
integration puppetmaster in bad shape: Warning: Error 400 on SERVER: Cannot allocate memory - fork(2) |
[releng] |
| 10:58 |
<hashar> |
beta : salt '*' cmd.run 'rm /etc/apt/apt.conf.d/20auto-upgrades.ucf-dist' |
[releng] |
| 10:52 |
<hashar> |
Beta cluster puppetmaster is now deployment-puppetmaster.deployment-prep.eqiad.wmflabs . Migrated all instances (solves https://phabricator.wikimedia.org/T106649 ) |
[releng] |
| 10:30 |
<hashar> |
regenerated puppet cert on deployment-salt , the old puppetmaster now a puppet client |
[releng] |
| 10:23 |
<hashar> |
running apt-get upgrade on deployment-parsoidcache02 |
[releng] |
| 09:32 |
<hashar> |
puppet broken on deployment-fluorine : Error: Could not request certificate: Neither PUB key nor PRIV key:: header too long |
[releng] |
| 08:39 |
<hashar> |
Disabling puppet agent on ALL beta cluster instances |
[releng] |
| 08:18 |
<hashar> |
creating deployment-puppetmaster m1.medium :D |
[releng] |
| 01:57 |
<jzerebecki> |
reconnected slave and needed to kill a few pending beta jobs, works again |
[releng] |
| 01:50 |
<jzerebecki> |
trying https://www.mediawiki.org/wiki/Continuous_integration/Jenkins#Hung_beta_code.2Fdb_update |
[releng] |
| 01:09 |
<legoktm> |
beta-mediawiki-config-update-eqiad jobs stuck |
[releng] |
| 00:15 |
<jzerebecki> |
reloading zuul 369e6eb..73dc1f6 for https://gerrit.wikimedia.org/r/#/c/223527/ |
[releng] |