401-450 of 3319 results (13ms)
2015-05-27 §
21:07 <AaronSchulz> Deleted 4G of logs on jobrunner01 [releng]
15:09 <hashar> Jenkins slaves are all back up. Root cause was some ssh algorithm in their sshd which is not supported by Jenkins jsch embedded lib. [releng]
14:30 <hashar> manually rebasing puppet git on deployment-salt (stalled) [releng]
14:27 <hashar> restarting deployment-salt / some process is 100% wa/IO [releng]
13:38 <hashar> restarted integration puppetmaster (memory leak) [releng]
13:35 <hashar> integration-puppetmaster apparently out of memory [releng]
13:30 <hashar> All Jenkins slaves are disconnected due to some ssh error. CI is down. [releng]
2015-05-24 §
18:39 <YuviKTM> purged old logs kept on NFS [releng]
10:27 <duh> deploying https://gerrit.wikimedia.org/r/213218 [releng]
2015-05-23 §
21:43 <legoktm> deploying https://gerrit.wikimedia.org/r/212960 [releng]
2015-05-20 §
20:58 <cscott> updated OCG to version ca4f64852de5b1de782b292b50038fbd2dd84266 [releng]
17:19 <thcipriani|afk> add --fail to curl inside mwext-Wikibase-qunit jenkins job [releng]
15:59 <bd808> Applied role::beta::puppetmaster on deployment-salt to get Puppet logstash reports back [releng]
2015-05-19 §
02:54 <bd808> Primed keyholder agent via `sudo -u keyholder env SSH_AUTH_SOCK=/run/keyholder/agent.sock ssh-add /etc/keyholder.d/mwdeploy_rsa` [releng]
02:40 <Krinkle> deployment-bastion.eqiad magically back online and catching up jobs, though failing due to T99644 [releng]
02:36 <Krinkle> Jenkins is unable to launch slave agent on deployment-bastion.eqiad. Using "Jenkins Script Console" throws HTTP 503. [releng]
02:30 <Krinkle> Various beta-mediawiki-config-update-eqiad jobs have been stuck for over 13 hours. [releng]
2015-05-18 §
15:17 <andrewbogott> rebooting deployment-logstash1 [releng]
2015-05-15 §
20:50 <andrewbogott> rebooted deployment-bastion due to inconsistent run state after suspend/resume [releng]
2015-05-13 §
21:08 <cscott> updated OCG to version c7c75e5b03ad9096571dc6dbfcb7022c924ccb4f [releng]
2015-05-12 §
15:18 <hashar> downgrading hhvm on CI slaves [releng]
15:10 <hashar> mediawiki-phpunit-hhvm Jenkins job is broken due to an hhvm upgrade {{bug|T98876}} [releng]
00:48 <bd808> beta cluster central syslog going to logstash rather than deployment-bastion (see https://gerrit.wikimedia.org/r/#/c/210253) [releng]
00:36 <bd808> Cherry-picked https://gerrit.wikimedia.org/r/#/c/210253/ [releng]
00:16 <legoktm> deploying https://gerrit.wikimedia.org/r/210251 [releng]
2015-05-11 §
22:50 <legoktm> deploying https://gerrit.wikimedia.org/r/210219 [releng]
22:29 <bd808> removed duplicate local group l10nupdate from deployment-bastion that was shadowing the ldap group of the same name [releng]
22:24 <bd808> removed duplicate local group mwdeploy from deployment-bastion that was shadowing the ldap group of the same name [releng]
22:15 <bd808> Removed role::logging::mediawiki from deployment-bastion [releng]
20:55 <legoktm> deleted operations-puppet-tox-py27 workspace on integration-slave-precise-1012, it was corrupt (fatal: loose object b48ccc3ef5be2d7252eb0f0f417f1b5b7c23fd5f (stored in .git/objects/b4/8ccc3ef5be2d7252eb0f0f417f1b5b7c23fd5f) is corrupt) [releng]
13:54 <hashar> Jenkins: removing label hasContintPackages from production slaves, it is no more needed :) [releng]
2015-05-09 §
00:10 <bd808> Cherry-picked https://gerrit.wikimedia.org/r/#/c/209830 to deployment-bastion:/srv/deployment/scap/scap and deployed with trebuchet [releng]
2015-05-08 §
23:59 <bd808> Created /data/project/logs/WHERE_DID_THE_LOGS_GO.txt to point folks to the right places [releng]
23:54 <bd808> Switched MediaWiki debug logs to deployment-fluorine:/srv/mw-log [releng]
20:05 <bd808> Cherry-picked https://gerrit.wikimedia.org/r/#/c/209801 [releng]
18:15 <bd808> Cherry-picked https://gerrit.wikimedia.org/r/#/c/209769/ [releng]
05:14 <bd808> apache2 access logs now only locally on instances in /var/log/apache2/other_vhosts_access.log; error log in /var/log/apache2.log and still relayed to deployment-bastion and logstash (works like production now) [releng]
04:49 <bd808> Symbolic link not allowed or link target not accessible: /srv/mediawiki/docroot/bits/static/master/extensions [releng]
04:47 <bd808> cherry-picked https://gerrit.wikimedia.org/r/#/c/209680/ [releng]
2015-05-07 §
20:48 <bd808> Updated kibana to bb9fcf6 (Merge remote-tracking branch 'upstream/kibana3') [releng]
18:00 <greg-g> brought deployment-bastion.eqiad back online in Jenkins (after Krinkle disconnected it some hours ago). Jobs are processing [releng]
16:05 <bd808> Updated scap to 5d681af (Better handling for php lint checks) [releng]
14:05 <Krinkle> deployment-bastion.eqiad has been stuck for 10 hours. [releng]
14:05 <Krinkle> As of two days now, Jenkins always returns Wikimedia 503 Error page after logging in. Log in session itself is fine. [releng]
05:02 <legoktm> slaves are going up/down likely due to automated labs migration script [releng]
2015-05-06 §
15:13 <bd808> Updated scap to 57036d2 (Update statsd events) [releng]
2015-05-05 §
19:06 <jzerebecki> integration-slave-trusty-1015:~$ sudo -u jenkins-deploy rm -rf /mnt/jenkins-workspace/workspace/mwext-Wikibase-qunit/src/node_modules [releng]
15:42 <legoktm> deploying https://gerrit.wikimedia.org/r/208975 & https://gerrit.wikimedia.org/r/208976 [releng]
04:36 <legoktm> deploying https://gerrit.wikimedia.org/r/208899 [releng]
04:04 <legoktm> deploying https://gerrit.wikimedia.org/r/208889,90,91,92 [releng]