9501-9550 of 10000 results (37ms)
2015-04-22 §
19:27 <hashar> Zuul gearman is stalled. Disabling Jenkins gearman client to free up connections [releng]
17:58 <Krinkle> Creating integration-slave-trusty-1021 per T96629 (using ci1.medium type) [releng]
14:34 <hashar> beta: failures on instances are due to them being moved on different openstack compute nodes (virt***) [releng]
13:51 <jzerebecki> integration-slave-trusty-1015:~$ sudo -u jenkins-deploy rm -rf /mnt/jenkins-workspace/workspace/mwext-Wikibase-qunit/src/node_modules [releng]
12:48 <andrewbogott> migrating to new labvirt nodes [releng]
12:48 <hashar> beta: Andrew B. starting to migrate beta cluster instances on new virt servers [releng]
11:34 <hashar> integration: apt-get upgrade on integration-slave-trusty* instances [releng]
11:31 <hashar> integration: Zuul package has been uploaded for Trusty! Deleting the .deb from /home/hashar/ [releng]
2015-04-21 §
09:27 <hashar> Nodepool created it is first instance ever! :) [releng]
08:33 <_joe_> rollback installation of hhvm 3.6 [releng]
08:09 <_joe_> installing HHVM 3.6 and the corresponding extensions on deployment-mediawiki01 [releng]
01:51 <legoktm> deploying https://gerrit.wikimedia.org/r/205494 [releng]
2015-04-20 §
23:34 <legoktm> deploying https://gerrit.wikimedia.org/r/205465 [releng]
19:20 <legoktm> mediawiki-extensions-hhvm workspace on integration-slave-trusty-1011 had bad lock file, wiping [releng]
16:10 <hashar> deployment-salt kill -9 of puppetmaster processes [releng]
16:08 <hashar> deployment-salt killed git-sync-upstream netcat to labmon1001.eqiad.wmnet 8125 was eating all memory [releng]
16:04 <hashar> beta: manually rebasing operations/puppet on deployment-salt . Might have killed some live hack in the process :/ [releng]
13:58 <hashar> In Gerrit, hidden integration/jenkins-job-builder-config and integration/zuul-config historical repositories. Suggest by addshore on {{bug:T96522}} [releng]
03:39 <legoktm> deploying https://gerrit.wikimedia.org/r/205174 [releng]
2015-04-19 §
06:12 <legoktm> deploying https://gerrit.wikimedia.org/r/205076 [releng]
2015-04-18 §
05:18 <legoktm> deploying https://gerrit.wikimedia.org/r/204995 [releng]
03:09 <Krinkle> Finished set up of integration-slave-trusty-1017. Pooled. [releng]
2015-04-17 §
17:52 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/204812 [releng]
17:45 <Krinkle> Creating integration-slave-trusty-1017 [releng]
16:29 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/204791 [releng]
16:00 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/204783 [releng]
12:42 <hashar> restarting Jenkins [releng]
12:38 <hashar> Switching zuul on lanthanum.eqiad.wmnet to the Debian package version [releng]
12:14 <hashar> Switching Zuul scheduler on gallium.wikimedia.org to the Debian package version [releng]
12:12 <hashar> Jenkins: enabled plugin "ZMQ Event Publisher" and publishing all jobs result on TCP port 8888 [releng]
05:37 <legoktm> deploying https://gerrit.wikimedia.org/r/204706 [releng]
01:11 <Krinkle> Repool integration-slave-precise-1013 and integration-slave-trusty-1015 (live hack with libeatmydata enabled for mysql; T96308) [releng]
2015-04-16 §
22:08 <Krinkle> Rebooting integration-slave-precise-1013 (depooled; experimenting with libeatmydata) [releng]
22:07 <Krinkle> Rebooted integration-slave-trusty-1015 (experimenting with libeatmydata) [releng]
18:31 <Krinkle> Rebooting integration-slave-precise-1012 and integration-slave-trusty-1012 [releng]
17:57 <Krinkle> Repooled instances. Converstion of mysql.datadir to tmpfs worked, but puppet run has errors. Coren and Krinkle working on it. https://gerrit.wikimedia.org/r/#/c/204528/ (T96230) [releng]
17:22 <Krinkle> Gracefully depool integration slaves to deploy https://gerrit.wikimedia.org/r/#/c/204528/ (T96230) [releng]
14:35 <thcipriani> running dpkg --configure -a on deployment-bastion to correct puppet failures [releng]
2015-04-15 §
23:21 <Krinkle> beta-update-databases-eqiad stuck waiting for executors on a node that has plenty executors available [releng]
21:15 <hashar> Jenkins browser test jobs sometime deadlock because of the IRC notification plugin https://phabricator.wikimedia.org/T96183 [releng]
20:34 <hashar> hard restarting Jenkins [releng]
19:24 <Krinkle> Aborting browser tests jobs. Stuck for over 5 hours. [releng]
19:24 <Krinkle> Aborting beta-scap-eqiad. Has been stuck for 2 hours on "Notifying IRC" after "Connection time out" from scap. [releng]
08:22 <hashar> restarted Jenkins [releng]
08:20 <hashar> Exception in thread "RequestHandlerThread[#2]" java.lang.OutOfMemoryError: Java heap space [releng]
08:16 <hashar> Jenkins process went wild taking all CPU busy on gallium [releng]
2015-04-14 §
20:43 <legoktm> starting SULF on beta cluster [releng]
20:42 <marktraceur> stopping all beta jobs, aborting running (and stuck) beta DB update, kicking bastion, to try and get beta to update [releng]
19:49 <Krinkle> All systems go. [releng]
19:48 <Krinkle> Jenkins configuration panel won't load ("Loading..." stays indefine, "Uncaught TypeError: Cannot convert to object at prototype.js:195") [releng]