5301-5350 of 7640 results (25ms)
2015-03-03 §
18:28 <Krinkle> Lots of Jenkins builds are stuck half-way executing. No clear cause. Everything seems up. [releng]
17:18 <Krinkle> Reloading Zuul to deploy Icad0a26dc8 and Icac172b16 [releng]
15:39 <hashar> cancelled logrotate update of all jobs since that seems to kill the Jenkins/Zuul gearman connection. Probably because all jobs are registered on each config change. [releng]
15:31 <hashar> updating all jobs in Jenkins based on PS2 of https://gerrit.wikimedia.org/r/194109 [releng]
10:56 <hashar> Created instance i-000008fb with image "ubuntu-14.04-trusty" and hostname i-000008fb.eqiad.wmflabs. [releng]
10:52 <hashar> deleting integration-puppetmaster to recreate it with a new image {bug|T87484} . Will have to reapply I5335ea7cbfba33e84b3ddc6e3dd83a7232b8acfd and I30e5bfeac398e0f88e538c75554439fe82fcc1cf [releng]
03:47 <Krinkle> git-deploy: Deploying integration/slave-scripts 05a5593..1e64ed9 [releng]
01:11 <marxarelli> gzip'd /var/log/account/pacct.0 on deployment-bastion to free space [releng]
2015-03-02 §
21:35 <twentyafterfour> <Krenair> (per #mediawiki-core, have deleted the job queue key in redis, should get regenerated. also cleared screwed up log and restarted job runner service) [releng]
15:39 <Krinkle> Removing /usr/local/src/zuul from integration-slave12xx and integration-slave14xx to let puppet re-install zuul-cloner (T90984) [releng]
13:39 <Krinkle> integration-slave12xx and integration-slave14xx instances still depooled due to T90984 [releng]
2015-02-27 §
21:58 <Krinkle> Ragekilled all queued jobs related to beta and force restarted Jenkins slave agent on deployment-bastion.eqiad [releng]
21:56 <Krinkle> Job beta-update-databases-eqiad and node deployment-bastion.eqiad have been stuck for the past 4 hours [releng]
21:49 <marxarelli> Reloading Zuul to deploy I273270295fa5a29422a57af13f9e372bced96af1 and I81f5e785d26e21434cd66dc694b4cfe70c1fa494 [releng]
18:08 <Krenair> Kicked deployment-bastion node in jenkins to try to fix jobs [releng]
11:12 <YuviPanda> start mysql on deployment-db1 [releng]
06:42 <legoktm> deployed https://gerrit.wikimedia.org/r/193057 [releng]
01:01 <Krinkle> Keeping all integration-slave12xx and slave14xx instances depooled. [releng]
00:53 <Krinkle> Finished provisioning of integration-slave12xx and slave14xx instance. Initial testing failed due to "/usr/local/bin/zuul-cloner: No such file or directory" [releng]
2015-02-26 §
23:24 <Krinkle> integration-puppetmaster /var disk is full (1.8 of 1.9GB) - /var/log/puppet/reports is 1.1GB - purging [releng]
23:23 <Krinkle> Puppet failing on new instances due to "Error 400 on SERVER: cannot generate tempfile `/var/lib/puppet/yaml/" [releng]
13:27 <Krinkle> Provisioning the new integration-slave12xx and integration-slave14xx instances [releng]
11:53 <YuviPanda> created deployment-parsoid01-test to test patch to use role::parsoid on labs [releng]
05:05 <legoktm> deployed https://gerrit.wikimedia.org/r/192980 [releng]
03:48 <Krinkle> Creating integration-slave1201,02,03,04 and integration-slave1401,02,03,04,05 per T74011 (not yet setup/provisioned, keep depooled) [releng]
03:39 <Krinkle> Cleaned up and re-pooled integration-slave1006 (was depooled since yesterday) [releng]
03:39 <Krinkle> Cleaned up and re-pooled integration-slave1007 and integration-slave1008 (was auto-depooled by Jenkins) [releng]
01:54 <Krinkle> integration-slave1007 and integration-slave1008 were auto-deplooed due to main disk (/ and its /tmp) being < 900 MB free [releng]
01:20 <legoktm> actually deployed https://gerrit.wikimedia.org/r/192772 this time [releng]
01:16 <legoktm> deployed https://gerrit.wikimedia.org/r/192772 [releng]
2015-02-25 §
23:55 <Krinkle> Re-established Jenkins-Gearman connection [releng]
23:54 <Krinkle> Zuul queue is growing. Nothing is added to its dashboard. Jenkins executers all idle. Gearman deadlock? [releng]
20:38 <legoktm> deployed https://gerrit.wikimedia.org/r/192564 [releng]
20:18 <legoktm> deployed https://gerrit.wikimedia.org/r/192267 [releng]
17:22 <^d> reloading zuul to pick up utfnormal jobs [releng]
02:15 <Krinkle> integration-slave1006 has <700MB free disk space (including /tmp) [releng]
2015-02-24 §
18:41 <marxarelli> Running `jenkins-jobs update` to create browsertests-CentralAuth-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce [releng]
17:55 <Krinkle> It seems xdebug was enabled on integration slaves running trusty. This makes errors in build logs incomprehensible. [releng]
2015-02-21 §
03:01 <Krinkle> Reloading Zuul to deploy I3bcd3d17cb886740bd67b33b573aa25972ddb574 [releng]
2015-02-20 §
07:25 <Krinkle> Finished setting up integration-slave1010 and added it to Jenkins slave pool [releng]
00:54 <Krinkle> Setting up integration-slave1010 (replacement for integration-slave1009) [releng]
2015-02-19 §
23:13 <bd808> added Thcipriani to under_NDA sudoers group; WMF staff [releng]
19:45 <Krinkle> Destroying integration-slave1009 and re-imaging [releng]
19:02 <bd808> VICTORY! deployment-bastion jenkins slave unstuck [releng]
19:01 <bd808> toggling gearman plugin in jenkins admin console [releng]
18:59 <bd808> took deployment-bastion jenkins connection offline and online 5 times; gearman plugin still stuck [releng]
18:41 <bd808> cleaned up mess in /tmp on integration-slave1008 [releng]
18:38 <bd808> brought integration-slave1007 back online [releng]
18:37 <bd808> cleaned up mess in /tmp on integration-slave1007 [releng]
18:29 <bd808> restarting jenkins because I messed up and disabled gearman plugin earlier [releng]