751-800 of 4828 results (13ms)
2016-02-02 §
00:42 <legoktm> marked integration-slave-precise-1011 as offline [releng]
00:39 <legoktm> precise-1011 slave hasn't had a puppet run in 6 days [releng]
2016-02-01 §
23:53 <bd808> Logstash working again; I applied a change to the default mapping template for Elasticsearch that ensures that fields named "timestamp" are indexed as plain strings [releng]
23:46 <bd808> Elasticsearch index template for beta logstash cluster making crappy guesses about syslog events; dropped 2016-02-01 index; trying to fix default mappings [releng]
23:08 <bd808> HHVM logs causing rejections during document parse when inserting in Elasticsearch from logstash. They contain a "timestamp" field that looks like "Feb 1 22:56:39" which is making the mapper in Elasticsearch sad. [releng]
23:02 <bd808> Elasticsearch on deployment-logstash2 rejecting all documents with 400 status. Investigating [releng]
22:50 <bd808> Copying deployment-logstash2.deployment-prep:/var/log/logstash/logstash.log to /srv for debugging later [releng]
22:48 <bd808> deployment-logstash2.deployment-prep:/var/log/logstash/logstash.log is 11G of fail! [releng]
22:46 <bd808> root partition on deployment-logstash2 full [releng]
22:43 <bd808> No data in logstash since 2016-01-30T06:55:37.838Z; investigating [releng]
15:33 <hashar> Image ci-jessie-wikimedia-1454339883 in wmflabs-eqiad is ready [releng]
15:01 <hashar> Refreshing Nodepool image. Might have npm/grunt properly set up [releng]
03:15 <legoktm> deploying https://gerrit.wikimedia.org/r/267630 [releng]
2016-01-29 §
18:42 <thcipriani> updated scap on beta [releng]
16:44 <thcipriani> deployed scap updates on beta [releng]
11:58 <_joe_> upgraded hhvm to 3.6 wm8 in deployment-prep [releng]
2016-01-28 §
23:22 <MaxSem> Updated portals on betalabs to master [releng]
22:23 <hashar> salt '*slave-precise*' cmd.run 'apt-get install php5-ldap' ( https://phabricator.wikimedia.org/T124613 ) will need to be puppetized [releng]
18:17 <thcipriani> cleaning npm cache on slave machines: salt -v '*slave*' cmd.run 'sudo -i -u jenkins-deploy -- npm cache clean' [releng]
18:12 <thcipriani> running npm cache clean on integration-slave-precise-1011 sudo -i -u jenkins-deploy -- npm cache clean [releng]
15:25 <hashar> apt-get upgrade deployment-sca01 and deployment-sca02 [releng]
15:09 <hashar> fixing puppet.conf hostname on deployment-upload deployment-conftool deployment-tmh01 deployment-zookeeper01 and deployment-urldownloader [releng]
15:06 <hashar> fixing puppet.con hostname on deployment-upload.deployment-prep.eqiad.wmflabs and running puppet [releng]
15:00 <hashar> Running puppet on deployment-memc02 and deployment-elastic07 . It is catching up with lot of changes [releng]
14:59 <hashar> fixing puppet hostnames on deployment-elastic07 [releng]
14:59 <hashar> fixing puppet hostnames on deployment-memc02 [releng]
14:55 <hashar> Deleted salt keys deployment-pdf01.eqiad.wmflabs and deployment-memc04.eqiad.wmflabs (obsolete, entries with '.deployment-prep.' are already there) [releng]
07:38 <jzerebecki> reload zuul for 4951444..43a030b [releng]
05:55 <jzerebecki> doing https://www.mediawiki.org/wiki/Continuous_integration/Jenkins#Hung_beta_code.2Fdb_update [releng]
03:49 <mobrovac> deployment-prep re-enabled puppet on deployment-restbase0x [releng]
02:49 <mobrovac> deployment-prep deployment-restbase01 disabled puppet to set up cassandra for [releng]
02:27 <mobrovac> deployment-prep recreating deployment-restbase01 for T125003 [releng]
02:23 <mobrovac> deployment-prep deployment-restbase02 disabled puppet to recreate deployment-restbase01 for T125003 [releng]
01:42 <mobrovac> deployment-prep recreating deployment-sca02 for T125003 [releng]
01:28 <mobrovac> deployment-prep recreating deployment-sca01 for T125003 [releng]
00:36 <mobrovac> deployment-prep re-imaging deployment-mathoid for T125003 [releng]
00:02 <jzerebecki> integration-slave-trusty-1016:~$ sudo -i rm -rf /mnt/jenkins-workspace/workspace/mwext-testextension-hhvm/src/skins/Donate [releng]
2016-01-27 §
23:49 <jzerebecki> integration-slave-precise-1011:~$ sudo -i /etc/init.d/salt-minion restart [releng]
23:46 <jzerebecki> work around https://phabricator.wikimedia.org/T117710 : salt --show-timeout '*slave*' cmd.run 'rm -rf /mnt/jenkins-workspace/workspace/mwext-testextension-hhvm/src/skins/BlueSky' [releng]
21:19 <cscott> updated OCG to version 64050af0456a43344b32e3e93561a79207565eaf (should be no-op after yesterday's deploy) [releng]
10:29 <hashar> triggered bunch of browser tests, deployment-redis01 was dead/faulty [releng]
10:08 <hashar> mass restarting redis-server process on deployment-redis01 (for https://phabricator.wikimedia.org/T124677 ) [releng]
10:07 <hashar> mass restarting redis-server process on deployment-redis01 [releng]
09:00 <hashar> beta: commenting out "latency-monitor-threshold 100" parameter from any /etc/redis/redis.conf we have ( https://phabricator.wikimedia.org/T124677 ). Puppet will not reapply it unless distribution is Jessie [releng]
2016-01-26 §
16:51 <cscott> updated OCG to version 64050af0456a43344b32e3e93561a79207565eaf [releng]
12:14 <hashar> Added Jenkins IRC bot (wmf-insecte) to #wikimedia-perf for https://gerrit.wikimedia.org/r/#/c/265631/ [releng]
09:30 <hashar> restarting Jenkins to upgrade the gearman plugin with https://review.openstack.org/#/c/271543/ [releng]
04:18 <bd808> integration-slave-jessie-1001:/mnt full; cleaned up 15G of files in /mnt/pbuilder/build (27 hours after the last time I did that) [releng]
2016-01-25 §
18:59 <twentyafterfour> started redis-server on deployment-redis01 by commenting out latency-monitor-threshold from the redis.conf [releng]
15:22 <hashar> CI: fixing kernels not upgrading via: rm /boot/grub/menu.lst ; update-grub -y (i.e.: regenerate the Grub menu from scratch) [releng]