51-100 of 100 results (9ms)
2016-02-01 §
22:48 <bd808> deployment-logstash2.deployment-prep:/var/log/logstash/logstash.log is 11G of fail! [releng]
22:43 <bd808> No data in logstash since 2016-01-30T06:55:37.838Z; investigating [releng]
2015-08-11 §
00:45 <bd808> Logstash properly creating new indices again and logs are being collected [releng]
00:42 <bd808> Fixed default mapping for logstash indices to have 0 replicas [releng]
00:40 <bd808> Crap! logstash errors are my fault. I updated the default index mapping and neglected to correct the replica count. Missing all data from 2015-08-08 to now [releng]
00:36 <bd808> Elasticsearch index for logstash-2015.08.10 missing/corrupt [releng]
00:32 <bd808> Started logstash on deployment-logstash2; process had died from OOM [releng]
2015-08-07 §
17:55 <bd808> logstash-beta.wmflabs.org working again; broken since Ib10deb5 was merged [releng]
2015-08-06 §
17:01 <bd808> fixed rebase conflict with logstash cherry-picks [releng]
2015-08-04 §
18:01 <bd808> Upgraded logstash to 1.5.3 on deployment-logstash2 [releng]
2015-07-23 §
15:40 <urandom> applied wip logstash & cassandra changes (https://gerrit.wikimedia.org/r/#/c/226025/) to deployment-prep [releng]
2015-07-11 §
04:18 <bd808> Logstash cluster upgrade complete! Kibana working again [releng]
2015-07-08 §
22:00 <bd808> Kibana messed up. Half of the logstash elasticsearch indices are gone from deployment-logstash2 [releng]
2015-06-29 §
23:19 <bd808> Moved logstash irc bot from logstash1 to logstash2 [releng]
2015-06-05 §
23:55 <bd808> added deployment-logstash2 host and told cluster to move logstash all data there [releng]
2015-06-03 §
11:58 <kart_> Cherry-picked 213840 to test logstash [releng]
2015-05-20 §
15:59 <bd808> Applied role::beta::puppetmaster on deployment-salt to get Puppet logstash reports back [releng]
2015-05-12 §
00:48 <bd808> beta cluster central syslog going to logstash rather than deployment-bastion (see https://gerrit.wikimedia.org/r/#/c/210253) [releng]
2015-05-08 §
05:14 <bd808> apache2 access logs now only locally on instances in /var/log/apache2/other_vhosts_access.log; error log in /var/log/apache2.log and still relayed to deployment-bastion and logstash (works like production now) [releng]
2015-04-23 §
00:29 <bd808> cherry-picked and applied https://gerrit.wikimedia.org/r/#/c/205969/ (logstash: Convert $::realm switches to hiera) [releng]
2015-04-22 §
23:57 <bd808> cherry-picked and applied https://gerrit.wikimedia.org/r/#/c/205968 (remove redis from logstash) [releng]
2014-12-19 §
18:23 <bd808> redis input to logstash stuck; restarted service [releng]
2014-12-12 §
18:35 <bd808> Added puppet config to record !log messages in logstash [releng]
2014-12-10 §
22:17 <bd808> restarted logstash on logstash1001. redis event queue not being processed [releng]
2014-12-01 §
17:59 <bd808> Testing rsyslog event forwarding to logstash via puppet cherry-pick [releng]
2014-11-26 §
00:51 <bd808> cherry-picked patch for redis logstash input from MW {{gerrit|175896}} [releng]
2014-11-14 §
20:02 <anomie> Cherry-picking https://gerrit.wikimedia.org/r/#/c/173336/ for testing in logstash [releng]
2014-10-21 §
09:29 <hashar> forget me deployment-logstash1 has a puppet agent error but it is simply because the agent is disabled "'debugging logstash config'" [releng]
2014-10-20 §
17:41 <bd808> Disabled redis input plugin and restarted logstash on deployment-logstash1 [releng]
17:39 <bd808> Disabled puppet on deployment-logstash1 for some live hacking of logstash config [releng]
2014-09-30 §
15:13 <bd808> Fixed logstash by installing http://packages.elasticsearch.org/logstash/1.4/debian/pool/main/l/logstash-contrib/logstash-contrib_1.4.2-1-efd53ef_all.deb [releng]
15:02 <bd808> Logstash doesn't bundle the prune filter by default any more -- http://logstash.net/docs/1.4.2/filters/prune [releng]
14:59 <bd808> Logstash rules need to be adjusted for latest upstream version: "Couldn't find any filter plugin named 'prune'" [releng]
2014-06-07 §
02:44 <bd808> Restarted logstash on deployment-logstash1; last even logged at 2014-06-06T22:11:04 [releng]
2014-06-02 §
05:14 <bd808> Restarted logstash on deployment-logstash1; Last event logged at 2014-06-01T0722:56 [releng]
2014-05-28 §
17:53 <bd808> Restarted logstash on deployment-logstash1; last event logged at 2014-05-28T12:11:37 [releng]
2014-05-21 §
14:50 <bd808> restarted logstash on deployment-logstash1; getting really tired of these soft crashes [releng]
2014-05-19 §
14:34 <bd808> Restarted logstash service on deployment-logstash1; it stopped logging new events at 10:37:13Z [releng]
2014-05-15 §
22:14 <bd808> Restarted logstash on deployment-logstash1 yet again; memory leak from invalid encoding bug [releng]
00:14 <bd808> Disabled puppet on deployment-logstash1 to test a local logstash config change [releng]
2014-05-14 §
23:33 <bd808> Added irc input to logstash via I409fec9 [releng]
2014-05-13 §
09:28 <bd808> Restarted logstash service on deployment-logstash1 [releng]
09:28 <bd808> Logstash events stop at 2014-05-11T18:36:35Z; Log file shows many "Failed parsing date from field" errors which probably triggered the known upstream memory leak bug [releng]
2014-05-10 §
18:02 <bd808> Restarted logstash on deployment-logstash1 [releng]
2014-05-06 §
17:54 <bd808> Restarted logstash on deployment-logstash1 [releng]
17:53 <bd808> Logstash in beta hasn't recorded any events since 2014-05-04T04:32:36. [releng]
2014-04-04 §
21:39 <bd808> Restarted logstash; it stopped processing events again at 2014-04-04T19:56:46Z [releng]
2014-04-03 §
15:33 <bd808> Restarted logstash on deploymnet-logstash1; Stuck in a bad state due to jvm oom logged at 2014-04-03T12:03:43Z [releng]
2014-03-31 §
16:00 <bd808> Restarted logstash service on deployment-logstash1; no new log events seen since 2014-03-28T10:57 [releng]
2014-03-26 §
14:45 <bd808> created proxy https://logstash-beta.wmflabs.org for logstash instance [releng]