351-400 of 783 results (10ms)
2018-11-08 §
09:37 <godog> keep 2x not 3x copies of older (>15d) logstash elasticsearch indices [production]
09:29 <godog> temporarily set elasticsearch logstash watermark to low:0.85 and high:0.9 [production]
2018-10-31 §
20:27 <ladsgroup@deploy1001> Finished deploy [ores/deploy@70ba14b]: Upgrade to celery4 and flask 0.12.4, logstash fixes: T181546 T181630 T168921 T205256 T169586 T208258 T178441 (duration: 21m 29s) [production]
20:06 <ladsgroup@deploy1001> Started deploy [ores/deploy@70ba14b]: Upgrade to celery4 and flask 0.12.4, logstash fixes: T181546 T181630 T168921 T205256 T169586 T208258 T178441 [production]
2018-10-24 §
14:23 <herron> scheduled icinga downtime and disabling puppet on logstash hosts. deploying role::kafka::logging to logstash elasticserach data hosts [production]
2018-10-18 §
17:33 <ladsgroup@deploy1001> Finished deploy [ores/deploy@4ac4c8b]: Logstash support for ores: T181546 T169586 T168921 T181630 T205256 (duration: 23m 48s) [production]
17:19 <herron> aborted enabling kafka on logstash elasticsearch cluster due to puppet errors. reverted change T206454 [production]
17:09 <ladsgroup@deploy1001> Started deploy [ores/deploy@4ac4c8b]: Logstash support for ores: T181546 T169586 T168921 T181630 T205256 [production]
16:57 <herron> enabling kafka on logstash elasticsearch cluster T206454 [production]
2018-09-25 §
16:59 <gehel> forcing puppet run on all elastic nodes (including logstash) to recover prometheus metric exporter [production]
2018-09-19 §
16:52 <gehel> rolling restart of elasticsearch / logstash for new systemd unit completed - new systemd unit is elasticsearch_5@production-logstash-eqiad [production]
12:18 <gehel> rolling restart of elasticsearch / logstash for new systemd unit [production]
2018-08-31 §
08:57 <gehel> elasticsearch data directory migration on all logstash nodes [production]
2018-08-30 §
16:15 <gehel> restart of logstash to move data directory - T198351 [production]
2018-08-27 §
15:12 <herron> decreasing logstash elasticsearch index replica count to 1 on indices older than 1 day [production]
15:12 <godog> set transient low watermark to 80% for elasticsearch logstash cluster to allow shard replica allocation - T201971 [production]
2018-08-09 §
19:02 <herron> logstash1008:~# systemctl restart logstash T200960 [production]
2018-08-08 §
17:55 <herron> performing rolling restart of logstash instances T200960 [production]
2018-08-07 §
19:45 <herron> upload prometheus-logstash-exporter_0.1.2-1 to apt.wikimedia.org/jessie-wikimedia/main [production]
19:45 <herron> upload prometheus-logstash-exporter_0.1.2-1 to apt.wikimedia.org/stretch-wikimedia/main [production]
14:18 <herron> double again logstash jvm heap size to 1g and rolling restart logstash instances [production]
10:33 <godog> bounce logstash on logstash1007 for tests [production]
2018-08-06 §
18:04 <herron> double logstash jvm heap size from 256m to 512m and rolling restart of logstash instances T200960 [production]
08:23 <_joe_> restarting logstash on logstash1007, losing packets as well [production]
08:03 <_joe_> restarting logstash on logstash1009, losing packets again [production]
07:51 <_joe_> restarting logstash on logstash1008, losing packets again [production]
2018-08-04 §
14:50 <godog> roll-restart logstash because of increasing packet loss [production]
2018-08-02 §
11:17 <godog> temporarily remove multiline filter from logstash to allow using multiple workers - T200960 [production]
07:20 <TimStarling> restarting logstash on logstash1008 and logstash1009 [production]
07:06 <TimStarling> on logstash1007 increased net.core.rmem_default from 212992 to 851968 in soft state and restarted logstash [production]
06:59 <TimStarling> on logstash1007 restarting logstash [production]
2018-07-27 §
19:04 <mutante> terbium is being removed from ferm rules on elasticsearch/relforge, logstash/collector, mariadb/labtestwikitech and mw-maintenance itself (T192092) [production]
2018-06-25 §
23:30 <twentyafterfour> After monitoring logstash, everything appears stable. Evening SWAT is complete. [production]
2018-06-06 §
20:47 <ebernhardson> sighup logstash on logstash100[789] to reload config for gerrit.wikimedia.org/r/437657 [production]
2018-06-01 §
16:13 <herron> enabled new logstash tcp input with TLS enabled for syslogs on port 16514 T193766 [production]
2018-05-30 §
14:21 <herron> changing logstash elasticsearch index prefix for syslogs to 'logstash-syslog' https://gerrit.wikimedia.org/r/431860 [production]
2018-05-14 §
23:39 <bawolff@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/433089/ - re-enable sending csp logs to logstash (duration: 01m 01s) [production]
2018-05-09 §
19:53 <ottomata> rolling restart eventbus service to deploy logstash config [production]
19:51 <otto@tin> Finished deploy [eventlogging/eventbus@aa9eb2c]: logstash T193230 (duration: 01m 17s) [production]
19:49 <otto@tin> Started deploy [eventlogging/eventbus@aa9eb2c]: logstash T193230 [production]
19:48 <otto@tin> Finished deploy [eventlogging/eventbus@aa9eb2c]: logstash T193230 (duration: 00m 15s) [production]
19:48 <otto@tin> Started deploy [eventlogging/eventbus@aa9eb2c]: logstash T193230 [production]
19:38 <otto@tin> Finished deploy [eventlogging/eventbus@c70e8c5]: logstash - T193230 (duration: 03m 33s) [production]
19:35 <otto@tin> Started deploy [eventlogging/eventbus@c70e8c5]: logstash - T193230 [production]
2018-05-07 §
09:46 <moritzm> rolling restart of Kibana logstash nodes to pick up Java security updates [production]
08:19 <moritzm> rolling restart of logstash to pick up Java security updates [production]
2018-05-01 §
20:00 <ottomata> rolling restart of eventbus to apply new logstash formatter version T193230 [production]
19:27 <ottomata> rolling restart of eventbus to apply logstash tag https://phabricator.wikimedia.org/T193230 [production]
19:00 <ottomata> rolling restart of eventlogging-service-eventbus to apply logstash logging configs - T193230 [production]
2018-04-11 §
15:14 <ema> restart pybal on lvs1003 for logstash-{json,syslog} UDP monitoring config changes https://gerrit.wikimedia.org/r/#/c/425253/ [production]