251-300 of 783 results (10ms)
2020-02-19 §
09:49 <akosiaris> T245516. Deploy mathoid chart version 0.0.27, removing logstash gelf configuration [production]
00:10 <niharika29@deploy1001> Synchronized wmf-config/logging.php: Make the logstash and authmanager-statsd Monolog handlers compatible (duration: 01m 04s) [production]
2020-02-03 §
14:44 <moritzm> restarting apache on an-tool*. cloudmetrics*, logstash*, grafana1002 to pick up libidn security update [production]
2020-01-22 §
14:53 <vgutierrez> copied python3-logstash to apt.w.o (buster) - T242093 [production]
2019-12-11 §
09:33 <godog> roll-restart logstash in codfw/eqiad after https://gerrit.wikimedia.org/r/c/operations/puppet/+/556173 [production]
2019-12-03 §
11:31 <godog> refresh kibana fields for logstash-* [production]
09:05 <godog> downtime new logstash hosts in codfw/eqiad until thurs [production]
2019-11-26 §
19:22 <shdubsh> restore codfw logstash to baseline - T215904 [production]
19:09 <shdubsh> stop logstash codfw, generate some consumer lag, and set batch size to 2000 - T215904 [production]
18:55 <shdubsh> stop logstash codfw, generate some consumer lag - T215904 [production]
18:33 <shdubsh> stop logstash on logstash200[5-6] for metrics collection - T215904 [production]
2019-11-15 §
10:24 <godog> roll-restart logstash to apply configuration change [production]
2019-11-08 §
15:37 <herron> beginning rolling service restarts on logstash hosts for java security updates [production]
2019-11-07 §
10:58 <moritzm> installing Java security updates on kafka-main/logstash [production]
2019-10-31 §
20:27 <shdubsh> restarting logstash on logstash1008 to test level->severity filter selector [production]
10:54 <godog> bounce logstash on logstash2004 [production]
10:13 <godog> bounce logstash on logstash2004 [production]
09:37 <godog> temporarily stop logstash on logstash2005 to test performance with two ingesters only - T215904 [production]
09:23 <godog> temporarily stop logstash on logstash2006 to test performance with two ingesters only - T215904 [production]
2019-10-30 §
15:48 <godog> roll restart logstash after https://gerrit.wikimedia.org/r/c/operations/puppet/+/544217 [production]
2019-10-28 §
13:30 <godog> roll restart logstash in codfw/eqiad to apply new config [production]
08:48 <godog> bump udp_localhost kafka-logging topics to 6 partitions and roll-restart logstash and rsyslog - T215904 [production]
2019-10-24 §
07:42 <godog> bump rsyslog- topics partitions to 6 and roll-restart logstash frontends [production]
2019-10-23 §
09:40 <godog> roll restart logstash to pick up new rsyslog-notice partitions [production]
08:23 <godog> roll restart logstash in codfw/eqiad to pick up new kafka partitions [production]
07:28 <hashar> logstash: refreshing index fields for logstash-* indices (via https://logstash.wikimedia.org/app/kibana#/management/kibana/indices/logstash-* ) # T234564 [production]
2019-10-21 §
08:52 <godog> roll-restart logstash to apply https://gerrit.wikimedia.org/r/c/operations/puppet/+/544209 [production]
2019-10-07 §
18:16 <godog> roll-restart logstash to apply https://gerrit.wikimedia.org/r/c/operations/puppet/+/539978 [production]
2019-10-01 §
14:08 <herron> beginning rolling reboots of eqiad and codfw logstash collectors [production]
2019-09-21 §
01:08 <shdubsh> removed input-kafka-rsyslog-shipper-eqiad/codfw from logstash inputs logstash1008 and logstash1009 [production]
00:46 <shdubsh> restarting logstash on logstash1008 without udp-localhost-eqiad/codfw configs [production]
2019-08-06 §
19:50 <herron> disabling puppet on logstash collectors for rolling deploy of https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/528306/ T166107 [production]
14:52 <herron> restarting logstash service on logstash1007 to pick up puppet managed log4j2 config [production]
2019-08-01 §
14:51 <herron> performing rolling restarts of eqiad logstash cluster for security updates [production]
2019-07-31 §
14:28 <herron> beginning rolling reboots of codfw logstash hosts for security updates [production]
2019-07-23 §
23:43 <shdubsh> reverting logstash mitigations and re-enable puppet [production]
22:06 <herron> puppet temporarily disabled on eqiad/codfw logstash collectors while catching up with backlog. see /etc/logstash/conf.d/01-filter_temp_drops.conf [production]
21:52 <herron> logstash - temporarily dropping logs matching [message] =~ /^SlowTimer/ due to UTF-8 parsing errors that are stopping the logstash processing pipeline. will re-enable after logstash has caught up with the backlog [production]
20:59 <shdubsh> temporarily disable input-kafka-rsyslog-shipper and drop memcached logs on logstash nodes [production]
19:06 <herron> restarting logstash on logstash100[789] [production]
17:52 <moritzm> installing Java security updates on kafka/main and Logstash servers [production]
2019-07-16 §
00:03 <shdubsh> restart logstash to revert mitigations - T228089 [production]
2019-07-15 §
19:05 <shdubsh> restarting logstash on logstash1008 [production]
2019-07-11 §
23:49 <eevans@deploy1001> Finished deploy [cassandra/logstash-logback-encoder@d085ffa]: deploy logback to restbase1017 (T222960) (duration: 00m 47s) [production]
23:48 <eevans@deploy1001> Started deploy [cassandra/logstash-logback-encoder@d085ffa]: deploy logback to restbase1017 (T222960) [production]
23:47 <eevans@deploy1001> Finished deploy [cassandra/logstash-logback-encoder@d085ffa]: (no justification provided) (duration: 01m 56s) [production]
23:45 <eevans@deploy1001> Started deploy [cassandra/logstash-logback-encoder@d085ffa]: (no justification provided) [production]
23:36 <eevans@deploy1001> Started deploy [cassandra/logstash-logback-encoder@d085ffa]: (no justification provided) [production]
2019-06-07 §
18:56 <herron> performing rolling reboots of logstash codfw frontends for security updates [production]
2019-06-06 §
19:47 <herron> performing rolling reboot of eqiad logstash hw for MDS security updates [production]