101-150 of 783 results (14ms)
2024-02-14 §
19:43 <eevans@deploy2002> Started deploy [cassandra/logstash-logback-encoder@0521449]: Deploying to updated target list — T353550 [production]
19:42 <eevans@deploy2002> Finished deploy [cassandra/logstash-logback-encoder@0521449]: Deploying to updated target list — T353550 (duration: 00m 45s) [production]
19:41 <eevans@deploy2002> Started deploy [cassandra/logstash-logback-encoder@0521449]: Deploying to updated target list — T353550 [production]
19:39 <eevans@deploy2002> Finished deploy [cassandra/logstash-logback-encoder@0521449]: Deploying to updated target list — T353550 (duration: 01m 17s) [production]
19:38 <eevans@deploy2002> Started deploy [cassandra/logstash-logback-encoder@0521449]: Deploying to updated target list — T353550 [production]
19:36 <eevans@deploy2002> Finished deploy [cassandra/logstash-logback-encoder@0521449]: Deploying to updated target list — T353550 (duration: 00m 16s) [production]
19:35 <eevans@deploy2002> Started deploy [cassandra/logstash-logback-encoder@0521449]: Deploying to updated target list — T353550 [production]
19:31 <eevans@deploy2002> Finished deploy [cassandra/logstash-logback-encoder@5c2dd00]: Deploying to updated target list — T353550 (duration: 00m 20s) [production]
19:30 <eevans@deploy2002> Started deploy [cassandra/logstash-logback-encoder@5c2dd00]: Deploying to updated target list — T353550 [production]
19:28 <eevans@deploy2002> Finished deploy [cassandra/logstash-logback-encoder@5c2dd00]: Deploying to updated target list — T353550 (duration: 00m 41s) [production]
19:27 <eevans@deploy2002> Started deploy [cassandra/logstash-logback-encoder@5c2dd00]: Deploying to updated target list — T353550 [production]
2024-01-25 §
13:25 <topranks> stopping logstash service on logstash2025 to faciliate VM migration T355549 [production]
2024-01-18 §
21:14 <dreamyjazz@deploy2002> Finished scap: Backport for [[gerrit:991555|Log to statsd HTTP status codes and reduce logstash log levels (T355216)]] (duration: 09m 00s) [production]
21:08 <dreamyjazz@deploy2002> dreamyjazz: Backport for [[gerrit:991555|Log to statsd HTTP status codes and reduce logstash log levels (T355216)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:05 <dreamyjazz@deploy2002> Started scap: Backport for [[gerrit:991555|Log to statsd HTTP status codes and reduce logstash log levels (T355216)]] [production]
2024-01-16 §
14:24 <dreamyjazz@deploy2002> Finished scap: Backport for [[gerrit:990760|Add more statsd counters and add logstash logging (T351419)]] (duration: 07m 15s) [production]
14:18 <dreamyjazz@deploy2002> dreamyjazz: Backport for [[gerrit:990760|Add more statsd counters and add logstash logging (T351419)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
14:16 <dreamyjazz@deploy2002> Started scap: Backport for [[gerrit:990760|Add more statsd counters and add logstash logging (T351419)]] [production]
2024-01-04 §
20:41 <ryankemper> [apifeatureusage] T350703 Restarted `logstash` on `apifeatureusage[1,2]001` [production]
2023-12-16 §
01:21 <eevans@deploy2002> Finished deploy [cassandra/logstash-logback-encoder@fb10de1]: (no justification provided) (duration: 00m 10s) [production]
01:21 <eevans@deploy2002> Started deploy [cassandra/logstash-logback-encoder@fb10de1]: (no justification provided) [production]
2023-12-07 §
17:09 <herron@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cleanup logstash/kibana records T299700 - herron@cumin1001" [production]
17:08 <herron@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cleanup logstash/kibana records T299700 - herron@cumin1001" [production]
2023-11-16 §
15:21 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: revert logstash changes - bking@cumin2002 - T324335 [production]
15:17 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: revert logstash changes - bking@cumin2002 - T324335 [production]
2023-11-07 §
18:30 <herron> performing rolling memory increase on logstash collector VMs T350434 [production]
2023-10-23 §
18:31 <herron> sretest1001:~/tmp/backfill$ promtool tsdb create-blocks-from rules --start 1672531200 --end 1698080718 --url http://prometheus.svc.eqiad.wmnet/ops/ logstash-requests.yaml T349521 [production]
2023-08-02 §
14:19 <fabfur> importing python-logstash in bookworm-wikimedia (T342154) [production]
2023-05-31 §
07:35 <godog> apache2 restarted on logstash1032 before I could get a backtrace to debug logstash lag [production]
2023-02-27 §
22:31 <ryankemper> [apifeatureusage] T329957 Restarted `logstash` on `apifeatureusage[1-2]001` [production]
2023-01-30 §
13:29 <godog> bounce logstash on logstash1025 -- GC unhappy causing kafka lag [production]
2023-01-19 §
16:48 <godog> roll-restart opensearch-dashboards in logstash collectors eqiad - T327161 [production]
16:18 <jmm@cumin2002> END (FAIL) - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors (exit_code=1) rolling restart_daemons on A:logstash-collector [production]
16:08 <jmm@cumin2002> START - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors rolling restart_daemons on A:logstash-collector [production]
2022-11-03 §
22:40 <cwhite> logstash eqiad - opensearch 2.2.0 upgrade complete T304440 [production]
2022-10-25 §
19:39 <cwhite> logstash opensearch 2.2.0 codfw transition complete T304440 [production]
2022-09-12 §
13:35 <Lucas_WMDE> manually applying [[gerrit:830691]] on mwdebug1001 to test if T317520 affects production (expected to cause getExpensiveParserFunctionLimit-related logstash errors) [production]
2022-09-06 §
18:25 <cwhite> reduce codfw replicas 2 to 1 for logstash-(webrequest|k8s) partitions. Make space for failed logstash2027 - T316996 [production]
2022-08-05 §
13:57 <jbond> upload logstash-logback-encoder-7.2 to puppet7 component [production]
2022-06-02 §
17:39 <cwhite> rolling restart of eqiad logstash cluster [production]
17:11 <cwhite> rolling restart of codfw logstash cluster [production]
17:09 <cwhite> restart logstash on apifeatureusage hosts [production]
2022-04-27 §
21:12 <ebernhardson@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:787074|Forward CirrusSearchDeprecation logs to logstash (T218994)]] (duration: 00m 56s) [production]
2022-04-07 §
12:49 <akosiaris> sudo gnt-cluster modify -H kvm:migration_downtime=3000 for ganeti01.svc.codfw.wmnet and ganeti01.svc.eqiad.wmnet to combat some logstash VM migration issues. [production]
2022-03-31 §
19:21 <cwhite> remove openjdk-8-jre from eqiad logstash nodes T301770 [production]
2022-03-23 §
23:25 <cwhite> remove openjdk-8-jre from codfw logstash nodes T301770 [production]
2022-03-17 §
16:00 <moritzm> restarting apache on logstash* [production]
2022-03-03 §
16:30 <godog> roll-restart logstash to pick up config changes - T291946 [production]
2022-03-01 §
16:12 <moritzm> restarting apache on logstash nodes to pick up expat update [production]
2022-02-22 §
19:24 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts logstash[2004-2006].codfw.wmnet [production]