1-50 of 786 results (12ms)
2025-12-01 §
10:00 <jmm@cumin2002> END (PASS) - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors (exit_code=0) rolling restart_daemons on A:logstash-collector [production]
09:53 <jmm@cumin2002> START - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors rolling restart_daemons on A:logstash-collector [production]
2025-11-26 §
00:30 <denisse> Upgrading envoy on logstash hosts - T405808 [production]
2025-09-23 §
16:22 <denisse> Upgrade Envoy to v1.29.12 on logstash hosts - T403663 [production]
2025-09-10 §
09:44 <jmm@cumin2002> END (PASS) - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors (exit_code=0) rolling restart_daemons on A:logstash-collector [production]
09:38 <moritzm> upgrading Envoy on Logstash T402584 [production]
09:37 <jmm@cumin2002> START - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors rolling restart_daemons on A:logstash-collector [production]
2025-08-25 §
11:04 <kharlan@deploy1003> Finished scap sync-world: Backport for [[gerrit:1181664|hcaptcha: Instrument siteverify API call (T402492)]], [[gerrit:1181665|hCaptcha: Log errors to Logstash (T402767)]] (duration: 14m 26s) [production]
10:56 <kharlan@deploy1003> kharlan: Backport for [[gerrit:1181664|hcaptcha: Instrument siteverify API call (T402492)]], [[gerrit:1181665|hCaptcha: Log errors to Logstash (T402767)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
10:50 <kharlan@deploy1003> Started scap sync-world: Backport for [[gerrit:1181664|hcaptcha: Instrument siteverify API call (T402492)]], [[gerrit:1181665|hCaptcha: Log errors to Logstash (T402767)]] [production]
2025-07-24 §
13:46 <dreamyjazz@deploy1003> Finished scap sync-world: Backport for [[gerrit:1172323|Make TSP extensions have warning logs in logstash]] (duration: 08m 51s) [production]
13:39 <dreamyjazz@deploy1003> dreamyjazz: Backport for [[gerrit:1172323|Make TSP extensions have warning logs in logstash]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
13:37 <dreamyjazz@deploy1003> Started scap sync-world: Backport for [[gerrit:1172323|Make TSP extensions have warning logs in logstash]] [production]
2025-06-09 §
13:31 <taavi@deploy1003> Finished scap sync-world: Backport for [[gerrit:1153363|logging: Allow sampling of Logstash logs (T395967)]], [[gerrit:1153364|logging: Sample some high-volume log streams (T394402)]] (duration: 24m 30s) [production]
13:21 <taavi@deploy1003> taavi, tgr: Backport for [[gerrit:1153363|logging: Allow sampling of Logstash logs (T395967)]], [[gerrit:1153364|logging: Sample some high-volume log streams (T394402)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
13:06 <taavi@deploy1003> Started scap sync-world: Backport for [[gerrit:1153363|logging: Allow sampling of Logstash logs (T395967)]], [[gerrit:1153364|logging: Sample some high-volume log streams (T394402)]] [production]
2025-05-20 §
15:21 <jmm@cumin2002> END (PASS) - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors (exit_code=0) rolling restart_daemons on A:logstash-collector [production]
15:13 <jmm@cumin2002> START - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors rolling restart_daemons on A:logstash-collector [production]
2025-03-24 §
16:57 <btullis@cumin1002> END (PASS) - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash (exit_code=0) rolling restart_daemons on A:apifeatureusage [production]
16:55 <btullis@cumin1002> START - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash rolling restart_daemons on A:apifeatureusage [production]
2025-03-13 §
14:35 <jmm@cumin2002> END (PASS) - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors (exit_code=0) rolling restart_daemons on A:logstash-collector [production]
14:27 <jmm@cumin2002> START - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors rolling restart_daemons on A:logstash-collector [production]
2024-12-11 §
14:11 <btullis@cumin1002> END (PASS) - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash (exit_code=0) rolling restart_daemons on A:apifeatureusage [production]
14:08 <btullis@cumin1002> START - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash rolling restart_daemons on A:apifeatureusage [production]
2024-09-25 §
07:19 <jmm@cumin2002> END (PASS) - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors (exit_code=0) rolling restart_daemons on A:logstash-collector [production]
07:02 <jmm@cumin2002> START - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors rolling restart_daemons on A:logstash-collector [production]
2024-09-24 §
16:38 <swfrench@cumin1002> conftool action : set/pooled=false; selector: dnsdisc=logstash,name=codfw [production]
16:37 <swfrench@cumin1002> conftool action : set/pooled=true; selector: dnsdisc=logstash,name=eqiad [production]
2024-09-16 §
16:12 <rzl> rzl@cumin1002:~$ sudo cumin logstash[2023,2025,2030-2032].codfw.wmnet,logstash[1025,1030,1032].eqiad.wmnet 'systemctl restart opensearch-dashboards' # only hosts where status is failed [production]
2024-09-09 §
17:44 <jforrester@deploy1003> Finished scap sync-world: Backport for [[gerrit:1071266|ZObjectStore::findZTesterResult: Trim our own error so we don't break logstash (T374241)]] (duration: 12m 05s) [production]
17:33 <jforrester@deploy1003> jforrester: Backport for [[gerrit:1071266|ZObjectStore::findZTesterResult: Trim our own error so we don't break logstash (T374241)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
17:31 <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1071266|ZObjectStore::findZTesterResult: Trim our own error so we don't break logstash (T374241)]] [production]
2024-09-04 §
10:11 <btullis@cumin1002> END (PASS) - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash (exit_code=0) rolling restart_daemons on A:apifeatureusage [production]
10:08 <btullis@cumin1002> START - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash rolling restart_daemons on A:apifeatureusage [production]
2024-08-27 §
09:04 <tappof@cumin2002> END (PASS) - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors (exit_code=0) rolling restart_daemons on P{O:logging::opensearch::collector and logstash*.codfw.wmnet} and (A:logstash-collector) [production]
09:00 <tappof@cumin2002> START - Cookbook sre.o11y.roll-restart-reboot-logstash-collectors rolling restart_daemons on P{O:logging::opensearch::collector and logstash*.codfw.wmnet} and (A:logstash-collector) [production]
2024-08-16 §
23:27 <eevans@deploy1003> Started deploy [cassandra/logstash-logback-encoder@42653e6] (beta): Test deploy [production]
20:26 <eevans@deploy1003> Finished deploy [cassandra/logstash-logback-encoder@42653e6] (aqs): Test (duration: 00m 32s) [production]
20:26 <eevans@deploy1003> Started deploy [cassandra/logstash-logback-encoder@42653e6] (aqs): Test [production]
20:15 <eevans@deploy1003> Finished deploy [cassandra/logstash-logback-encoder@42653e6] (beta): Beta deploy (duration: 00m 31s) [production]
20:14 <eevans@deploy1003> Started deploy [cassandra/logstash-logback-encoder@42653e6] (beta): Beta deploy [production]
2024-07-29 §
12:46 <godog> upgrade and roll-restart benthos@mw_accesslog_sampler on logstash hosts [production]
10:12 <godog> bounce benthos@mw_accesslog_sampler on logstash collectors [production]
2024-07-24 §
21:44 <ryankemper@cumin2002> END (PASS) - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash (exit_code=0) rolling reboot on A:apifeatureusage [production]
21:35 <ryankemper@cumin2002> START - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash rolling reboot on A:apifeatureusage [production]
2024-06-21 §
23:54 <cwhite> delete remaining 2024.03 log indexes to make room on logstash eqiad and codfw T368180 [production]
2024-06-12 §
15:28 <denisse@cumin2002> conftool action : set/pooled=true; selector: dnsdisc=logstash,name=eqiad [production]
2024-05-13 §
12:44 <brouberol@cumin2002> END (PASS) - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash (exit_code=0) rolling restart_daemons on A:apifeatureusage [production]
2024-05-09 §
15:27 <eevans@deploy1002> Finished deploy [cassandra/logstash-logback-encoder@42653e6] (aqs): (no justification provided) (duration: 00m 33s) [production]
15:27 <eevans@deploy1002> Started deploy [cassandra/logstash-logback-encoder@42653e6] (aqs): (no justification provided) [production]