2701-2750 of 10000 results (22ms)
2025-06-06 §
19:11 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: T383811 - bking@cumin2002 [production]
17:20 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: T383811 - bking@cumin2002 [production]
17:08 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: T383811 - bking@cumin2002 [production]
17:06 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: T383811 - bking@cumin2002 [production]
2025-06-04 §
19:36 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
19:23 <bking@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
19:22 <bking@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
2025-06-02 §
21:22 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: cirrussearch205*,cirrussearch2060* for T395855 - bking@cumin2002 [production]
21:22 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: cirrussearch205*,cirrussearch2060* for T395855 - bking@cumin2002 [production]
21:16 <bking@cumin2002> conftool action : set/pooled=no; selector: name=cirrussearch2055.codfw.wmnet|cirrussearch2056.codfw.wmnet|cirrussearch2057.codfw.wmnet|cirrussearch2058.codfw.wmnet|cirrussearch2059.codfw.wmnet|cirrussearch2060.codfw.wmnet|cirrussearch2091.codfw.wmnet [production]
21:13 <bking@cumin2002> conftool action : set/pooled=yes:weight=10; selector: name=cirrussearch.*.codfw.wmnet [production]
14:35 <bking@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:35 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts elastic1067.eqiad.wmnet [production]
14:35 <bking@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: elastic1067.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - bking@cumin2002" [production]
14:35 <bking@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: elastic1067.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - bking@cumin2002" [production]
13:43 <bking@cumin2002> START - Cookbook sre.dns.netbox [production]
13:38 <bking@cumin2002> START - Cookbook sre.hosts.decommission for hosts elastic1067.eqiad.wmnet [production]
13:37 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cirrussearch[1064-1066].eqiad.wmnet [production]
13:37 <bking@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cirrussearch[1064-1066].eqiad.wmnet decommissioned, removing all IPs except the asset tag one - bking@cumin2002" [production]
13:37 <bking@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:37 <bking@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cirrussearch[1064-1066].eqiad.wmnet decommissioned, removing all IPs except the asset tag one - bking@cumin2002" [production]
13:21 <bking@cumin2002> START - Cookbook sre.dns.netbox [production]
13:09 <bking@cumin2002> START - Cookbook sre.hosts.decommission for hosts cirrussearch[1064-1066].eqiad.wmnet [production]
2025-05-30 §
21:08 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts cirrussearch[1055-1059].eqiad.wmnet [production]
21:08 <bking@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:08 <bking@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cirrussearch[1055-1059].eqiad.wmnet decommissioned, removing all IPs except the asset tag one - bking@cumin2002" [production]
21:08 <bking@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cirrussearch[1055-1059].eqiad.wmnet decommissioned, removing all IPs except the asset tag one - bking@cumin2002" [production]
21:04 <bking@cumin2002> START - Cookbook sre.dns.netbox [production]
20:49 <bking@cumin2002> START - Cookbook sre.hosts.decommission for hosts cirrussearch[1055-1059].eqiad.wmnet [production]
20:04 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) for hosts relforge[1003-1004].eqiad.wmnet [production]
20:00 <bking@deploy1003> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
20:00 <bking@deploy1003> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
19:51 <bking@deploy1003> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
19:51 <bking@deploy1003> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
19:45 <bking@cumin2002> START - Cookbook sre.hosts.decommission for hosts relforge[1003-1004].eqiad.wmnet [production]
2025-05-29 §
19:05 <bking@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on relforge[1003-1004,1008-1009].eqiad.wmnet with reason: noisy alerts [production]
16:25 <bking@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on cirrussearch[2112-2113].codfw.wmnet with reason: firmware update [production]
16:24 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cirrussearch[2111-2112].codfw.wmnet [production]
16:24 <bking@cumin2002> START - Cookbook sre.hosts.remove-downtime for cirrussearch[2111-2112].codfw.wmnet [production]
16:17 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: cirrussearch2112*,cirrussearch2113* for T394543 - bking@cumin2002 [production]
16:17 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: cirrussearch2112*,cirrussearch2113* for T394543 - bking@cumin2002 [production]
2025-05-28 §
21:54 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts cirrussearch1053.eqiad.wmnet [production]
21:53 <bking@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:51 <bking@cumin2002> START - Cookbook sre.dns.netbox [production]
21:43 <bking@cumin2002> START - Cookbook sre.hosts.decommission for hosts cirrussearch1053.eqiad.wmnet [production]
17:20 <bking@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on elastic[1054,1067,1103].eqiad.wmnet with reason: downtime until decom [production]
15:17 <inflatador> bking@mwmaint1002 ban Elastic/CS hosts prior to decom T394350 [production]
13:20 <bking@cumin2002> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: T383811 - bking@cumin2002 [production]
13:19 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: T383811 - bking@cumin2002 [production]
2025-05-27 §
22:08 <bking@cumin2002> conftool action : set/pooled=yes:weight=10; selector: name=cirrussearch1096.eqiad.wmnet|cirrussearch1097.eqiad.wmnet|cirrussearch1098.eqiad.wmnet|cirrussearch1099.eqiad.wmnet|cirrussearch1100.eqiad.wmnet|cirrussearch1101.eqiad.wmnet|cirrussearch1102.eqiad.wmnet|cirrussearch1107.eqiad.wmnet|cirrussearch1110.eqiad.wmnet|cirrussearch1124.eqiad.wmnet|cirrussearch1125.eqiad.wmnet [production]