6851-6900 of 7789 results (22ms)
2023-02-07 §
20:44 <bking@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host an-airflow1005.eqiad.wmnet [production]
19:57 <bking@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) an-airflow1005.eqiad.wmnet on all recursors [production]
19:57 <bking@cumin1001> START - Cookbook sre.dns.wipe-cache an-airflow1005.eqiad.wmnet on all recursors [production]
19:57 <bking@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:57 <bking@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM an-airflow1005.eqiad.wmnet - bking@cumin1001" [production]
19:56 <bking@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM an-airflow1005.eqiad.wmnet - bking@cumin1001" [production]
19:53 <bking@cumin1001> START - Cookbook sre.dns.netbox [production]
19:53 <bking@cumin1001> START - Cookbook sre.ganeti.makevm for new host an-airflow1005.eqiad.wmnet [production]
18:02 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for 13 hosts [production]
18:02 <bking@cumin2002> START - Cookbook sre.hosts.remove-downtime for 13 hosts [production]
17:55 <inflatador> bking@cumin1001 repooling elastic and wdqs hosts post-maintenance T327925 [production]
2023-02-06 §
22:51 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on 13 hosts with reason: switch upgrade [production]
22:51 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on 13 hosts with reason: switch upgrade [production]
22:42 <inflatador> bking@cumin2002 banning Elastic nodes from cluster in preparation for T327925 [production]
2023-02-02 §
20:02 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host elastic2037.codfw.wmnet [production]
19:55 <bking@cumin1001> START - Cookbook sre.hosts.reboot-single for host elastic2037.codfw.wmnet [production]
2023-01-31 §
18:25 <bking@deploy1002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
18:25 <bking@deploy1002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
16:40 <bking@deploy1002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
16:38 <bking@deploy1002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
16:37 <bking@deploy1002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
16:37 <bking@deploy1002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
2023-01-30 §
19:36 <bking@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
19:35 <bking@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
17:34 <bking@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
17:34 <bking@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
17:24 <inflatador> bking@build2001 rebuilding docker images for 884351 complete [production]
17:22 <inflatador> bking@build2001 rebuilding docker images for 884351 [production]
2023-01-25 §
21:49 <bking@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/services/flink-app-example: apply [production]
21:49 <bking@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/services/flink-app-example: apply [production]
21:44 <bking@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/services/flink-app-example: apply [production]
21:44 <bking@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/services/flink-app-example: apply [production]
21:25 <bking@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
21:24 <bking@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
2023-01-18 §
22:35 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: raise heap memory to 12G - bking@cumin1001 - T323646 [production]
22:03 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: raise heap memory to 12G - bking@cumin1001 - T323646 [production]
2023-01-10 §
15:21 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host search-loader2001.codfw.wmnet [production]
15:17 <bking@cumin1001> START - Cookbook sre.hosts.reboot-single for host search-loader2001.codfw.wmnet [production]
14:53 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host search-loader1001.eqiad.wmnet [production]
14:49 <bking@cumin1001> START - Cookbook sre.hosts.reboot-single for host search-loader1001.eqiad.wmnet [production]
14:34 <bking@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host apifeatureusage2001.codfw.wmnet [production]
14:21 <bking@cumin1001> START - Cookbook sre.hosts.reboot-single for host apifeatureusage2001.codfw.wmnet [production]
14:14 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host apifeatureusage1001.eqiad.wmnet [production]
14:06 <bking@cumin1001> START - Cookbook sre.hosts.reboot-single for host apifeatureusage1001.eqiad.wmnet [production]
00:48 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.UPGRADE (3 nodes at a time) for ElasticSearch cluster search_codfw: plugin upgrade - bking@cumin1001 - T324247 [production]
2023-01-09 §
22:33 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (3 nodes at a time) for ElasticSearch cluster search_codfw: plugin upgrade - bking@cumin1001 - T324247 [production]
22:32 <bking@cumin1001> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.UPGRADE (3 nodes at a time) for ElasticSearch cluster search_codfw: plugin upgrade - bking@cumin1001 - T324247 [production]
21:45 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (3 nodes at a time) for ElasticSearch cluster search_codfw: plugin upgrade - bking@cumin1001 - T324247 [production]
21:34 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic plugin upgrade - bking@cumin1001 - T324247 [production]
20:52 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic plugin upgrade - bking@cumin1001 - T324247 [production]