7801-7850 of 8776 results (24ms)
2023-02-17 §
22:09 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin1001 - T329957 [production]
22:06 <bking@cumin1001> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin1001 - T329957 [production]
22:05 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin1001 - T329957 [production]
17:10 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T329957 [production]
17:06 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T329957 [production]
17:06 <bking@cumin1001> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T329957 [production]
17:06 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T329957 [production]
2023-02-16 §
07:25 <bking@cumin1001> END (FAIL) - Cookbook sre.ganeti.reimage (exit_code=99) for host an-airflow1005.eqiad.wmnet with OS buster [production]
2023-02-15 §
16:24 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-airflow1005.eqiad.wmnet with reason: host reimage [production]
16:21 <bking@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-airflow1005.eqiad.wmnet with reason: host reimage [production]
16:12 <bking@cumin1001> START - Cookbook sre.ganeti.reimage for host an-airflow1005.eqiad.wmnet with OS buster [production]
16:12 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for an-airflow1005.eqiad.wmnet [production]
16:12 <bking@cumin1001> START - Cookbook sre.hosts.remove-downtime for an-airflow1005.eqiad.wmnet [production]
15:33 <bking@deploy1002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
15:33 <bking@deploy1002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
15:30 <bking@deploy1002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
15:29 <bking@deploy1002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
15:27 <bking@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
15:27 <bking@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
15:26 <bking@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
15:24 <bking@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
11:23 <bking@cumin1001> END (FAIL) - Cookbook sre.ganeti.reimage (exit_code=99) for host an-airflow1005.eqiad.wmnet with OS bullseye [production]
2023-02-14 §
22:26 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on an-airflow1005.eqiad.wmnet with reason: new OS but some puppet stuff doesn't work yet [production]
22:26 <bking@cumin1001> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on an-airflow1005.eqiad.wmnet with reason: new OS but some puppet stuff doesn't work yet [production]
20:24 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-airflow1005.eqiad.wmnet with reason: host reimage [production]
20:21 <bking@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-airflow1005.eqiad.wmnet with reason: host reimage [production]
20:09 <bking@cumin1001> START - Cookbook sre.ganeti.reimage for host an-airflow1005.eqiad.wmnet with OS bullseye [production]
15:50 <inflatador> bking@deploy1002 'deploying rdf-streaming-updater prod eqiad T304914' [production]
2023-02-13 §
15:51 <bking@cumin1001> START - Cookbook sre.wdqs.data-reload [production]
14:05 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) [production]
14:05 <bking@cumin1001> START - Cookbook sre.wdqs.data-reload [production]
2023-02-09 §
20:50 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge logging config change - bking@cumin1001 - T324335 [production]
20:47 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge logging config change - bking@cumin1001 - T324335 [production]
20:31 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge logging config change - bking@cumin1001 - T324335 [production]
20:27 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge logging config change - bking@cumin1001 - T324335 [production]
2023-02-07 §
22:09 <bking@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "provision new Ganeti VM an-airflow1005 - bking@cumin1001 - T327970" [production]
22:06 <bking@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "provision new Ganeti VM an-airflow1005 - bking@cumin1001 - T327970" [production]
20:44 <bking@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host an-airflow1005.eqiad.wmnet [production]
19:57 <bking@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) an-airflow1005.eqiad.wmnet on all recursors [production]
19:57 <bking@cumin1001> START - Cookbook sre.dns.wipe-cache an-airflow1005.eqiad.wmnet on all recursors [production]
19:57 <bking@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:57 <bking@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM an-airflow1005.eqiad.wmnet - bking@cumin1001" [production]
19:56 <bking@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM an-airflow1005.eqiad.wmnet - bking@cumin1001" [production]
19:53 <bking@cumin1001> START - Cookbook sre.dns.netbox [production]
19:53 <bking@cumin1001> START - Cookbook sre.ganeti.makevm for new host an-airflow1005.eqiad.wmnet [production]
18:02 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for 13 hosts [production]
18:02 <bking@cumin2002> START - Cookbook sre.hosts.remove-downtime for 13 hosts [production]
17:55 <inflatador> bking@cumin1001 repooling elastic and wdqs hosts post-maintenance T327925 [production]
2023-02-06 §
22:51 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on 13 hosts with reason: switch upgrade [production]
22:51 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on 13 hosts with reason: switch upgrade [production]