1601-1650 of 10000 results (19ms)
2026-04-01 §
15:22 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T421714, prepare newly-reimaged host) xfer commons from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
15:22 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T421714, prepare newly-reimaged host) xfer commons from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
15:22 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T421714, prepare newly-reimaged host) xfer wikidata from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
15:20 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T421714, prepare newly-reimaged host) xfer wikidata from wcqs1001.eqiad.wmnet -> wcqs1003.eqiad.wmnet, repooling both afterwards [production]
14:59 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wcqs1003.eqiad.wmnet with OS bullseye [production]
14:28 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wcqs1003.eqiad.wmnet with reason: host reimage [production]
14:22 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on wcqs1003.eqiad.wmnet with reason: host reimage [production]
14:05 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wcqs1003 [production]
14:05 <bking@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wcqs1003 [production]
14:04 <bking@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host wcqs1003 [production]
14:04 <bking@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wcqs1003.eqiad.wmnet 9.32.64.10.in-addr.arpa 9.0.0.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
14:04 <bking@cumin2002> START - Cookbook sre.dns.wipe-cache wcqs1003.eqiad.wmnet 9.32.64.10.in-addr.arpa 9.0.0.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
14:04 <bking@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:04 <bking@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wcqs1003 - bking@cumin2002" [production]
14:04 <bking@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wcqs1003 - bking@cumin2002" [production]
14:00 <bking@cumin2002> START - Cookbook sre.dns.netbox [production]
14:00 <bking@cumin2002> START - Cookbook sre.hosts.move-vlan for host wcqs1003 [production]
13:59 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host wcqs1003.eqiad.wmnet with OS bullseye [production]
2026-03-30 §
14:24 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti-jumbo1001.eqiad.wmnet with OS trixie [production]
14:08 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti-jumbo1001.eqiad.wmnet with reason: host reimage [production]
14:02 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti-jumbo1001.eqiad.wmnet with reason: host reimage [production]
13:51 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host ganeti-jumbo1001 [production]
13:51 <bking@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ganeti-jumbo1001 [production]
13:49 <bking@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host ganeti-jumbo1001 [production]
13:49 <bking@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) ganeti-jumbo1001.eqiad.wmnet 140.48.64.10.in-addr.arpa 0.4.1.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
13:49 <bking@cumin2002> START - Cookbook sre.dns.wipe-cache ganeti-jumbo1001.eqiad.wmnet 140.48.64.10.in-addr.arpa 0.4.1.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
13:49 <bking@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:49 <bking@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host ganeti-jumbo1001 - bking@cumin2002" [production]
13:49 <bking@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host ganeti-jumbo1001 - bking@cumin2002" [production]
13:34 <bking@cumin2002> START - Cookbook sre.dns.netbox [production]
13:34 <bking@cumin2002> START - Cookbook sre.hosts.move-vlan for host ganeti-jumbo1001 [production]
13:33 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host ganeti-jumbo1001.eqiad.wmnet with OS trixie [production]
2026-03-26 §
14:23 <bking@deploy2002> helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-test: apply [production]
14:22 <bking@deploy2002> helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-test: apply [production]
14:21 <bking@deploy2002> helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-test: apply [production]
14:21 <bking@deploy2002> helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-test: apply [production]
13:46 <bking@deploy2002> helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-test: apply [production]
13:45 <bking@deploy2002> helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-test: apply [production]
2026-03-24 §
19:02 <inflatador> bking@apt1002 `sudo -E reprepro -C component/opensearch2 include trixie-wikimedia ~/wmf-opensearch-search-plugins-2.19.5+3-trixie/wmf-opensearch-search-plugins_2.19.5+3_amd64.changes` [production]
2026-03-23 §
18:59 <inflatador> bking@deploy2002 restarting opensearch-semantic-search eqiad to renew certs [production]
15:03 <bking@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) opensearch-ipoid.discovery.wmnet on all recursors [production]
15:03 <bking@cumin2002> START - Cookbook sre.dns.wipe-cache opensearch-ipoid.discovery.wmnet on all recursors [production]
15:03 <bking@cumin2002> conftool action : set/pooled=true; selector: dnsdisc=k8s-ingress-dse-aa,name=eqiad [production]
14:59 <bking@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) opensearch-ipoid.discovery.wmnet on all recursors [production]
14:58 <bking@cumin2002> START - Cookbook sre.dns.wipe-cache opensearch-ipoid.discovery.wmnet on all recursors [production]
14:58 <bking@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) opensearch-test.discovery.wmnet on all recursors [production]
14:58 <bking@cumin2002> START - Cookbook sre.dns.wipe-cache opensearch-test.discovery.wmnet on all recursors [production]
14:45 <bking@cumin2002> conftool action : set/pooled=false; selector: dnsdisc=k8s-ingress-dse-aa,name=eqiad [production]
14:37 <bking@cumin2002> conftool action : set/pooled=true; selector: dnsdisc=k8s-ingress-dse-aa,name=eqiad [production]
14:36 <bking@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) k8s-ingress-dse-aa.discovery.wmnet on all recursors [production]