901-950 of 10000 results (126ms)
2025-06-24 ยง
12:11 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
12:11 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc1051.eqiad.wmnet [production]
12:10 <cmooney@cumin1003> END (PASS) - Cookbook sre.dns.netbox_snippets (exit_code=0) Generate and push DNS records from Netbox data [production]
12:10 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
12:09 <cmooney@cumin1003> END (FAIL) - Cookbook sre.dns.netbox_snippets (exit_code=99) Generate and push DNS records from Netbox data [production]
12:09 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
12:09 <mvernon@cumin1002> START - Cookbook sre.swift.check-dbs Checking container DBs of wikipedia-commons-local-thumb.06 [production]
12:09 <mvernon@cumin1002> END (PASS) - Cookbook sre.swift.check-dbs (exit_code=0) Checking container DBs of wikipedia-commons-local-thumb.05 [production]
12:06 <jiji@cumin1003> START - Cookbook sre.hosts.reboot-single for host mc2042.codfw.wmnet [production]
12:05 <jiji@cumin1003> START - Cookbook sre.hosts.reboot-single for host mc1051.eqiad.wmnet [production]
12:05 <cmooney@cumin1003> END (PASS) - Cookbook sre.dns.netbox_snippets (exit_code=0) Generate and push DNS records from Netbox data [production]
12:05 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
12:05 <Emperor> check thumbnail db integrity T383053 [production]
12:04 <akosiaris@cumin1003> START - Cookbook sre.hosts.reimage for host aux-k8s-worker1009.eqiad.wmnet with OS bookworm [production]
12:03 <jgiannelos@deploy1003> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
12:03 <jgiannelos@deploy1003> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
12:02 <cmooney@cumin1003> END (PASS) - Cookbook sre.dns.netbox_snippets (exit_code=0) Generate and push DNS records from Netbox data [production]
12:02 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
12:02 <cmooney@cumin1003> END (PASS) - Cookbook sre.dns.netbox_snippets (exit_code=0) Generate and push DNS records from Netbox data [production]
12:02 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
12:02 <akosiaris@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aux-k8s-worker1008.eqiad.wmnet with reason: host reimage [production]
11:59 <cmooney@cumin1003> END (PASS) - Cookbook sre.dns.netbox_snippets (exit_code=0) Generate and push DNS records from Netbox data [production]
11:58 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
11:58 <cmooney@cumin1003> END (FAIL) - Cookbook sre.dns.netbox_snippets (exit_code=99) Generate and push DNS records from Netbox data [production]
11:58 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
11:57 <mvernon@cumin1002> START - Cookbook sre.swift.check-dbs Checking container DBs of wikipedia-commons-local-thumb.05 [production]
11:57 <hnowlan@deploy1003> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
11:56 <akosiaris@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on aux-k8s-worker1008.eqiad.wmnet with reason: host reimage [production]
11:56 <hnowlan@deploy1003> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
11:54 <stevemunene@cumin1002> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host an-worker1176.eqiad.wmnet [production]
11:50 <cmooney@cumin1003> END (PASS) - Cookbook sre.dns.netbox_snippets (exit_code=0) Generate and push DNS records from Netbox data [production]
11:50 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
11:48 <cmooney@cumin1003> END (FAIL) - Cookbook sre.dns.netbox_snippets (exit_code=99) Generate and push DNS records from Netbox data [production]
11:48 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
11:46 <mvernon@cumin1002> END (PASS) - Cookbook sre.swift.check-dbs (exit_code=0) Checking container DBs of wikipedia-commons-local-thumb.04 [production]
11:45 <akosiaris@cumin1003> START - Cookbook sre.hosts.reimage for host aux-k8s-worker1008.eqiad.wmnet with OS bookworm [production]
11:44 <jelto> bump kubernetes-client to newest version on deploy1003 and deploy2002 - T387548 [production]
11:43 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc2041.codfw.wmnet [production]
11:39 <mvernon@cumin1002> START - Cookbook sre.swift.check-dbs Checking container DBs of wikipedia-commons-local-thumb.04 [production]
11:38 <mvernon@cumin1002> END (PASS) - Cookbook sre.swift.check-dbs (exit_code=0) Checking container DBs of wikipedia-commons-local-thumb.03 [production]
11:37 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc1052.eqiad.wmnet [production]
11:37 <jiji@cumin1003> START - Cookbook sre.hosts.reboot-single for host mc2041.codfw.wmnet [production]
11:36 <cmooney@cumin1003> END (FAIL) - Cookbook sre.dns.netbox_snippets (exit_code=99) Generate and push DNS records from Netbox data [production]
11:36 <cmooney@cumin1003> START - Cookbook sre.dns.netbox_snippets Generate and push DNS records from Netbox data [production]
11:32 <jiji@cumin1003> START - Cookbook sre.hosts.reboot-single for host mc1052.eqiad.wmnet [production]
11:31 <mvernon@cumin1002> START - Cookbook sre.swift.check-dbs Checking container DBs of wikipedia-commons-local-thumb.03 [production]
11:31 <mvernon@cumin1002> END (PASS) - Cookbook sre.swift.check-dbs (exit_code=0) Checking container DBs of wikipedia-commons-local-thumb.02 [production]
11:31 <hnowlan@deploy1003> helmfile [staging] DONE helmfile.d/services/mobileapps: sync [production]
11:28 <jelto> bump kubernetes-client to newest version on kubestagemaster100[3-5] - T387548 [production]
11:28 <jiji@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc2040.codfw.wmnet [production]