3401-3450 of 10000 results (91ms)
2023-10-18 ยง
16:43 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp1101.eqiad.wmnet with reason: host reimage [production]
16:40 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp1102.eqiad.wmnet with reason: host reimage [production]
16:40 <jclark@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp1101.eqiad.wmnet with reason: host reimage [production]
16:39 <jclark@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp1100.eqiad.wmnet with reason: host reimage [production]
16:37 <jclark@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp1102.eqiad.wmnet with reason: host reimage [production]
16:33 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1010.eqiad.wmnet with OS bullseye [production]
16:30 <jforrester@deploy2002> helmfile [staging] DONE helmfile.d/services/wikifunctions: apply [production]
16:30 <jforrester@deploy2002> helmfile [staging] START helmfile.d/services/wikifunctions: apply [production]
16:29 <jforrester@deploy2002> helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply [production]
16:28 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp1103.eqiad.wmnet with OS bullseye [production]
16:28 <jclark@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
16:28 <jforrester@deploy2002> helmfile [eqiad] START helmfile.d/services/wikifunctions: apply [production]
16:28 <jforrester@deploy2002> helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply [production]
16:26 <jforrester@deploy2002> helmfile [codfw] START helmfile.d/services/wikifunctions: apply [production]
16:25 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cp1102.eqiad.wmnet with OS bullseye [production]
16:24 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
16:24 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1102'] [production]
16:23 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cp1101.eqiad.wmnet with OS bullseye [production]
16:22 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cp1100.eqiad.wmnet with OS bullseye [production]
16:22 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1100'] [production]
16:20 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1100'] [production]
16:20 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1101'] [production]
16:20 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1101'] [production]
16:19 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host cp1110.eqiad.wmnet with OS bullseye [production]
16:19 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1110'] [production]
16:18 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1110'] [production]
16:18 <jforrester@deploy2002> helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply [production]
16:18 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1102'] [production]
16:18 <jclark@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:18 <jclark@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt cp1102 - jclark@cumin1001" [production]
16:17 <jforrester@deploy2002> helmfile [codfw] START helmfile.d/services/wikifunctions: apply [production]
16:17 <jforrester@deploy2002> helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply [production]
16:17 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt cp1102 - jclark@cumin1001" [production]
16:17 <jforrester@deploy2002> helmfile [eqiad] START helmfile.d/services/wikifunctions: apply [production]
16:16 <jforrester@deploy2002> helmfile [staging] DONE helmfile.d/services/wikifunctions: apply [production]
16:15 <jforrester@deploy2002> helmfile [staging] START helmfile.d/services/wikifunctions: apply [production]
16:15 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
16:14 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1102'] [production]
16:14 <jclark@cumin1001> END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts ['cp1110'] [production]
16:14 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1110'] [production]
16:14 <jclark@cumin1001> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cp1102 [production]
16:13 <jclark@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host cp1102 [production]
16:11 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1010.eqiad.wmnet with reason: host reimage [production]
16:10 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1110'] [production]
16:08 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1110'] [production]
16:08 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp1103.eqiad.wmnet with reason: host reimage [production]
16:08 <eevans@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1010.eqiad.wmnet with reason: host reimage [production]
16:08 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1102'] [production]
16:07 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1102'] [production]
16:07 <jclark@cumin1001> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cp1102 [production]