|
2023-11-27
ยง
|
| 15:45 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['elastic2103'] |
[production] |
| 15:45 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2108'] |
[production] |
| 15:44 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2107'] |
[production] |
| 15:44 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2106'] |
[production] |
| 15:44 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2105'] |
[production] |
| 15:44 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2104'] |
[production] |
| 15:44 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2103'] |
[production] |
| 15:44 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic2104.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:44 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic2103.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:44 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic2106.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:43 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic2107.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:43 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic2105.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:43 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic2108.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:35 |
<jclark@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host an-worker1157.eqiad.wmnet with OS bullseye |
[production] |
| 15:31 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host elastic2108.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:31 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host elastic2107.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:31 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host elastic2106.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:31 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host elastic2105.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:31 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host elastic2104.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:31 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host elastic2103.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:29 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host elastic2101.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:29 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host elastic2097.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:27 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['elastic2098'] |
[production] |
| 15:27 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['elastic2099'] |
[production] |
| 15:27 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['elastic2102'] |
[production] |
| 15:21 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['elastic2100'] |
[production] |
| 15:20 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2102'] |
[production] |
| 15:20 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2100'] |
[production] |
| 15:20 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2099'] |
[production] |
| 15:20 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2098'] |
[production] |
| 15:20 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['elastic2102'] |
[production] |
| 15:20 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['elastic2100'] |
[production] |
| 15:20 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['elastic2099'] |
[production] |
| 15:20 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['elastic2098'] |
[production] |
| 15:20 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2102'] |
[production] |
| 15:19 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2100'] |
[production] |
| 15:19 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2099'] |
[production] |
| 15:19 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['elastic2098'] |
[production] |
| 15:19 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic2100.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:19 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic2099.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:19 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic2102.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:19 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic2098.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:14 |
<fabfur> |
set `pooled=yes` on cp11.* hosts in eqiad T349244 |
[production] |
| 15:11 |
<jclark@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host an-worker1160.eqiad.wmnet with OS bullseye |
[production] |
| 15:10 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirtlocal1001.eqiad.wmnet with reason: host reimage |
[production] |
| 15:09 |
<hashar> |
restarting CI Jenkins |
[production] |
| 15:07 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host elastic2102.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:07 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host elastic2101.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:07 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host elastic2100.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 15:07 |
<fabfur> |
`nfctl select name='cp10.*',service=ats-be set/pooled=inactive` (cdn and ats-be not used anymore on these hosts) T349244 |
[production] |