|
2026-10-27
ยง
|
| 09:51 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1048.eqiad.wmnet |
[production] |
| 09:51 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1048.eqiad.wmnet |
[production] |
| 09:49 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) config_reloading P{lvs4009.ulsfo.wmnet} and A:liberica |
[production] |
| 09:49 |
<vgutierrez@cumin1004> |
START - Cookbook sre.loadbalancer.admin config_reloading P{lvs4009.ulsfo.wmnet} and A:liberica |
[production] |
| 09:49 |
<godog> |
extend silence ATSBackendErrorsHigh eventstreams.discovery.wmnet to all sites for another ~8h - T442462 |
[production] |
| 09:48 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) config_reloading P{lvs4009.ulsfo.wmnet} and A:liberica |
[production] |
| 09:48 |
<vgutierrez@cumin1004> |
START - Cookbook sre.loadbalancer.admin config_reloading P{lvs4009.ulsfo.wmnet} and A:liberica |
[production] |
| 09:46 |
<jmm@cumin2003> |
START - Cookbook sre.hosts.reboot-single for host ganeti1048.eqiad.wmnet |
[production] |
| 09:46 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2044.codfw.wmnet |
[production] |
| 09:45 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2043.codfw.wmnet |
[production] |
| 09:45 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2043.codfw.wmnet |
[production] |
| 09:41 |
<elukey@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host sretest2006.codfw.wmnet (push config only) |
[production] |
| 09:41 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.dhcp for host sretest2006.codfw.wmnet (push config only) |
[production] |
| 09:41 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS trixie |
[production] |
| 09:39 |
<elukey@cumin1004> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest2006.codfw.wmnet with OS trixie |
[production] |
| 09:39 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1048.eqiad.wmnet |
[production] |
| 09:39 |
<jmm@cumin2003> |
START - Cookbook sre.hosts.reboot-single for host ganeti2043.codfw.wmnet |
[production] |
| 09:37 |
<elukey@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host sretest2006.codfw.wmnet (push config only) |
[production] |
| 09:37 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.dhcp for host sretest2006.codfw.wmnet (push config only) |
[production] |
| 09:37 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS trixie |
[production] |
| 09:36 |
<elukey@cumin1004> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host ml-serve1016.eqiad.wmnet with OS trixie |
[production] |
| 09:36 |
<elukey@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host ml-serve1016.eqiad.wmnet (remove config only) |
[production] |
| 09:36 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.dhcp for host ml-serve1016.eqiad.wmnet (remove config only) |
[production] |
| 09:35 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1047.eqiad.wmnet |
[production] |
| 09:34 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1047.eqiad.wmnet |
[production] |
| 09:29 |
<slyngshede@cumin1004> |
cookbooks.sre.cdn.roll-reboot finished rebooting cp2055.codfw.wmnet |
[production] |
| 09:29 |
<jmm@cumin2003> |
START - Cookbook sre.hosts.reboot-single for host ganeti1047.eqiad.wmnet |
[production] |
| 09:27 |
<elukey@cumin1004> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest2006.codfw.wmnet with OS trixie |
[production] |
| 09:25 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2043.codfw.wmnet |
[production] |
| 09:24 |
<moritzm> |
installing ca-certificates updates for trixie |
[production] |
| 09:24 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2042.codfw.wmnet |
[production] |
| 09:23 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2042.codfw.wmnet |
[production] |
| 09:18 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1047.eqiad.wmnet |
[production] |
| 09:18 |
<jmm@cumin2003> |
START - Cookbook sre.hosts.reboot-single for host ganeti2042.codfw.wmnet |
[production] |
| 09:17 |
<elukey@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host ml-serve1016.eqiad.wmnet (push config only) |
[production] |
| 09:17 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.dhcp for host ml-serve1016.eqiad.wmnet (push config only) |
[production] |
| 09:17 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.reimage for host ml-serve1016.eqiad.wmnet with OS trixie |
[production] |
| 09:16 |
<elukey@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host sretest2006.codfw.wmnet (push config only) |
[production] |
| 09:16 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.dhcp for host sretest2006.codfw.wmnet (push config only) |
[production] |
| 09:16 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS trixie |
[production] |
| 09:15 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs4010.ulsfo.wmnet with OS trixie |
[production] |
| 09:15 |
<elukey@cumin1004> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest2006.codfw.wmnet with OS trixie |
[production] |
| 09:11 |
<fabfur@cumin1004> |
cookbooks.sre.cdn.roll-reboot finished rebooting cp3076.esams.wmnet |
[production] |
| 09:08 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1045.eqiad.wmnet |
[production] |
| 09:08 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1045.eqiad.wmnet |
[production] |
| 09:07 |
<tappof> |
bump space for prometheus k8s-(dse|aux|ml-serve) in eqiad |
[production] |
| 09:05 |
<elukey@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host sretest2006.codfw.wmnet (push config only) |
[production] |
| 09:05 |
<hashar@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1358880|Allow those with CheckUser right to view private filters on WMF projects (T442464)]] (duration: 15m 12s) |
[production] |
| 09:05 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.dhcp for host sretest2006.codfw.wmnet (push config only) |
[production] |
| 09:05 |
<elukey@cumin1004> |
START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS trixie |
[production] |