|
2023-09-28
ยง
|
| 23:05 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2171:3316 (T343198)', diff saved to https://phabricator.wikimedia.org/P52743 and previous config saved to /var/cache/conftool/dbconfig/20230928-230512-arnaudb.json |
[production] |
| 22:57 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2110 (T343198)', diff saved to https://phabricator.wikimedia.org/P52742 and previous config saved to /var/cache/conftool/dbconfig/20230928-225705-arnaudb.json |
[production] |
| 22:40 |
<wfan> |
payments-wiki change from c4c9b938 to 20828b07 |
[production] |
| 22:03 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudservices2005-dev.codfw.wmnet with reason: host reimage |
[production] |
| 22:02 |
<bking@cumin1001> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1007.eqiad.wmnet with OS bullseye |
[production] |
| 22:00 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudservices2005-dev.codfw.wmnet with reason: host reimage |
[production] |
| 21:58 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1033.eqiad.wmnet |
[production] |
| 21:58 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1033.eqiad.wmnet |
[production] |
| 21:58 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1030.eqiad.wmnet |
[production] |
| 21:57 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1030.eqiad.wmnet |
[production] |
| 21:57 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1027.eqiad.wmnet |
[production] |
| 21:57 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1027.eqiad.wmnet |
[production] |
| 21:57 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1026.eqiad.wmnet |
[production] |
| 21:57 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1026.eqiad.wmnet |
[production] |
| 21:56 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1025.eqiad.wmnet |
[production] |
| 21:56 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1025.eqiad.wmnet |
[production] |
| 21:56 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1032.eqiad.wmnet |
[production] |
| 21:56 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1032.eqiad.wmnet |
[production] |
| 21:55 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1029.eqiad.wmnet |
[production] |
| 21:55 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1029.eqiad.wmnet |
[production] |
| 21:55 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1029.eqiad.wmnet |
[production] |
| 21:55 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1029.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1024.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1024.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1023.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1023.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1022.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1022.eqiad.wmnet |
[production] |
| 21:53 |
<wfan> |
payments-wiki change from 505a616d to 20828b07 |
[production] |
| 21:52 |
<bking@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudelastic1007.eqiad.wmnet with OS bullseye |
[production] |
| 21:42 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1031.eqiad.wmnet |
[production] |
| 21:42 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1031.eqiad.wmnet |
[production] |
| 21:40 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudservices2005-dev.codfw.wmnet with OS bookworm |
[production] |
| 21:30 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1028.eqiad.wmnet |
[production] |
| 21:30 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1028.eqiad.wmnet |
[production] |
| 21:28 |
<bking@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['cloudelastic1007.wikimedia.org'] |
[production] |
| 21:25 |
<bking@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1007.wikimedia.org'] |
[production] |
| 21:25 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cloudelastic1007.wikimedia.org'] |
[production] |
| 21:14 |
<bking@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1007.wikimedia.org'] |
[production] |
| 21:13 |
<bking@cumin1001> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1007.eqiad.wmnet with OS bullseye |
[production] |
| 21:07 |
<thcipriani@deploy2002> |
Finished scap: Backport for [[gerrit:961471|Drop the desktop improvements dblist group (T347444)]] (duration: 11m 22s) |
[production] |
| 21:00 |
<thcipriani@deploy2002> |
jdlrobson and thcipriani: Continuing with sync |
[production] |
| 20:59 |
<bking@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudelastic1007.eqiad.wmnet with OS bullseye |
[production] |
| 20:57 |
<thcipriani@deploy2002> |
jdlrobson and thcipriani: Backport for [[gerrit:961471|Drop the desktop improvements dblist group (T347444)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 20:55 |
<thcipriani@deploy2002> |
Started scap: Backport for [[gerrit:961471|Drop the desktop improvements dblist group (T347444)]] |
[production] |
| 20:55 |
<bking@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudelastic1007.eqiad.wmnet with OS bullseye |
[production] |
| 20:55 |
<bking@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudelastic1007.eqiad.wmnet with OS bullseye |
[production] |
| 20:52 |
<thcipriani@deploy2002> |
Finished scap: Backport for [[gerrit:961803|update sawikiquote logos (T341260)]], [[gerrit:961484|Wikimedia special project logo updates]] (duration: 16m 32s) |
[production] |
| 20:45 |
<thcipriani@deploy2002> |
anzx and jdlrobson and thcipriani: Continuing with sync |
[production] |
| 20:36 |
<thcipriani@deploy2002> |
anzx and jdlrobson and thcipriani: Backport for [[gerrit:961803|update sawikiquote logos (T341260)]], [[gerrit:961484|Wikimedia special project logo updates]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |