|
2023-09-28
ยง
|
| 23:42 |
<arnaudb@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2119.codfw.wmnet with reason: Maintenance |
[production] |
| 23:42 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2110 (T343198)', diff saved to https://phabricator.wikimedia.org/P52748 and previous config saved to /var/cache/conftool/dbconfig/20230928-234224-arnaudb.json |
[production] |
| 23:35 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2171:3316', diff saved to https://phabricator.wikimedia.org/P52747 and previous config saved to /var/cache/conftool/dbconfig/20230928-233525-arnaudb.json |
[production] |
| 23:27 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2110', diff saved to https://phabricator.wikimedia.org/P52746 and previous config saved to /var/cache/conftool/dbconfig/20230928-232718-arnaudb.json |
[production] |
| 23:20 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2171:3316', diff saved to https://phabricator.wikimedia.org/P52745 and previous config saved to /var/cache/conftool/dbconfig/20230928-232019-arnaudb.json |
[production] |
| 23:12 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2110', diff saved to https://phabricator.wikimedia.org/P52744 and previous config saved to /var/cache/conftool/dbconfig/20230928-231211-arnaudb.json |
[production] |
| 23:05 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2171:3316 (T343198)', diff saved to https://phabricator.wikimedia.org/P52743 and previous config saved to /var/cache/conftool/dbconfig/20230928-230512-arnaudb.json |
[production] |
| 22:57 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2110 (T343198)', diff saved to https://phabricator.wikimedia.org/P52742 and previous config saved to /var/cache/conftool/dbconfig/20230928-225705-arnaudb.json |
[production] |
| 22:40 |
<wfan> |
payments-wiki change from c4c9b938 to 20828b07 |
[production] |
| 22:03 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudservices2005-dev.codfw.wmnet with reason: host reimage |
[production] |
| 22:02 |
<bking@cumin1001> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1007.eqiad.wmnet with OS bullseye |
[production] |
| 22:00 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudservices2005-dev.codfw.wmnet with reason: host reimage |
[production] |
| 21:58 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1033.eqiad.wmnet |
[production] |
| 21:58 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1033.eqiad.wmnet |
[production] |
| 21:58 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1030.eqiad.wmnet |
[production] |
| 21:57 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1030.eqiad.wmnet |
[production] |
| 21:57 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1027.eqiad.wmnet |
[production] |
| 21:57 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1027.eqiad.wmnet |
[production] |
| 21:57 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1026.eqiad.wmnet |
[production] |
| 21:57 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1026.eqiad.wmnet |
[production] |
| 21:56 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1025.eqiad.wmnet |
[production] |
| 21:56 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1025.eqiad.wmnet |
[production] |
| 21:56 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1032.eqiad.wmnet |
[production] |
| 21:56 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1032.eqiad.wmnet |
[production] |
| 21:55 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1029.eqiad.wmnet |
[production] |
| 21:55 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1029.eqiad.wmnet |
[production] |
| 21:55 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1029.eqiad.wmnet |
[production] |
| 21:55 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1029.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1024.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1024.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1023.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1023.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1022.eqiad.wmnet |
[production] |
| 21:54 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1022.eqiad.wmnet |
[production] |
| 21:53 |
<wfan> |
payments-wiki change from 505a616d to 20828b07 |
[production] |
| 21:52 |
<bking@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudelastic1007.eqiad.wmnet with OS bullseye |
[production] |
| 21:42 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1031.eqiad.wmnet |
[production] |
| 21:42 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1031.eqiad.wmnet |
[production] |
| 21:40 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudservices2005-dev.codfw.wmnet with OS bookworm |
[production] |
| 21:30 |
<eevans@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1028.eqiad.wmnet |
[production] |
| 21:30 |
<eevans@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1028.eqiad.wmnet |
[production] |
| 21:28 |
<bking@cumin1001> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['cloudelastic1007.wikimedia.org'] |
[production] |
| 21:25 |
<bking@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1007.wikimedia.org'] |
[production] |
| 21:25 |
<bking@cumin1001> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cloudelastic1007.wikimedia.org'] |
[production] |
| 21:14 |
<bking@cumin1001> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1007.wikimedia.org'] |
[production] |
| 21:13 |
<bking@cumin1001> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1007.eqiad.wmnet with OS bullseye |
[production] |
| 21:07 |
<thcipriani@deploy2002> |
Finished scap: Backport for [[gerrit:961471|Drop the desktop improvements dblist group (T347444)]] (duration: 11m 22s) |
[production] |
| 21:00 |
<thcipriani@deploy2002> |
jdlrobson and thcipriani: Continuing with sync |
[production] |
| 20:59 |
<bking@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudelastic1007.eqiad.wmnet with OS bullseye |
[production] |
| 20:57 |
<thcipriani@deploy2002> |
jdlrobson and thcipriani: Backport for [[gerrit:961471|Drop the desktop improvements dblist group (T347444)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |