|
2023-11-23
ยง
|
| 12:11 |
<stevemunene@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host druid1008.eqiad.wmnet with OS bullseye |
[production] |
| 12:10 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'db2146 (re)pooling @ 10%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53745 and previous config saved to /var/cache/conftool/dbconfig/20231123-121054-arnaudb.json |
[production] |
| 12:08 |
<arnaudb@cumin1001> |
END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db2146.codfw.wmnet onto db2188.codfw.wmnet |
[production] |
| 12:06 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'db2149 (re)pooling @ 50%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53744 and previous config saved to /var/cache/conftool/dbconfig/20231123-120650-arnaudb.json |
[production] |
| 11:51 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'db2149 (re)pooling @ 40%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53743 and previous config saved to /var/cache/conftool/dbconfig/20231123-115145-arnaudb.json |
[production] |
| 11:36 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'db2149 (re)pooling @ 30%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53742 and previous config saved to /var/cache/conftool/dbconfig/20231123-113640-arnaudb.json |
[production] |
| 11:30 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host lists1004.eqiad.wmnet |
[production] |
| 11:25 |
<stevemunene@cumin1001> |
START - Cookbook sre.hosts.reimage for host druid1008.eqiad.wmnet with OS bullseye |
[production] |
| 11:24 |
<arnaudb@cumin1001> |
END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db2175.codfw.wmnet onto db2189.codfw.wmnet |
[production] |
| 11:23 |
<jmm@cumin2002> |
START - Cookbook sre.puppet.migrate-host for host lists1004.eqiad.wmnet |
[production] |
| 11:21 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'db2149 (re)pooling @ 20%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53741 and previous config saved to /var/cache/conftool/dbconfig/20231123-112135-arnaudb.json |
[production] |
| 11:16 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dborch1001.wikimedia.org |
[production] |
| 11:13 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host dborch1001.wikimedia.org |
[production] |
| 11:12 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . |
[production] |
| 11:12 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . |
[production] |
| 11:12 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . |
[production] |
| 11:12 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . |
[production] |
| 11:12 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . |
[production] |
| 11:11 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . |
[production] |
| 11:11 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . |
[production] |
| 11:11 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . |
[production] |
| 11:11 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . |
[production] |
| 11:11 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . |
[production] |
| 11:11 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . |
[production] |
| 11:11 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . |
[production] |
| 11:11 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . |
[production] |
| 11:10 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . |
[production] |
| 11:06 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'db2149 (re)pooling @ 10%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53740 and previous config saved to /var/cache/conftool/dbconfig/20231123-110630-arnaudb.json |
[production] |
| 11:03 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: orchestrator |
[production] |
| 10:59 |
<arnaudb@cumin1001> |
END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db2149.codfw.wmnet onto db2190.codfw.wmnet |
[production] |
| 10:52 |
<jmm@cumin2002> |
START - Cookbook sre.puppet.migrate-role for role: orchestrator |
[production] |
| 10:50 |
<arnaudb@cumin1001> |
START - Cookbook sre.mysql.clone of db2181.codfw.wmnet onto db2195.codfw.wmnet |
[production] |
| 10:47 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Cloning db2181 in db2195 for T343674', diff saved to https://phabricator.wikimedia.org/P53739 and previous config saved to /var/cache/conftool/dbconfig/20231123-104724-arnaudb.json |
[production] |
| 10:45 |
<arnaudb@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2195.codfw.wmnet with reason: provisionning db2195.codfw.wmnet - T343674 |
[production] |
| 10:45 |
<arnaudb@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2195.codfw.wmnet with reason: provisionning db2195.codfw.wmnet - T343674 |
[production] |
| 10:45 |
<arnaudb@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2181.codfw.wmnet with reason: provisionning db2195.codfw.wmnet - T343674 |
[production] |
| 10:45 |
<arnaudb@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2181.codfw.wmnet with reason: provisionning db2195.codfw.wmnet - T343674 |
[production] |
| 10:34 |
<stevemunene@cumin1001> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host druid1008.eqiad.wmnet with OS bullseye |
[production] |
| 10:31 |
<arnaudb@cumin1001> |
START - Cookbook sre.mysql.clone of db2146.codfw.wmnet onto db2188.codfw.wmnet |
[production] |
| 10:30 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: swift::proxy |
[production] |
| 10:28 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Cloning db2146 in db2188 for T343674', diff saved to https://phabricator.wikimedia.org/P53738 and previous config saved to /var/cache/conftool/dbconfig/20231123-102840-arnaudb.json |
[production] |
| 10:27 |
<arnaudb@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2188.codfw.wmnet with reason: provisionning db2188.codfw.wmnet - T343674 |
[production] |
| 10:27 |
<arnaudb@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2188.codfw.wmnet with reason: provisionning db2188.codfw.wmnet - T343674 |
[production] |
| 10:27 |
<arnaudb@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2146.codfw.wmnet with reason: provisionning db2188.codfw.wmnet - T343674 |
[production] |
| 10:27 |
<arnaudb@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2146.codfw.wmnet with reason: provisionning db2188.codfw.wmnet - T343674 |
[production] |
| 10:22 |
<stevemunene@cumin1001> |
START - Cookbook sre.hosts.reimage for host druid1008.eqiad.wmnet with OS bullseye |
[production] |
| 10:16 |
<jmm@cumin2002> |
START - Cookbook sre.puppet.migrate-role for role: swift::proxy |
[production] |
| 10:09 |
<arnaudb@cumin1001> |
START - Cookbook sre.mysql.clone of db2175.codfw.wmnet onto db2189.codfw.wmnet |
[production] |
| 10:06 |
<arnaudb@cumin1001> |
dbctl commit (dc=all): 'Cloning db2175 in db2189 for T343674', diff saved to https://phabricator.wikimedia.org/P53737 and previous config saved to /var/cache/conftool/dbconfig/20231123-100638-arnaudb.json |
[production] |
| 10:05 |
<arnaudb@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2189.codfw.wmnet with reason: provisionning db2189.codfw.wmnet - T343674 |
[production] |