|
2021-03-11
ยง
|
| 11:21 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1048.eqiad.wmnet |
[production] |
| 11:16 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1047.eqiad.wmnet |
[production] |
| 11:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 60%: Repool db1096:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14777 and previous config saved to /var/cache/conftool/dbconfig/20210311-111243-root.json |
[production] |
| 11:11 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1047.eqiad.wmnet |
[production] |
| 11:04 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1046.eqiad.wmnet |
[production] |
| 10:59 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1046.eqiad.wmnet |
[production] |
| 10:58 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1045.eqiad.wmnet |
[production] |
| 10:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 30%: Repool db1096:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14776 and previous config saved to /var/cache/conftool/dbconfig/20210311-105740-root.json |
[production] |
| 10:54 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1045.eqiad.wmnet |
[production] |
| 10:54 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1044.eqiad.wmnet |
[production] |
| 10:49 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1044.eqiad.wmnet |
[production] |
| 10:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 10%: Repool db1096:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14775 and previous config saved to /var/cache/conftool/dbconfig/20210311-104236-root.json |
[production] |
| 10:42 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1043.eqiad.wmnet |
[production] |
| 10:35 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1043.eqiad.wmnet |
[production] |
| 10:35 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1042.eqiad.wmnet |
[production] |
| 10:30 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1042.eqiad.wmnet |
[production] |
| 10:29 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1041.eqiad.wmnet |
[production] |
| 10:22 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1041.eqiad.wmnet |
[production] |
| 10:22 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1040.eqiad.wmnet |
[production] |
| 10:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1096:3315 for schema change', diff saved to https://phabricator.wikimedia.org/P14774 and previous config saved to /var/cache/conftool/dbconfig/20210311-101714-marostegui.json |
[production] |
| 10:16 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1040.eqiad.wmnet |
[production] |
| 10:16 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1039.eqiad.wmnet |
[production] |
| 10:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Add db2149 to dbctl, depooled, T275633', diff saved to https://phabricator.wikimedia.org/P14773 and previous config saved to /var/cache/conftool/dbconfig/20210311-101604-marostegui.json |
[production] |
| 10:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 100%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P14772 and previous config saved to /var/cache/conftool/dbconfig/20210311-101008-root.json |
[production] |
| 10:09 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1039.eqiad.wmnet |
[production] |
| 10:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db2109', diff saved to https://phabricator.wikimedia.org/P14771 and previous config saved to /var/cache/conftool/dbconfig/20210311-100705-marostegui.json |
[production] |
| 10:00 |
<mbsantos@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'proton' for release 'production' . |
[production] |
| 09:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 60%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P14770 and previous config saved to /var/cache/conftool/dbconfig/20210311-095504-root.json |
[production] |
| 09:51 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1038.eqiad.wmnet |
[production] |
| 09:45 |
<marostegui> |
Deploy schema change on s5 codfw master, lag will appear - T276150 T276156 |
[production] |
| 09:43 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1038.eqiad.wmnet |
[production] |
| 09:40 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 30%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P14769 and previous config saved to /var/cache/conftool/dbconfig/20210311-094000-root.json |
[production] |
| 09:35 |
<akosiaris@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'recommendation-api' for release 'production' . |
[production] |
| 09:31 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1037.eqiad.wmnet |
[production] |
| 09:31 |
<hashar> |
Restarting CI Jenkins |
[production] |
| 09:29 |
<akosiaris@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'recommendation-api' for release 'production' . |
[production] |
| 09:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 10%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P14768 and previous config saved to /var/cache/conftool/dbconfig/20210311-092457-root.json |
[production] |
| 09:24 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1037.eqiad.wmnet |
[production] |
| 09:23 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1036.eqiad.wmnet |
[production] |
| 09:19 |
<effie> |
upgrade memcached on mc1020, mc2020 |
[production] |
| 09:16 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1036.eqiad.wmnet |
[production] |
| 09:11 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1035.eqiad.wmnet |
[production] |
| 09:08 |
<akosiaris@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'recommendation-api' for release 'production' . |
[production] |
| 09:04 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1035.eqiad.wmnet |
[production] |
| 09:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1174', diff saved to https://phabricator.wikimedia.org/P14767 and previous config saved to /var/cache/conftool/dbconfig/20210311-090342-marostegui.json |
[production] |
| 09:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1079 (re)pooling @ 100%: Repool db1079 after schema change', diff saved to https://phabricator.wikimedia.org/P14766 and previous config saved to /var/cache/conftool/dbconfig/20210311-090312-root.json |
[production] |
| 09:03 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1033.eqiad.wmnet |
[production] |
| 08:58 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1033.eqiad.wmnet |
[production] |
| 08:58 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1032.eqiad.wmnet |
[production] |
| 08:51 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1032.eqiad.wmnet |
[production] |