2651-2700 of 10000 results (36ms)
2021-03-11 ยง
11:21 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1048.eqiad.wmnet [production]
11:16 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1047.eqiad.wmnet [production]
11:12 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 60%: Repool db1096:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14777 and previous config saved to /var/cache/conftool/dbconfig/20210311-111243-root.json [production]
11:11 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1047.eqiad.wmnet [production]
11:04 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1046.eqiad.wmnet [production]
10:59 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1046.eqiad.wmnet [production]
10:58 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1045.eqiad.wmnet [production]
10:57 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 30%: Repool db1096:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14776 and previous config saved to /var/cache/conftool/dbconfig/20210311-105740-root.json [production]
10:54 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1045.eqiad.wmnet [production]
10:54 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1044.eqiad.wmnet [production]
10:49 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1044.eqiad.wmnet [production]
10:42 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 10%: Repool db1096:3315 after schema change', diff saved to https://phabricator.wikimedia.org/P14775 and previous config saved to /var/cache/conftool/dbconfig/20210311-104236-root.json [production]
10:42 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1043.eqiad.wmnet [production]
10:35 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1043.eqiad.wmnet [production]
10:35 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1042.eqiad.wmnet [production]
10:30 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1042.eqiad.wmnet [production]
10:29 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1041.eqiad.wmnet [production]
10:22 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1041.eqiad.wmnet [production]
10:22 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1040.eqiad.wmnet [production]
10:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1096:3315 for schema change', diff saved to https://phabricator.wikimedia.org/P14774 and previous config saved to /var/cache/conftool/dbconfig/20210311-101714-marostegui.json [production]
10:16 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1040.eqiad.wmnet [production]
10:16 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1039.eqiad.wmnet [production]
10:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Add db2149 to dbctl, depooled, T275633', diff saved to https://phabricator.wikimedia.org/P14773 and previous config saved to /var/cache/conftool/dbconfig/20210311-101604-marostegui.json [production]
10:10 <marostegui@cumin1001> dbctl commit (dc=all): 'db1174 (re)pooling @ 100%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P14772 and previous config saved to /var/cache/conftool/dbconfig/20210311-101008-root.json [production]
10:09 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1039.eqiad.wmnet [production]
10:07 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2109', diff saved to https://phabricator.wikimedia.org/P14771 and previous config saved to /var/cache/conftool/dbconfig/20210311-100705-marostegui.json [production]
10:00 <mbsantos@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'proton' for release 'production' . [production]
09:55 <marostegui@cumin1001> dbctl commit (dc=all): 'db1174 (re)pooling @ 60%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P14770 and previous config saved to /var/cache/conftool/dbconfig/20210311-095504-root.json [production]
09:51 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1038.eqiad.wmnet [production]
09:45 <marostegui> Deploy schema change on s5 codfw master, lag will appear - T276150 T276156 [production]
09:43 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1038.eqiad.wmnet [production]
09:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1174 (re)pooling @ 30%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P14769 and previous config saved to /var/cache/conftool/dbconfig/20210311-094000-root.json [production]
09:35 <akosiaris@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'recommendation-api' for release 'production' . [production]
09:31 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1037.eqiad.wmnet [production]
09:31 <hashar> Restarting CI Jenkins [production]
09:29 <akosiaris@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'recommendation-api' for release 'production' . [production]
09:24 <marostegui@cumin1001> dbctl commit (dc=all): 'db1174 (re)pooling @ 10%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P14768 and previous config saved to /var/cache/conftool/dbconfig/20210311-092457-root.json [production]
09:24 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1037.eqiad.wmnet [production]
09:23 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1036.eqiad.wmnet [production]
09:19 <effie> upgrade memcached on mc1020, mc2020 [production]
09:16 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1036.eqiad.wmnet [production]
09:11 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1035.eqiad.wmnet [production]
09:08 <akosiaris@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'recommendation-api' for release 'production' . [production]
09:04 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1035.eqiad.wmnet [production]
09:03 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1174', diff saved to https://phabricator.wikimedia.org/P14767 and previous config saved to /var/cache/conftool/dbconfig/20210311-090342-marostegui.json [production]
09:03 <marostegui@cumin1001> dbctl commit (dc=all): 'db1079 (re)pooling @ 100%: Repool db1079 after schema change', diff saved to https://phabricator.wikimedia.org/P14766 and previous config saved to /var/cache/conftool/dbconfig/20210311-090312-root.json [production]
09:03 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1033.eqiad.wmnet [production]
08:58 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1033.eqiad.wmnet [production]
08:58 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1032.eqiad.wmnet [production]
08:51 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host ms-be1032.eqiad.wmnet [production]