|
2021-06-14
ยง
|
| 14:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1170:3312 (re)pooling @ 50%: Repool db1170:3312 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16504 and previous config saved to /var/cache/conftool/dbconfig/20210614-140203-root.json |
[production] |
| 14:01 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host maps2006.codfw.wmnet |
[production] |
| 13:54 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1033 (re)pooling @ 100%: Repool es1033 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16503 and previous config saved to /var/cache/conftool/dbconfig/20210614-135456-root.json |
[production] |
| 13:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1147 (re)pooling @ 10%: Repool db1147 after upgrade', diff saved to https://phabricator.wikimedia.org/P16502 and previous config saved to /var/cache/conftool/dbconfig/20210614-135025-root.json |
[production] |
| 13:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1032 (re)pooling @ 50%: Repool es1032 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16501 and previous config saved to /var/cache/conftool/dbconfig/20210614-135007-root.json |
[production] |
| 13:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1170:3317 (re)pooling @ 25%: Repool db1170:3317 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16500 and previous config saved to /var/cache/conftool/dbconfig/20210614-134713-root.json |
[production] |
| 13:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1170:3312 (re)pooling @ 25%: Repool db1170:3312 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16499 and previous config saved to /var/cache/conftool/dbconfig/20210614-134700-root.json |
[production] |
| 13:43 |
<otto@cumin1001> |
START - Cookbook sre.kafka.roll-restart-brokers |
[production] |
| 13:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1033 (re)pooling @ 75%: Repool es1033 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16498 and previous config saved to /var/cache/conftool/dbconfig/20210614-133953-root.json |
[production] |
| 13:38 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1147 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16497 and previous config saved to /var/cache/conftool/dbconfig/20210614-133801-marostegui.json |
[production] |
| 13:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1032 (re)pooling @ 25%: Repool es1032 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16496 and previous config saved to /var/cache/conftool/dbconfig/20210614-133503-root.json |
[production] |
| 13:34 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1034 (re)pooling @ 100%: Repool es1034 after upgrade', diff saved to https://phabricator.wikimedia.org/P16495 and previous config saved to /var/cache/conftool/dbconfig/20210614-133442-root.json |
[production] |
| 13:32 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1170:3317 (re)pooling @ 10%: Repool db1170:3317 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16494 and previous config saved to /var/cache/conftool/dbconfig/20210614-133210-root.json |
[production] |
| 13:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1170:3312 (re)pooling @ 10%: Repool db1170:3312 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16493 and previous config saved to /var/cache/conftool/dbconfig/20210614-133156-root.json |
[production] |
| 13:29 |
<effie> |
restart memcached on codfw |
[production] |
| 13:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1033 (re)pooling @ 50%: Repool es1033 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16492 and previous config saved to /var/cache/conftool/dbconfig/20210614-132449-root.json |
[production] |
| 13:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1170:3312 db1170:3317 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16491 and previous config saved to /var/cache/conftool/dbconfig/20210614-132235-marostegui.json |
[production] |
| 13:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1032 (re)pooling @ 10%: Repool es1032 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16490 and previous config saved to /var/cache/conftool/dbconfig/20210614-132000-root.json |
[production] |
| 13:19 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1034 (re)pooling @ 75%: Repool es1034 after upgrade', diff saved to https://phabricator.wikimedia.org/P16489 and previous config saved to /var/cache/conftool/dbconfig/20210614-131938-root.json |
[production] |
| 13:09 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1033 (re)pooling @ 25%: Repool es1033 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16488 and previous config saved to /var/cache/conftool/dbconfig/20210614-130946-root.json |
[production] |
| 13:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es1032 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16487 and previous config saved to /var/cache/conftool/dbconfig/20210614-130723-marostegui.json |
[production] |
| 13:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 100%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P16486 and previous config saved to /var/cache/conftool/dbconfig/20210614-130547-root.json |
[production] |
| 13:04 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1034 (re)pooling @ 50%: Repool es1034 after upgrade', diff saved to https://phabricator.wikimedia.org/P16485 and previous config saved to /var/cache/conftool/dbconfig/20210614-130435-root.json |
[production] |
| 12:54 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1033 (re)pooling @ 10%: Repool es1033 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16484 and previous config saved to /var/cache/conftool/dbconfig/20210614-125442-root.json |
[production] |
| 12:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 75%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P16483 and previous config saved to /var/cache/conftool/dbconfig/20210614-125043-root.json |
[production] |
| 12:49 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1034 (re)pooling @ 25%: Repool es1034 after upgrade', diff saved to https://phabricator.wikimedia.org/P16482 and previous config saved to /var/cache/conftool/dbconfig/20210614-124931-root.json |
[production] |
| 12:37 |
<XioNoX> |
configure OSPF link-protection on cr3/4-ulsfo - T167306 |
[production] |
| 12:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 50%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P16481 and previous config saved to /var/cache/conftool/dbconfig/20210614-123539-root.json |
[production] |
| 12:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es1033 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16480 and previous config saved to /var/cache/conftool/dbconfig/20210614-123512-marostegui.json |
[production] |
| 12:34 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1034 (re)pooling @ 10%: Repool es1034 after upgrade', diff saved to https://phabricator.wikimedia.org/P16479 and previous config saved to /var/cache/conftool/dbconfig/20210614-123427-root.json |
[production] |
| 12:23 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Restore es1028 original weight', diff saved to https://phabricator.wikimedia.org/P16478 and previous config saved to /var/cache/conftool/dbconfig/20210614-122322-marostegui.json |
[production] |
| 12:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Give some weight to es1028 while es1034 gets upgraded', diff saved to https://phabricator.wikimedia.org/P16477 and previous config saved to /var/cache/conftool/dbconfig/20210614-122242-marostegui.json |
[production] |
| 12:22 |
<dcausse> |
re-pooling wdqs1012 |
[production] |
| 12:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es1034 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16476 and previous config saved to /var/cache/conftool/dbconfig/20210614-122212-marostegui.json |
[production] |
| 12:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 25%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P16475 and previous config saved to /var/cache/conftool/dbconfig/20210614-122036-root.json |
[production] |
| 12:19 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps2005.codfw.wmnet |
[production] |
| 12:17 |
<XioNoX> |
configure OSPF link-protection on cr3-ulsfo:xe-0/1/1 - T167306 |
[production] |
| 12:13 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host maps2005.codfw.wmnet |
[production] |
| 12:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1148', diff saved to https://phabricator.wikimedia.org/P16474 and previous config saved to /var/cache/conftool/dbconfig/20210614-121101-marostegui.json |
[production] |
| 12:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1174 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16473 and previous config saved to /var/cache/conftool/dbconfig/20210614-121031-marostegui.json |
[production] |
| 12:10 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps2004.codfw.wmnet |
[production] |
| 12:03 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host maps2004.codfw.wmnet |
[production] |
| 12:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1148 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16472 and previous config saved to /var/cache/conftool/dbconfig/20210614-120112-marostegui.json |
[production] |
| 11:28 |
<effie> |
restart memcached on mc2019 |
[production] |
| 11:09 |
<effie> |
restart memcached on codfw memcached gutter pool (mc-gp2* hosts) |
[production] |
| 10:52 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps2003.codfw.wmnet |
[production] |
| 10:51 |
<topranks> |
T283163: Adding "metric-out minimum-igp" to all internal/Confed BGP groups on CR routers. |
[production] |
| 10:46 |
<effie> |
enable puppet on mc* |
[production] |
| 10:45 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host maps2003.codfw.wmnet |
[production] |
| 10:39 |
<effie> |
disable puppet on mc* hosts |
[production] |