|
2021-06-14
ยง
|
| 12:37 |
<XioNoX> |
configure OSPF link-protection on cr3/4-ulsfo - T167306 |
[production] |
| 12:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 50%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P16481 and previous config saved to /var/cache/conftool/dbconfig/20210614-123539-root.json |
[production] |
| 12:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es1033 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16480 and previous config saved to /var/cache/conftool/dbconfig/20210614-123512-marostegui.json |
[production] |
| 12:34 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1034 (re)pooling @ 10%: Repool es1034 after upgrade', diff saved to https://phabricator.wikimedia.org/P16479 and previous config saved to /var/cache/conftool/dbconfig/20210614-123427-root.json |
[production] |
| 12:23 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Restore es1028 original weight', diff saved to https://phabricator.wikimedia.org/P16478 and previous config saved to /var/cache/conftool/dbconfig/20210614-122322-marostegui.json |
[production] |
| 12:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Give some weight to es1028 while es1034 gets upgraded', diff saved to https://phabricator.wikimedia.org/P16477 and previous config saved to /var/cache/conftool/dbconfig/20210614-122242-marostegui.json |
[production] |
| 12:22 |
<dcausse> |
re-pooling wdqs1012 |
[production] |
| 12:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es1034 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16476 and previous config saved to /var/cache/conftool/dbconfig/20210614-122212-marostegui.json |
[production] |
| 12:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1174 (re)pooling @ 25%: Repool db1174 after schema change', diff saved to https://phabricator.wikimedia.org/P16475 and previous config saved to /var/cache/conftool/dbconfig/20210614-122036-root.json |
[production] |
| 12:19 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps2005.codfw.wmnet |
[production] |
| 12:17 |
<XioNoX> |
configure OSPF link-protection on cr3-ulsfo:xe-0/1/1 - T167306 |
[production] |
| 12:13 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host maps2005.codfw.wmnet |
[production] |
| 12:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1148', diff saved to https://phabricator.wikimedia.org/P16474 and previous config saved to /var/cache/conftool/dbconfig/20210614-121101-marostegui.json |
[production] |
| 12:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1174 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16473 and previous config saved to /var/cache/conftool/dbconfig/20210614-121031-marostegui.json |
[production] |
| 12:10 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps2004.codfw.wmnet |
[production] |
| 12:03 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host maps2004.codfw.wmnet |
[production] |
| 12:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1148 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P16472 and previous config saved to /var/cache/conftool/dbconfig/20210614-120112-marostegui.json |
[production] |
| 11:28 |
<effie> |
restart memcached on mc2019 |
[production] |
| 11:09 |
<effie> |
restart memcached on codfw memcached gutter pool (mc-gp2* hosts) |
[production] |
| 10:52 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps2003.codfw.wmnet |
[production] |
| 10:51 |
<topranks> |
T283163: Adding "metric-out minimum-igp" to all internal/Confed BGP groups on CR routers. |
[production] |
| 10:46 |
<effie> |
enable puppet on mc* |
[production] |
| 10:45 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host maps2003.codfw.wmnet |
[production] |
| 10:39 |
<effie> |
disable puppet on mc* hosts |
[production] |
| 10:33 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps2001.codfw.wmnet |
[production] |
| 10:27 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host maps2001.codfw.wmnet |
[production] |
| 10:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1131 (re)pooling @ 100%: Repool db1131 after schema change', diff saved to https://phabricator.wikimedia.org/P16471 and previous config saved to /var/cache/conftool/dbconfig/20210614-101839-root.json |
[production] |
| 10:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1131 (re)pooling @ 75%: Repool db1131 after schema change', diff saved to https://phabricator.wikimedia.org/P16469 and previous config saved to /var/cache/conftool/dbconfig/20210614-100336-root.json |
[production] |
| 09:56 |
<jbond@deploy1002> |
Finished deploy [netbox/deploy@e9f2382]: deploy v2.10.4-wmf4 (duration: 02m 37s) |
[production] |
| 09:54 |
<jbond@deploy1002> |
Started deploy [netbox/deploy@e9f2382]: deploy v2.10.4-wmf4 |
[production] |
| 09:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1131 (re)pooling @ 50%: Repool db1131 after schema change', diff saved to https://phabricator.wikimedia.org/P16467 and previous config saved to /var/cache/conftool/dbconfig/20210614-094832-root.json |
[production] |
| 09:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1131 (re)pooling @ 25%: Repool db1131 after schema change', diff saved to https://phabricator.wikimedia.org/P16466 and previous config saved to /var/cache/conftool/dbconfig/20210614-093329-root.json |
[production] |
| 09:22 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=maps1008.eqiad.wmnet |
[production] |
| 09:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1131 for schema change', diff saved to https://phabricator.wikimedia.org/P16465 and previous config saved to /var/cache/conftool/dbconfig/20210614-092234-marostegui.json |
[production] |
| 09:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1165 (re)pooling @ 100%: Repool db1165 after schema change', diff saved to https://phabricator.wikimedia.org/P16464 and previous config saved to /var/cache/conftool/dbconfig/20210614-092125-root.json |
[production] |
| 09:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1165 (re)pooling @ 75%: Repool db1165 after schema change', diff saved to https://phabricator.wikimedia.org/P16463 and previous config saved to /var/cache/conftool/dbconfig/20210614-090622-root.json |
[production] |
| 08:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1165 (re)pooling @ 50%: Repool db1165 after schema change', diff saved to https://phabricator.wikimedia.org/P16462 and previous config saved to /var/cache/conftool/dbconfig/20210614-085118-root.json |
[production] |
| 08:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1165 (re)pooling @ 25%: Repool db1165 after schema change', diff saved to https://phabricator.wikimedia.org/P16461 and previous config saved to /var/cache/conftool/dbconfig/20210614-083614-root.json |
[production] |
| 08:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1165 for schema change', diff saved to https://phabricator.wikimedia.org/P16460 and previous config saved to /var/cache/conftool/dbconfig/20210614-081239-marostegui.json |
[production] |
| 08:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 100%: Repool db1168 after schema change', diff saved to https://phabricator.wikimedia.org/P16459 and previous config saved to /var/cache/conftool/dbconfig/20210614-081031-root.json |
[production] |
| 08:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db2148', diff saved to https://phabricator.wikimedia.org/P16458 and previous config saved to /var/cache/conftool/dbconfig/20210614-080552-marostegui.json |
[production] |
| 07:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 75%: Repool db1168 after schema change', diff saved to https://phabricator.wikimedia.org/P16456 and previous config saved to /var/cache/conftool/dbconfig/20210614-075528-root.json |
[production] |
| 07:51 |
<marostegui> |
Depool clouddb1013 to upgrade mysql |
[production] |
| 07:40 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 50%: Repool db1168 after schema change', diff saved to https://phabricator.wikimedia.org/P16455 and previous config saved to /var/cache/conftool/dbconfig/20210614-074024-root.json |
[production] |
| 07:30 |
<marostegui> |
Reboot db2148 T284852 |
[production] |
| 07:29 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db2148 T284852', diff saved to https://phabricator.wikimedia.org/P16454 and previous config saved to /var/cache/conftool/dbconfig/20210614-072930-marostegui.json |
[production] |
| 07:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 25%: Repool db1168 after schema change', diff saved to https://phabricator.wikimedia.org/P16453 and previous config saved to /var/cache/conftool/dbconfig/20210614-072520-root.json |
[production] |
| 07:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1168 for schema change', diff saved to https://phabricator.wikimedia.org/P16452 and previous config saved to /var/cache/conftool/dbconfig/20210614-071839-marostegui.json |
[production] |
| 07:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1180 (re)pooling @ 100%: Repool db1180 after schema change', diff saved to https://phabricator.wikimedia.org/P16451 and previous config saved to /var/cache/conftool/dbconfig/20210614-071742-root.json |
[production] |
| 07:15 |
<dcausse> |
restart blazegraph and depool wdqs1012 |
[production] |