|
2022-01-24
ยง
|
| 09:37 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.reimage for host cp2040.codfw.wmnet with OS buster |
[production] |
| 09:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1029 (re)pooling @ 100%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19013 and previous config saved to /var/cache/conftool/dbconfig/20220124-093608-root.json |
[production] |
| 09:30 |
<vgutierrez> |
depool cp2040 to be reimaged as cache::upload_envoy - T271421 |
[production] |
| 09:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1022 (re)pooling @ 60%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19012 and previous config saved to /var/cache/conftool/dbconfig/20220124-092558-root.json |
[production] |
| 09:23 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1111', diff saved to https://phabricator.wikimedia.org/P19011 and previous config saved to /var/cache/conftool/dbconfig/20220124-092307-marostegui.json |
[production] |
| 09:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1029 (re)pooling @ 75%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19010 and previous config saved to /var/cache/conftool/dbconfig/20220124-092105-root.json |
[production] |
| 09:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1022 (re)pooling @ 50%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19009 and previous config saved to /var/cache/conftool/dbconfig/20220124-091054-root.json |
[production] |
| 09:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1111 (T285149)', diff saved to https://phabricator.wikimedia.org/P19008 and previous config saved to /var/cache/conftool/dbconfig/20220124-090803-marostegui.json |
[production] |
| 09:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1111 (T285149)', diff saved to https://phabricator.wikimedia.org/P19007 and previous config saved to /var/cache/conftool/dbconfig/20220124-090657-marostegui.json |
[production] |
| 09:06 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1111.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:06 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1111.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1099:3318 (T285149)', diff saved to https://phabricator.wikimedia.org/P19006 and previous config saved to /var/cache/conftool/dbconfig/20220124-090649-marostegui.json |
[production] |
| 09:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1029 (re)pooling @ 60%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19005 and previous config saved to /var/cache/conftool/dbconfig/20220124-090601-root.json |
[production] |
| 08:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1022 (re)pooling @ 40%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19004 and previous config saved to /var/cache/conftool/dbconfig/20220124-085551-root.json |
[production] |
| 08:51 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.addnode (exit_code=0) for new host ganeti1026.eqiad.wmnet to ganeti01.svc.eqiad.wmnet |
[production] |
| 08:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1099:3318', diff saved to https://phabricator.wikimedia.org/P19003 and previous config saved to /var/cache/conftool/dbconfig/20220124-085144-marostegui.json |
[production] |
| 08:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1029 (re)pooling @ 50%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19002 and previous config saved to /var/cache/conftool/dbconfig/20220124-085057-root.json |
[production] |
| 08:50 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.addnode for new host ganeti1026.eqiad.wmnet to ganeti01.svc.eqiad.wmnet |
[production] |
| 08:45 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1026.eqiad.wmnet |
[production] |
| 08:41 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti1026.eqiad.wmnet |
[production] |
| 08:40 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1022 (re)pooling @ 25%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19001 and previous config saved to /var/cache/conftool/dbconfig/20220124-084047-root.json |
[production] |
| 08:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1099:3318', diff saved to https://phabricator.wikimedia.org/P19000 and previous config saved to /var/cache/conftool/dbconfig/20220124-083640-marostegui.json |
[production] |
| 08:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1029 (re)pooling @ 40%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P18999 and previous config saved to /var/cache/conftool/dbconfig/20220124-083554-root.json |
[production] |
| 08:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1022 (re)pooling @ 20%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P18998 and previous config saved to /var/cache/conftool/dbconfig/20220124-082543-root.json |
[production] |
| 08:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1099:3318 (T285149)', diff saved to https://phabricator.wikimedia.org/P18997 and previous config saved to /var/cache/conftool/dbconfig/20220124-082135-marostegui.json |
[production] |
| 08:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1029 (re)pooling @ 25%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P18996 and previous config saved to /var/cache/conftool/dbconfig/20220124-082050-root.json |
[production] |
| 08:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1099:3318 (T285149)', diff saved to https://phabricator.wikimedia.org/P18995 and previous config saved to /var/cache/conftool/dbconfig/20220124-082029-marostegui.json |
[production] |
| 08:20 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1099.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:20 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1099.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1178 (T285149)', diff saved to https://phabricator.wikimedia.org/P18994 and previous config saved to /var/cache/conftool/dbconfig/20220124-082022-marostegui.json |
[production] |
| 08:15 |
<moritzm> |
draining instances off ganeti1014 for reimage |
[production] |
| 08:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1022 (re)pooling @ 10%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P18993 and previous config saved to /var/cache/conftool/dbconfig/20220124-081040-root.json |
[production] |
| 08:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1029 (re)pooling @ 20%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P18992 and previous config saved to /var/cache/conftool/dbconfig/20220124-080546-root.json |
[production] |
| 08:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P18991 and previous config saved to /var/cache/conftool/dbconfig/20220124-080517-marostegui.json |
[production] |
| 07:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1022 (re)pooling @ 5%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P18990 and previous config saved to /var/cache/conftool/dbconfig/20220124-075536-root.json |
[production] |
| 07:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1029 (re)pooling @ 10%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P18989 and previous config saved to /var/cache/conftool/dbconfig/20220124-075043-root.json |
[production] |
| 07:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P18988 and previous config saved to /var/cache/conftool/dbconfig/20220124-075012-marostegui.json |
[production] |
| 07:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1029 (re)pooling @ 5%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P18987 and previous config saved to /var/cache/conftool/dbconfig/20220124-073539-root.json |
[production] |
| 07:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1178 (T285149)', diff saved to https://phabricator.wikimedia.org/P18986 and previous config saved to /var/cache/conftool/dbconfig/20220124-073507-marostegui.json |
[production] |
| 07:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1029 (re)pooling @ 1%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P18985 and previous config saved to /var/cache/conftool/dbconfig/20220124-072035-root.json |
[production] |
| 06:44 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es1022.eqiad.wmnet with OS bullseye |
[production] |
| 06:34 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1178 (T285149)', diff saved to https://phabricator.wikimedia.org/P18984 and previous config saved to /var/cache/conftool/dbconfig/20220124-063448-marostegui.json |
[production] |
| 06:34 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1178.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:34 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1178.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:34 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1104 (T285149)', diff saved to https://phabricator.wikimedia.org/P18983 and previous config saved to /var/cache/conftool/dbconfig/20220124-063440-marostegui.json |
[production] |
| 06:23 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es1029.eqiad.wmnet with OS bullseye |
[production] |
| 06:19 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1104', diff saved to https://phabricator.wikimedia.org/P18982 and previous config saved to /var/cache/conftool/dbconfig/20220124-061936-marostegui.json |
[production] |
| 06:05 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.reimage for host es1022.eqiad.wmnet with OS bullseye |
[production] |
| 06:04 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1104', diff saved to https://phabricator.wikimedia.org/P18981 and previous config saved to /var/cache/conftool/dbconfig/20220124-060431-marostegui.json |
[production] |
| 06:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es1022 T299123', diff saved to https://phabricator.wikimedia.org/P18980 and previous config saved to /var/cache/conftool/dbconfig/20220124-060248-marostegui.json |
[production] |