|
2023-08-22
ยง
|
| 08:12 |
<moritzm> |
bounce ferm on aux-k8s-ctrl1001 |
[production] |
| 08:10 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'db1163 (re)pooling @ 25%: 10', diff saved to https://phabricator.wikimedia.org/P50825 and previous config saved to /var/cache/conftool/dbconfig/20230822-081054-ladsgroup.json |
[production] |
| 08:08 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1127', diff saved to https://phabricator.wikimedia.org/P50824 and previous config saved to /var/cache/conftool/dbconfig/20230822-080836-ladsgroup.json |
[production] |
| 08:05 |
<moritzm> |
installing Linux 4.19.289-2 on Buster hosts |
[production] |
| 08:05 |
<stevemunene@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1117.eqiad.wmnet with reason: host reimage |
[production] |
| 08:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling es2021 (T344589)', diff saved to https://phabricator.wikimedia.org/P50823 and previous config saved to /var/cache/conftool/dbconfig/20230822-080413-ladsgroup.json |
[production] |
| 08:04 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on es2021.codfw.wmnet with reason: Maintenance |
[production] |
| 08:03 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on es2021.codfw.wmnet with reason: Maintenance |
[production] |
| 08:03 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling es1021 (T344589)', diff saved to https://phabricator.wikimedia.org/P50822 and previous config saved to /var/cache/conftool/dbconfig/20230822-080328-ladsgroup.json |
[production] |
| 08:03 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on es1021.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:03 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on es1021.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:02 |
<stevemunene@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1117.eqiad.wmnet with reason: host reimage |
[production] |
| 08:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1210 (T343718)', diff saved to https://phabricator.wikimedia.org/P50821 and previous config saved to /var/cache/conftool/dbconfig/20230822-080155-ladsgroup.json |
[production] |
| 08:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2108 (T344589)', diff saved to https://phabricator.wikimedia.org/P50820 and previous config saved to /var/cache/conftool/dbconfig/20230822-080119-ladsgroup.json |
[production] |
| 07:57 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2171:3315 (T343718)', diff saved to https://phabricator.wikimedia.org/P50819 and previous config saved to /var/cache/conftool/dbconfig/20230822-075728-ladsgroup.json |
[production] |
| 07:57 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2171.codfw.wmnet with reason: Maintenance |
[production] |
| 07:57 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2171.codfw.wmnet with reason: Maintenance |
[production] |
| 07:57 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2157 (T343718)', diff saved to https://phabricator.wikimedia.org/P50818 and previous config saved to /var/cache/conftool/dbconfig/20230822-075717-ladsgroup.json |
[production] |
| 07:54 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2108 (T344589)', diff saved to https://phabricator.wikimedia.org/P50817 and previous config saved to /var/cache/conftool/dbconfig/20230822-075453-ladsgroup.json |
[production] |
| 07:54 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2108.codfw.wmnet with reason: Maintenance |
[production] |
| 07:54 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2108.codfw.wmnet with reason: Maintenance |
[production] |
| 07:53 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1127 (T344589)', diff saved to https://phabricator.wikimedia.org/P50816 and previous config saved to /var/cache/conftool/dbconfig/20230822-075329-ladsgroup.json |
[production] |
| 07:51 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1163.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:51 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1163.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:50 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2100.codfw.wmnet with reason: Maintenance |
[production] |
| 07:50 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2100.codfw.wmnet with reason: Maintenance |
[production] |
| 07:48 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host centrallog1002.eqiad.wmnet |
[production] |
| 07:48 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2103.codfw.wmnet with reason: Maintenance |
[production] |
| 07:48 |
<stevemunene@cumin1001> |
START - Cookbook sre.hosts.reimage for host an-worker1117.eqiad.wmnet with OS bullseye |
[production] |
| 07:48 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2103.codfw.wmnet with reason: Maintenance |
[production] |
| 07:48 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mwlog1002.eqiad.wmnet |
[production] |
| 07:47 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1127 (T344589)', diff saved to https://phabricator.wikimedia.org/P50815 and previous config saved to /var/cache/conftool/dbconfig/20230822-074725-ladsgroup.json |
[production] |
| 07:47 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1127.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:47 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1127.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:46 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2098.codfw.wmnet with reason: Maintenance |
[production] |
| 07:46 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2098.codfw.wmnet with reason: Maintenance |
[production] |
| 07:43 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1210 (T343718)', diff saved to https://phabricator.wikimedia.org/P50814 and previous config saved to /var/cache/conftool/dbconfig/20230822-074358-ladsgroup.json |
[production] |
| 07:43 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1210.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:43 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1210.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:43 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1200 (T343718)', diff saved to https://phabricator.wikimedia.org/P50813 and previous config saved to /var/cache/conftool/dbconfig/20230822-074338-ladsgroup.json |
[production] |
| 07:42 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2157', diff saved to https://phabricator.wikimedia.org/P50812 and previous config saved to /var/cache/conftool/dbconfig/20230822-074210-ladsgroup.json |
[production] |
| 07:42 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host mwlog1002.eqiad.wmnet |
[production] |
| 07:41 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host centrallog1002.eqiad.wmnet |
[production] |
| 07:41 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:41 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:41 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1223 (T344589)', diff saved to https://phabricator.wikimedia.org/P50811 and previous config saved to /var/cache/conftool/dbconfig/20230822-074106-ladsgroup.json |
[production] |
| 07:37 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mwlog2002.codfw.wmnet |
[production] |
| 07:37 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host centrallog2002.codfw.wmnet |
[production] |
| 07:33 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host graphite1005.eqiad.wmnet |
[production] |
| 07:30 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host mwlog2002.codfw.wmnet |
[production] |