|
2022-11-25
ยง
|
| 08:50 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1175.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166 (T321126)', diff saved to https://phabricator.wikimedia.org/P41087 and previous config saved to /var/cache/conftool/dbconfig/20221125-085040-marostegui.json |
[production] |
| 08:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P41086 and previous config saved to /var/cache/conftool/dbconfig/20221125-083534-marostegui.json |
[production] |
| 08:35 |
<moritzm> |
installing libarchive security updates |
[production] |
| 08:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P41085 and previous config saved to /var/cache/conftool/dbconfig/20221125-082027-marostegui.json |
[production] |
| 08:09 |
<moritzm> |
rebalance Ganeti group C/codfw following reboots |
[production] |
| 08:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166 (T321126)', diff saved to https://phabricator.wikimedia.org/P41084 and previous config saved to /var/cache/conftool/dbconfig/20221125-080521-marostegui.json |
[production] |
| 07:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1166 (T321126)', diff saved to https://phabricator.wikimedia.org/P41083 and previous config saved to /var/cache/conftool/dbconfig/20221125-075521-marostegui.json |
[production] |
| 07:55 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1166.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:55 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1166.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157 (T321126)', diff saved to https://phabricator.wikimedia.org/P41082 and previous config saved to /var/cache/conftool/dbconfig/20221125-075500-marostegui.json |
[production] |
| 07:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P41081 and previous config saved to /var/cache/conftool/dbconfig/20221125-073953-marostegui.json |
[production] |
| 07:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P41080 and previous config saved to /var/cache/conftool/dbconfig/20221125-072447-marostegui.json |
[production] |
| 07:09 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157 (T321126)', diff saved to https://phabricator.wikimedia.org/P41079 and previous config saved to /var/cache/conftool/dbconfig/20221125-070940-marostegui.json |
[production] |
| 06:59 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1157 (T321126)', diff saved to https://phabricator.wikimedia.org/P41078 and previous config saved to /var/cache/conftool/dbconfig/20221125-065930-marostegui.json |
[production] |
| 06:59 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1157.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:59 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1157.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:51 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1145.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:50 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1145.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1112 (T321126)', diff saved to https://phabricator.wikimedia.org/P41077 and previous config saved to /var/cache/conftool/dbconfig/20221125-065049-marostegui.json |
[production] |
| 06:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P41076 and previous config saved to /var/cache/conftool/dbconfig/20221125-063543-marostegui.json |
[production] |
| 06:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P41075 and previous config saved to /var/cache/conftool/dbconfig/20221125-062036-marostegui.json |
[production] |
| 06:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1112 (T321126)', diff saved to https://phabricator.wikimedia.org/P41074 and previous config saved to /var/cache/conftool/dbconfig/20221125-060530-marostegui.json |
[production] |
| 05:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1112 (T321126)', diff saved to https://phabricator.wikimedia.org/P41073 and previous config saved to /var/cache/conftool/dbconfig/20221125-055517-marostegui.json |
[production] |
| 05:55 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:54 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:54 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1112.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:54 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1112.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:46 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1102.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:45 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1102.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:44 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1109.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:44 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1109.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:34 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2165.codfw.wmnet with reason: Maintenance |
[production] |
| 05:34 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2165.codfw.wmnet with reason: Maintenance |
[production] |
| 01:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2181 (T321126)', diff saved to https://phabricator.wikimedia.org/P41072 and previous config saved to /var/cache/conftool/dbconfig/20221125-013324-marostegui.json |
[production] |
| 01:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P41071 and previous config saved to /var/cache/conftool/dbconfig/20221125-011818-marostegui.json |
[production] |
| 01:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P41070 and previous config saved to /var/cache/conftool/dbconfig/20221125-010311-marostegui.json |
[production] |
| 00:51 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1194 (T322618)', diff saved to https://phabricator.wikimedia.org/P41069 and previous config saved to /var/cache/conftool/dbconfig/20221125-005150-ladsgroup.json |
[production] |
| 00:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2181 (T321126)', diff saved to https://phabricator.wikimedia.org/P41068 and previous config saved to /var/cache/conftool/dbconfig/20221125-004805-marostegui.json |
[production] |
| 00:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2181 (T321126)', diff saved to https://phabricator.wikimedia.org/P41067 and previous config saved to /var/cache/conftool/dbconfig/20221125-004554-marostegui.json |
[production] |
| 00:45 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2181.codfw.wmnet with reason: Maintenance |
[production] |
| 00:45 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2181.codfw.wmnet with reason: Maintenance |
[production] |
| 00:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2168:3318 (T321126)', diff saved to https://phabricator.wikimedia.org/P41066 and previous config saved to /var/cache/conftool/dbconfig/20221125-004533-marostegui.json |
[production] |
| 00:36 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1194', diff saved to https://phabricator.wikimedia.org/P41065 and previous config saved to /var/cache/conftool/dbconfig/20221125-003643-ladsgroup.json |
[production] |
| 00:30 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2168:3318', diff saved to https://phabricator.wikimedia.org/P41064 and previous config saved to /var/cache/conftool/dbconfig/20221125-003026-marostegui.json |
[production] |
| 00:21 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1194', diff saved to https://phabricator.wikimedia.org/P41063 and previous config saved to /var/cache/conftool/dbconfig/20221125-002137-ladsgroup.json |
[production] |
| 00:21 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'db1181 (re)pooling @ 100%: Maint done', diff saved to https://phabricator.wikimedia.org/P41062 and previous config saved to /var/cache/conftool/dbconfig/20221125-002119-ladsgroup.json |
[production] |
| 00:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2168:3318', diff saved to https://phabricator.wikimedia.org/P41061 and previous config saved to /var/cache/conftool/dbconfig/20221125-001520-marostegui.json |
[production] |
| 00:06 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1194 (T322618)', diff saved to https://phabricator.wikimedia.org/P41060 and previous config saved to /var/cache/conftool/dbconfig/20221125-000630-ladsgroup.json |
[production] |
| 00:06 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'db1181 (re)pooling @ 75%: Maint done', diff saved to https://phabricator.wikimedia.org/P41059 and previous config saved to /var/cache/conftool/dbconfig/20221125-000614-ladsgroup.json |
[production] |