|
2022-11-25
ยง
|
| 13:50 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db1102.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:44 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2105', diff saved to https://phabricator.wikimedia.org/P41110 and previous config saved to /var/cache/conftool/dbconfig/20221125-134359-marostegui.json |
[production] |
| 13:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2105 (T321126)', diff saved to https://phabricator.wikimedia.org/P41109 and previous config saved to /var/cache/conftool/dbconfig/20221125-132853-marostegui.json |
[production] |
| 13:11 |
<gehel> |
re-enabling puppet on wcqs1001 - data transfer completed - T321605 |
[production] |
| 12:59 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2105 (T321126)', diff saved to https://phabricator.wikimedia.org/P41108 and previous config saved to /var/cache/conftool/dbconfig/20221125-125935-marostegui.json |
[production] |
| 12:59 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2105.codfw.wmnet with reason: Maintenance |
[production] |
| 12:59 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2105.codfw.wmnet with reason: Maintenance |
[production] |
| 12:51 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:50 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1198 (T321126)', diff saved to https://phabricator.wikimedia.org/P41107 and previous config saved to /var/cache/conftool/dbconfig/20221125-125046-marostegui.json |
[production] |
| 12:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1198', diff saved to https://phabricator.wikimedia.org/P41106 and previous config saved to /var/cache/conftool/dbconfig/20221125-123540-marostegui.json |
[production] |
| 12:26 |
<moritzm> |
installing vim security updates |
[production] |
| 12:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1198', diff saved to https://phabricator.wikimedia.org/P41105 and previous config saved to /var/cache/conftool/dbconfig/20221125-122033-marostegui.json |
[production] |
| 12:13 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.addnode (exit_code=0) for new host ganeti2031.codfw.wmnet to cluster codfw and group B |
[production] |
| 12:08 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.addnode for new host ganeti2031.codfw.wmnet to cluster codfw and group B |
[production] |
| 12:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1198 (T321126)', diff saved to https://phabricator.wikimedia.org/P41104 and previous config saved to /var/cache/conftool/dbconfig/20221125-120527-marostegui.json |
[production] |
| 11:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1198 (T321126)', diff saved to https://phabricator.wikimedia.org/P41103 and previous config saved to /var/cache/conftool/dbconfig/20221125-115222-marostegui.json |
[production] |
| 11:52 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1198.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:52 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1198.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1189 (T321126)', diff saved to https://phabricator.wikimedia.org/P41102 and previous config saved to /var/cache/conftool/dbconfig/20221125-115201-marostegui.json |
[production] |
| 11:38 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2031.codfw.wmnet |
[production] |
| 11:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P41101 and previous config saved to /var/cache/conftool/dbconfig/20221125-113654-marostegui.json |
[production] |
| 11:32 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2031.codfw.wmnet |
[production] |
| 11:24 |
<elukey> |
restart turnilo on an-tool1007 to pick up new settings for webrequest_sampled_live |
[production] |
| 11:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P41100 and previous config saved to /var/cache/conftool/dbconfig/20221125-112148-marostegui.json |
[production] |
| 11:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1189 (T321126)', diff saved to https://phabricator.wikimedia.org/P41099 and previous config saved to /var/cache/conftool/dbconfig/20221125-110642-marostegui.json |
[production] |
| 10:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1189 (T321126)', diff saved to https://phabricator.wikimedia.org/P41098 and previous config saved to /var/cache/conftool/dbconfig/20221125-105036-marostegui.json |
[production] |
| 10:50 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1189.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:50 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1189.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179 (T321126)', diff saved to https://phabricator.wikimedia.org/P41097 and previous config saved to /var/cache/conftool/dbconfig/20221125-105015-marostegui.json |
[production] |
| 10:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P41096 and previous config saved to /var/cache/conftool/dbconfig/20221125-103509-marostegui.json |
[production] |
| 10:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P41095 and previous config saved to /var/cache/conftool/dbconfig/20221125-102002-marostegui.json |
[production] |
| 10:04 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179 (T321126)', diff saved to https://phabricator.wikimedia.org/P41094 and previous config saved to /var/cache/conftool/dbconfig/20221125-100456-marostegui.json |
[production] |
| 09:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1179 (T321126)', diff saved to https://phabricator.wikimedia.org/P41093 and previous config saved to /var/cache/conftool/dbconfig/20221125-094643-marostegui.json |
[production] |
| 09:46 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1179.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:46 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1179.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175 (T321126)', diff saved to https://phabricator.wikimedia.org/P41092 and previous config saved to /var/cache/conftool/dbconfig/20221125-094622-marostegui.json |
[production] |
| 09:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P41091 and previous config saved to /var/cache/conftool/dbconfig/20221125-093115-marostegui.json |
[production] |
| 09:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P41090 and previous config saved to /var/cache/conftool/dbconfig/20221125-091609-marostegui.json |
[production] |
| 09:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175 (T321126)', diff saved to https://phabricator.wikimedia.org/P41089 and previous config saved to /var/cache/conftool/dbconfig/20221125-090102-marostegui.json |
[production] |
| 08:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1175 (T321126)', diff saved to https://phabricator.wikimedia.org/P41088 and previous config saved to /var/cache/conftool/dbconfig/20221125-085101-marostegui.json |
[production] |
| 08:50 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1175.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:50 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1175.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166 (T321126)', diff saved to https://phabricator.wikimedia.org/P41087 and previous config saved to /var/cache/conftool/dbconfig/20221125-085040-marostegui.json |
[production] |
| 08:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P41086 and previous config saved to /var/cache/conftool/dbconfig/20221125-083534-marostegui.json |
[production] |
| 08:35 |
<moritzm> |
installing libarchive security updates |
[production] |
| 08:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P41085 and previous config saved to /var/cache/conftool/dbconfig/20221125-082027-marostegui.json |
[production] |
| 08:09 |
<moritzm> |
rebalance Ganeti group C/codfw following reboots |
[production] |
| 08:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1166 (T321126)', diff saved to https://phabricator.wikimedia.org/P41084 and previous config saved to /var/cache/conftool/dbconfig/20221125-080521-marostegui.json |
[production] |
| 07:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1166 (T321126)', diff saved to https://phabricator.wikimedia.org/P41083 and previous config saved to /var/cache/conftool/dbconfig/20221125-075521-marostegui.json |
[production] |