|
2022-11-28
ยง
|
| 18:35 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db2175.codfw.wmnet with reason: Maintenance |
[production] |
| 18:35 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3312 (T323827)', diff saved to https://phabricator.wikimedia.org/P41430 and previous config saved to /var/cache/conftool/dbconfig/20221128-183511-ladsgroup.json |
[production] |
| 18:30 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P41429 and previous config saved to /var/cache/conftool/dbconfig/20221128-183048-ladsgroup.json |
[production] |
| 18:30 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P41428 and previous config saved to /var/cache/conftool/dbconfig/20221128-183028-ladsgroup.json |
[production] |
| 18:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P41427 and previous config saved to /var/cache/conftool/dbconfig/20221128-182511-ladsgroup.json |
[production] |
| 18:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2110', diff saved to https://phabricator.wikimedia.org/P41426 and previous config saved to /var/cache/conftool/dbconfig/20221128-182458-marostegui.json |
[production] |
| 18:20 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3312', diff saved to https://phabricator.wikimedia.org/P41425 and previous config saved to /var/cache/conftool/dbconfig/20221128-182004-ladsgroup.json |
[production] |
| 18:15 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179 (T323907)', diff saved to https://phabricator.wikimedia.org/P41424 and previous config saved to /var/cache/conftool/dbconfig/20221128-181541-ladsgroup.json |
[production] |
| 18:15 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P41423 and previous config saved to /var/cache/conftool/dbconfig/20221128-181522-ladsgroup.json |
[production] |
| 18:10 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P41421 and previous config saved to /var/cache/conftool/dbconfig/20221128-181004-ladsgroup.json |
[production] |
| 18:09 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2110', diff saved to https://phabricator.wikimedia.org/P41420 and previous config saved to /var/cache/conftool/dbconfig/20221128-180951-marostegui.json |
[production] |
| 18:05 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3312', diff saved to https://phabricator.wikimedia.org/P41419 and previous config saved to /var/cache/conftool/dbconfig/20221128-180458-ladsgroup.json |
[production] |
| 18:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1179 (T323907)', diff saved to https://phabricator.wikimedia.org/P41418 and previous config saved to /var/cache/conftool/dbconfig/20221128-180452-ladsgroup.json |
[production] |
| 18:04 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1179.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:04 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1179.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175 (T323907)', diff saved to https://phabricator.wikimedia.org/P41417 and previous config saved to /var/cache/conftool/dbconfig/20221128-180431-ladsgroup.json |
[production] |
| 18:00 |
<jbond@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be2050.codfw.wmnet with OS bullseye |
[production] |
| 18:00 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1156 (T323827)', diff saved to https://phabricator.wikimedia.org/P41415 and previous config saved to /var/cache/conftool/dbconfig/20221128-180015-ladsgroup.json |
[production] |
| 17:54 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2177 (T323827)', diff saved to https://phabricator.wikimedia.org/P41414 and previous config saved to /var/cache/conftool/dbconfig/20221128-175458-ladsgroup.json |
[production] |
| 17:54 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2110 (T321126)', diff saved to https://phabricator.wikimedia.org/P41413 and previous config saved to /var/cache/conftool/dbconfig/20221128-175445-marostegui.json |
[production] |
| 17:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2110 (T321126)', diff saved to https://phabricator.wikimedia.org/P41412 and previous config saved to /var/cache/conftool/dbconfig/20221128-175232-marostegui.json |
[production] |
| 17:52 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2110.codfw.wmnet with reason: Maintenance |
[production] |
| 17:52 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2110.codfw.wmnet with reason: Maintenance |
[production] |
| 17:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2106 (T321126)', diff saved to https://phabricator.wikimedia.org/P41411 and previous config saved to /var/cache/conftool/dbconfig/20221128-175210-marostegui.json |
[production] |
| 17:49 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3312 (T323827)', diff saved to https://phabricator.wikimedia.org/P41410 and previous config saved to /var/cache/conftool/dbconfig/20221128-174951-ladsgroup.json |
[production] |
| 17:49 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P41409 and previous config saved to /var/cache/conftool/dbconfig/20221128-174925-ladsgroup.json |
[production] |
| 17:43 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1156 (T323827)', diff saved to https://phabricator.wikimedia.org/P41408 and previous config saved to /var/cache/conftool/dbconfig/20221128-174324-ladsgroup.json |
[production] |
| 17:43 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 17:43 |
<jbond@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be2050.codfw.wmnet with reason: host reimage |
[production] |
| 17:43 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 20:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 17:43 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1156.eqiad.wmnet with reason: Maintenance |
[production] |
| 17:42 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db1156.eqiad.wmnet with reason: Maintenance |
[production] |
| 17:42 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1146:3312 (T323827)', diff saved to https://phabricator.wikimedia.org/P41407 and previous config saved to /var/cache/conftool/dbconfig/20221128-174213-ladsgroup.json |
[production] |
| 17:39 |
<jbond@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be2050.codfw.wmnet with reason: host reimage |
[production] |
| 17:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2106', diff saved to https://phabricator.wikimedia.org/P41406 and previous config saved to /var/cache/conftool/dbconfig/20221128-173704-marostegui.json |
[production] |
| 17:35 |
<jnuche@deploy1002> |
Installation of scap version "4.29.2" completed for 558 hosts |
[production] |
| 17:35 |
<jnuche@deploy1002> |
Installing scap version "4.29.2" for 558 hosts |
[production] |
| 17:34 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P41405 and previous config saved to /var/cache/conftool/dbconfig/20221128-173418-ladsgroup.json |
[production] |
| 17:32 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2170:3312 (T323827)', diff saved to https://phabricator.wikimedia.org/P41404 and previous config saved to /var/cache/conftool/dbconfig/20221128-173227-ladsgroup.json |
[production] |
| 17:32 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2170.codfw.wmnet with reason: Maintenance |
[production] |
| 17:32 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db2170.codfw.wmnet with reason: Maintenance |
[production] |
| 17:32 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2148 (T323827)', diff saved to https://phabricator.wikimedia.org/P41403 and previous config saved to /var/cache/conftool/dbconfig/20221128-173206-ladsgroup.json |
[production] |
| 17:27 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1146:3312', diff saved to https://phabricator.wikimedia.org/P41402 and previous config saved to /var/cache/conftool/dbconfig/20221128-172707-ladsgroup.json |
[production] |
| 17:24 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2177 (T323827)', diff saved to https://phabricator.wikimedia.org/P41401 and previous config saved to /var/cache/conftool/dbconfig/20221128-172442-ladsgroup.json |
[production] |
| 17:24 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2177.codfw.wmnet with reason: Maintenance |
[production] |
| 17:24 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db2177.codfw.wmnet with reason: Maintenance |
[production] |
| 17:24 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2156 (T323827)', diff saved to https://phabricator.wikimedia.org/P41400 and previous config saved to /var/cache/conftool/dbconfig/20221128-172419-ladsgroup.json |
[production] |
| 17:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2106', diff saved to https://phabricator.wikimedia.org/P41399 and previous config saved to /var/cache/conftool/dbconfig/20221128-172157-marostegui.json |
[production] |
| 17:21 |
<jbond@cumin2002> |
START - Cookbook sre.hosts.reimage for host ms-be2050.codfw.wmnet with OS bullseye |
[production] |
| 17:20 |
<jbond@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ms-be2050.codfw.wmnet with OS bullseye |
[production] |