|
2022-11-29
§
|
| 01:12 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2109 (T323907)', diff saved to https://phabricator.wikimedia.org/P41527 and previous config saved to /var/cache/conftool/dbconfig/20221129-011227-ladsgroup.json |
[production] |
| 01:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2179 (T321126)', diff saved to https://phabricator.wikimedia.org/P41526 and previous config saved to /var/cache/conftool/dbconfig/20221129-010332-marostegui.json |
[production] |
| 00:57 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P41525 and previous config saved to /var/cache/conftool/dbconfig/20221129-005755-ladsgroup.json |
[production] |
| 00:57 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2109', diff saved to https://phabricator.wikimedia.org/P41524 and previous config saved to /var/cache/conftool/dbconfig/20221129-005720-ladsgroup.json |
[production] |
| 00:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2179', diff saved to https://phabricator.wikimedia.org/P41522 and previous config saved to /var/cache/conftool/dbconfig/20221129-004825-marostegui.json |
[production] |
| 00:42 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P41521 and previous config saved to /var/cache/conftool/dbconfig/20221129-004249-ladsgroup.json |
[production] |
| 00:42 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2109', diff saved to https://phabricator.wikimedia.org/P41520 and previous config saved to /var/cache/conftool/dbconfig/20221129-004214-ladsgroup.json |
[production] |
| 00:38 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1101:3317 (T323907)', diff saved to https://phabricator.wikimedia.org/P41519 and previous config saved to /var/cache/conftool/dbconfig/20221129-003804-ladsgroup.json |
[production] |
| 00:37 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1101.eqiad.wmnet with reason: Maintenance |
[production] |
| 00:37 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1101.eqiad.wmnet with reason: Maintenance |
[production] |
| 00:37 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1098:3317 (T323907)', diff saved to https://phabricator.wikimedia.org/P41518 and previous config saved to /var/cache/conftool/dbconfig/20221129-003742-ladsgroup.json |
[production] |
| 00:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2179', diff saved to https://phabricator.wikimedia.org/P41517 and previous config saved to /var/cache/conftool/dbconfig/20221129-003319-marostegui.json |
[production] |
| 00:29 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host arclamp1001.eqiad.wmnet with OS bullseye |
[production] |
| 00:27 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157 (T322618)', diff saved to https://phabricator.wikimedia.org/P41516 and previous config saved to /var/cache/conftool/dbconfig/20221129-002742-ladsgroup.json |
[production] |
| 00:27 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2109 (T323907)', diff saved to https://phabricator.wikimedia.org/P41515 and previous config saved to /var/cache/conftool/dbconfig/20221129-002707-ladsgroup.json |
[production] |
| 00:22 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1098:3317', diff saved to https://phabricator.wikimedia.org/P41514 and previous config saved to /var/cache/conftool/dbconfig/20221129-002236-ladsgroup.json |
[production] |
| 00:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2179 (T321126)', diff saved to https://phabricator.wikimedia.org/P41513 and previous config saved to /var/cache/conftool/dbconfig/20221129-001812-marostegui.json |
[production] |
| 00:16 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on arclamp1001.eqiad.wmnet with reason: host reimage |
[production] |
| 00:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2179 (T321126)', diff saved to https://phabricator.wikimedia.org/P41512 and previous config saved to /var/cache/conftool/dbconfig/20221129-001559-marostegui.json |
[production] |
| 00:16 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2179.codfw.wmnet with reason: Maintenance |
[production] |
| 00:15 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2179.codfw.wmnet with reason: Maintenance |
[production] |
| 00:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2172 (T321126)', diff saved to https://phabricator.wikimedia.org/P41511 and previous config saved to /var/cache/conftool/dbconfig/20221129-001548-marostegui.json |
[production] |
| 00:12 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on arclamp1001.eqiad.wmnet with reason: host reimage |
[production] |
| 00:07 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1098:3317', diff saved to https://phabricator.wikimedia.org/P41510 and previous config saved to /var/cache/conftool/dbconfig/20221129-000729-ladsgroup.json |
[production] |
| 00:07 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host arclamp1001.eqiad.wmnet with OS bullseye |
[production] |
| 00:05 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1157 (T322618)', diff saved to https://phabricator.wikimedia.org/P41509 and previous config saved to /var/cache/conftool/dbconfig/20221129-000545-ladsgroup.json |
[production] |
| 00:05 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1157.eqiad.wmnet with reason: Maintenance |
[production] |
| 00:05 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1157.eqiad.wmnet with reason: Maintenance |
[production] |
| 00:03 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1145.eqiad.wmnet with reason: Maintenance |
[production] |
| 00:03 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1145.eqiad.wmnet with reason: Maintenance |
[production] |
| 00:03 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1112 (T322618)', diff saved to https://phabricator.wikimedia.org/P41508 and previous config saved to /var/cache/conftool/dbconfig/20221129-000341-ladsgroup.json |
[production] |
| 00:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2109 (T323907)', diff saved to https://phabricator.wikimedia.org/P41507 and previous config saved to /var/cache/conftool/dbconfig/20221129-000153-ladsgroup.json |
[production] |
| 00:01 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2109.codfw.wmnet with reason: Maintenance |
[production] |
| 00:01 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2109.codfw.wmnet with reason: Maintenance |
[production] |
| 00:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2105 (T323907)', diff saved to https://phabricator.wikimedia.org/P41506 and previous config saved to /var/cache/conftool/dbconfig/20221129-000143-ladsgroup.json |
[production] |
| 00:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P41505 and previous config saved to /var/cache/conftool/dbconfig/20221129-000042-marostegui.json |
[production] |
|
2022-11-28
§
|
| 23:58 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:58 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:58 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1197 (T323827)', diff saved to https://phabricator.wikimedia.org/P41504 and previous config saved to /var/cache/conftool/dbconfig/20221128-235817-ladsgroup.json |
[production] |
| 23:52 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1098:3317 (T323907)', diff saved to https://phabricator.wikimedia.org/P41503 and previous config saved to /var/cache/conftool/dbconfig/20221128-235223-ladsgroup.json |
[production] |
| 23:48 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P41502 and previous config saved to /var/cache/conftool/dbconfig/20221128-234834-ladsgroup.json |
[production] |
| 23:46 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2105', diff saved to https://phabricator.wikimedia.org/P41501 and previous config saved to /var/cache/conftool/dbconfig/20221128-234636-ladsgroup.json |
[production] |
| 23:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P41500 and previous config saved to /var/cache/conftool/dbconfig/20221128-234535-marostegui.json |
[production] |
| 23:43 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P41499 and previous config saved to /var/cache/conftool/dbconfig/20221128-234311-ladsgroup.json |
[production] |
| 23:33 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P41498 and previous config saved to /var/cache/conftool/dbconfig/20221128-233328-ladsgroup.json |
[production] |
| 23:33 |
<ebernhardson@deploy1002> |
Finished deploy [search/mjolnir/deploy@d361052]: msearch_daemon: Remove cluster selection/load monitor (duration: 00m 51s) |
[production] |
| 23:32 |
<ebernhardson@deploy1002> |
Started deploy [search/mjolnir/deploy@d361052]: msearch_daemon: Remove cluster selection/load monitor |
[production] |
| 23:31 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2105', diff saved to https://phabricator.wikimedia.org/P41497 and previous config saved to /var/cache/conftool/dbconfig/20221128-233130-ladsgroup.json |
[production] |
| 23:30 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2172 (T321126)', diff saved to https://phabricator.wikimedia.org/P41496 and previous config saved to /var/cache/conftool/dbconfig/20221128-233028-marostegui.json |
[production] |
| 23:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2172 (T321126)', diff saved to https://phabricator.wikimedia.org/P41495 and previous config saved to /var/cache/conftool/dbconfig/20221128-232815-marostegui.json |
[production] |