|
2023-08-29
§
|
| 02:53 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on moss-be2003.codfw.wmnet with reason: host reimage |
[production] |
| 02:50 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on moss-be2003.codfw.wmnet with reason: host reimage |
[production] |
| 02:01 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.reimage for host moss-be2003.codfw.wmnet with OS bullseye |
[production] |
| 01:45 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1145.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:44 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1145.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:44 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3314 (T343718)', diff saved to https://phabricator.wikimedia.org/P51797 and previous config saved to /var/cache/conftool/dbconfig/20230829-014452-ladsgroup.json |
[production] |
| 01:29 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3314', diff saved to https://phabricator.wikimedia.org/P51796 and previous config saved to /var/cache/conftool/dbconfig/20230829-012946-ladsgroup.json |
[production] |
| 01:14 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3314', diff saved to https://phabricator.wikimedia.org/P51795 and previous config saved to /var/cache/conftool/dbconfig/20230829-011440-ladsgroup.json |
[production] |
| 00:59 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3314 (T343718)', diff saved to https://phabricator.wikimedia.org/P51794 and previous config saved to /var/cache/conftool/dbconfig/20230829-005933-ladsgroup.json |
[production] |
| 00:49 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance |
[production] |
| 00:49 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance |
[production] |
| 00:49 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1219 (T343718)', diff saved to https://phabricator.wikimedia.org/P51793 and previous config saved to /var/cache/conftool/dbconfig/20230829-004925-ladsgroup.json |
[production] |
| 00:42 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2136 (T343718)', diff saved to https://phabricator.wikimedia.org/P51792 and previous config saved to /var/cache/conftool/dbconfig/20230829-004217-ladsgroup.json |
[production] |
| 00:42 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2136.codfw.wmnet with reason: Maintenance |
[production] |
| 00:42 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2136.codfw.wmnet with reason: Maintenance |
[production] |
| 00:42 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2119 (T343718)', diff saved to https://phabricator.wikimedia.org/P51791 and previous config saved to /var/cache/conftool/dbconfig/20230829-004207-ladsgroup.json |
[production] |
| 00:34 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1219', diff saved to https://phabricator.wikimedia.org/P51790 and previous config saved to /var/cache/conftool/dbconfig/20230829-003418-ladsgroup.json |
[production] |
| 00:27 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2119', diff saved to https://phabricator.wikimedia.org/P51789 and previous config saved to /var/cache/conftool/dbconfig/20230829-002700-ladsgroup.json |
[production] |
| 00:22 |
<eileen> |
civicrm upgraded from 6a2cdf10 to d13e6e0c |
[production] |
| 00:19 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1219', diff saved to https://phabricator.wikimedia.org/P51788 and previous config saved to /var/cache/conftool/dbconfig/20230829-001912-ladsgroup.json |
[production] |
| 00:17 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) |
[production] |
| 00:11 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2119', diff saved to https://phabricator.wikimedia.org/P51787 and previous config saved to /var/cache/conftool/dbconfig/20230829-001154-ladsgroup.json |
[production] |
| 00:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1219 (T343718)', diff saved to https://phabricator.wikimedia.org/P51786 and previous config saved to /var/cache/conftool/dbconfig/20230829-000406-ladsgroup.json |
[production] |
|
2023-08-28
§
|
| 23:56 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2119 (T343718)', diff saved to https://phabricator.wikimedia.org/P51785 and previous config saved to /var/cache/conftool/dbconfig/20230828-235648-ladsgroup.json |
[production] |
| 23:23 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1219 (T343718)', diff saved to https://phabricator.wikimedia.org/P51784 and previous config saved to /var/cache/conftool/dbconfig/20230828-232344-ladsgroup.json |
[production] |
| 23:23 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1219.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:23 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1219.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:20 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.restart |
[production] |
| 23:11 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.restart (exit_code=99) |
[production] |
| 23:11 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.restart |
[production] |
| 23:11 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) |
[production] |
| 22:53 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1218.eqiad.wmnet with reason: Maintenance |
[production] |
| 22:53 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1218.eqiad.wmnet with reason: Maintenance |
[production] |
| 22:53 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1207 (T343718)', diff saved to https://phabricator.wikimedia.org/P51783 and previous config saved to /var/cache/conftool/dbconfig/20230828-225306-ladsgroup.json |
[production] |
| 22:38 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P51782 and previous config saved to /var/cache/conftool/dbconfig/20230828-223800-ladsgroup.json |
[production] |
| 22:37 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1144:3314 (T343718)', diff saved to https://phabricator.wikimedia.org/P51781 and previous config saved to /var/cache/conftool/dbconfig/20230828-223740-ladsgroup.json |
[production] |
| 22:37 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1144.eqiad.wmnet with reason: Maintenance |
[production] |
| 22:37 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1144.eqiad.wmnet with reason: Maintenance |
[production] |
| 22:37 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1143 (T343718)', diff saved to https://phabricator.wikimedia.org/P51780 and previous config saved to /var/cache/conftool/dbconfig/20230828-223719-ladsgroup.json |
[production] |
| 22:33 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.restart |
[production] |
| 22:32 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.restart (exit_code=99) |
[production] |
| 22:22 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P51779 and previous config saved to /var/cache/conftool/dbconfig/20230828-222254-ladsgroup.json |
[production] |
| 22:22 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1143', diff saved to https://phabricator.wikimedia.org/P51778 and previous config saved to /var/cache/conftool/dbconfig/20230828-222212-ladsgroup.json |
[production] |
| 22:14 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.restart |
[production] |
| 22:14 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.restart (exit_code=99) |
[production] |
| 22:14 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.restart |
[production] |
| 22:07 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1207 (T343718)', diff saved to https://phabricator.wikimedia.org/P51777 and previous config saved to /var/cache/conftool/dbconfig/20230828-220747-ladsgroup.json |
[production] |
| 22:07 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1143', diff saved to https://phabricator.wikimedia.org/P51776 and previous config saved to /var/cache/conftool/dbconfig/20230828-220706-ladsgroup.json |
[production] |
| 21:57 |
<ryankemper@deploy1002> |
Finished deploy [wdqs/wdqs@16e3dcf]: T337296 restart services for new federation endpoint (duration: 01m 12s) |
[production] |
| 21:56 |
<ryankemper@deploy1002> |
Started deploy [wdqs/wdqs@16e3dcf]: T337296 restart services for new federation endpoint |
[production] |