|
2022-11-16
ยง
|
| 03:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2177 (T321130)', diff saved to https://phabricator.wikimedia.org/P39890 and previous config saved to /var/cache/conftool/dbconfig/20221116-035750-marostegui.json |
[production] |
| 03:51 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1119', diff saved to https://phabricator.wikimedia.org/P39889 and previous config saved to /var/cache/conftool/dbconfig/20221116-035124-ladsgroup.json |
[production] |
| 03:46 |
<eileen> |
civicrm upgraded from fa71f219 to 85c98fc7 |
[production] |
| 03:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P39888 and previous config saved to /var/cache/conftool/dbconfig/20221116-034243-marostegui.json |
[production] |
| 03:36 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1119', diff saved to https://phabricator.wikimedia.org/P39887 and previous config saved to /var/cache/conftool/dbconfig/20221116-033617-ladsgroup.json |
[production] |
| 03:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P39886 and previous config saved to /var/cache/conftool/dbconfig/20221116-032737-marostegui.json |
[production] |
| 03:21 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1119 (T318605)', diff saved to https://phabricator.wikimedia.org/P39885 and previous config saved to /var/cache/conftool/dbconfig/20221116-032111-ladsgroup.json |
[production] |
| 03:16 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2146 (T318605)', diff saved to https://phabricator.wikimedia.org/P39884 and previous config saved to /var/cache/conftool/dbconfig/20221116-031642-ladsgroup.json |
[production] |
| 03:16 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2146.codfw.wmnet with reason: Maintenance |
[production] |
| 03:16 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2146.codfw.wmnet with reason: Maintenance |
[production] |
| 03:16 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2145 (T318605)', diff saved to https://phabricator.wikimedia.org/P39883 and previous config saved to /var/cache/conftool/dbconfig/20221116-031621-ladsgroup.json |
[production] |
| 03:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2177 (T321130)', diff saved to https://phabricator.wikimedia.org/P39882 and previous config saved to /var/cache/conftool/dbconfig/20221116-031230-marostegui.json |
[production] |
| 03:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2145', diff saved to https://phabricator.wikimedia.org/P39881 and previous config saved to /var/cache/conftool/dbconfig/20221116-030115-ladsgroup.json |
[production] |
| 02:57 |
<sukhe@cumin2002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 02:46 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2145', diff saved to https://phabricator.wikimedia.org/P39880 and previous config saved to /var/cache/conftool/dbconfig/20221116-024608-ladsgroup.json |
[production] |
| 02:38 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2177 (T321130)', diff saved to https://phabricator.wikimedia.org/P39879 and previous config saved to /var/cache/conftool/dbconfig/20221116-023815-marostegui.json |
[production] |
| 02:38 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2177.codfw.wmnet with reason: Maintenance |
[production] |
| 02:37 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2177.codfw.wmnet with reason: Maintenance |
[production] |
| 02:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2156 (T321130)', diff saved to https://phabricator.wikimedia.org/P39878 and previous config saved to /var/cache/conftool/dbconfig/20221116-023743-marostegui.json |
[production] |
| 02:31 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2145 (T318605)', diff saved to https://phabricator.wikimedia.org/P39877 and previous config saved to /var/cache/conftool/dbconfig/20221116-023101-ladsgroup.json |
[production] |
| 02:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P39876 and previous config saved to /var/cache/conftool/dbconfig/20221116-022236-marostegui.json |
[production] |
| 02:19 |
<sukhe@cumin2002> |
START - Cookbook sre.hosts.reimage for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 02:13 |
<sukhe@cumin2002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 02:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P39875 and previous config saved to /var/cache/conftool/dbconfig/20221116-020730-marostegui.json |
[production] |
| 01:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2156 (T321130)', diff saved to https://phabricator.wikimedia.org/P39874 and previous config saved to /var/cache/conftool/dbconfig/20221116-015223-marostegui.json |
[production] |
| 01:45 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1119 (T318605)', diff saved to https://phabricator.wikimedia.org/P39873 and previous config saved to /var/cache/conftool/dbconfig/20221116-014502-ladsgroup.json |
[production] |
| 01:44 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1119.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:44 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1119.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:44 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1118 (T318605)', diff saved to https://phabricator.wikimedia.org/P39872 and previous config saved to /var/cache/conftool/dbconfig/20221116-014441-ladsgroup.json |
[production] |
| 01:43 |
<sukhe@cumin2002> |
START - Cookbook sre.hosts.reimage for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 01:43 |
<sukhe@cumin2002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 01:29 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1118', diff saved to https://phabricator.wikimedia.org/P39871 and previous config saved to /var/cache/conftool/dbconfig/20221116-012934-ladsgroup.json |
[production] |
| 01:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2156 (T321130)', diff saved to https://phabricator.wikimedia.org/P39870 and previous config saved to /var/cache/conftool/dbconfig/20221116-012726-marostegui.json |
[production] |
| 01:27 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2094.codfw.wmnet with reason: Maintenance |
[production] |
| 01:27 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db2094.codfw.wmnet with reason: Maintenance |
[production] |
| 01:27 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2156.codfw.wmnet with reason: Maintenance |
[production] |
| 01:26 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2156.codfw.wmnet with reason: Maintenance |
[production] |
| 01:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149 (T321130)', diff saved to https://phabricator.wikimedia.org/P39869 and previous config saved to /var/cache/conftool/dbconfig/20221116-012649-marostegui.json |
[production] |
| 01:15 |
<sukhe@cumin2002> |
START - Cookbook sre.hosts.reimage for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 01:14 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1118', diff saved to https://phabricator.wikimedia.org/P39867 and previous config saved to /var/cache/conftool/dbconfig/20221116-011427-ladsgroup.json |
[production] |
| 01:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P39866 and previous config saved to /var/cache/conftool/dbconfig/20221116-011143-marostegui.json |
[production] |
| 01:03 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2145 (T318605)', diff saved to https://phabricator.wikimedia.org/P39865 and previous config saved to /var/cache/conftool/dbconfig/20221116-010330-ladsgroup.json |
[production] |
| 01:03 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2145.codfw.wmnet with reason: Maintenance |
[production] |
| 01:03 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2145.codfw.wmnet with reason: Maintenance |
[production] |
| 00:59 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1118 (T318605)', diff saved to https://phabricator.wikimedia.org/P39864 and previous config saved to /var/cache/conftool/dbconfig/20221116-005921-ladsgroup.json |
[production] |
| 00:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P39863 and previous config saved to /var/cache/conftool/dbconfig/20221116-005636-marostegui.json |
[production] |
| 00:53 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbprov2004.codfw.wmnet with OS bullseye |
[production] |
| 00:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149 (T321130)', diff saved to https://phabricator.wikimedia.org/P39862 and previous config saved to /var/cache/conftool/dbconfig/20221116-004130-marostegui.json |
[production] |
| 00:33 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2004.codfw.wmnet with reason: host reimage |
[production] |
| 00:29 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2004.codfw.wmnet with reason: host reimage |
[production] |