|
2022-11-16
§
|
| 01:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2156 (T321130)', diff saved to https://phabricator.wikimedia.org/P39874 and previous config saved to /var/cache/conftool/dbconfig/20221116-015223-marostegui.json |
[production] |
| 01:45 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1119 (T318605)', diff saved to https://phabricator.wikimedia.org/P39873 and previous config saved to /var/cache/conftool/dbconfig/20221116-014502-ladsgroup.json |
[production] |
| 01:44 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1119.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:44 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1119.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:44 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1118 (T318605)', diff saved to https://phabricator.wikimedia.org/P39872 and previous config saved to /var/cache/conftool/dbconfig/20221116-014441-ladsgroup.json |
[production] |
| 01:43 |
<sukhe@cumin2002> |
START - Cookbook sre.hosts.reimage for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 01:43 |
<sukhe@cumin2002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 01:29 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1118', diff saved to https://phabricator.wikimedia.org/P39871 and previous config saved to /var/cache/conftool/dbconfig/20221116-012934-ladsgroup.json |
[production] |
| 01:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2156 (T321130)', diff saved to https://phabricator.wikimedia.org/P39870 and previous config saved to /var/cache/conftool/dbconfig/20221116-012726-marostegui.json |
[production] |
| 01:27 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2094.codfw.wmnet with reason: Maintenance |
[production] |
| 01:27 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db2094.codfw.wmnet with reason: Maintenance |
[production] |
| 01:27 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2156.codfw.wmnet with reason: Maintenance |
[production] |
| 01:26 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2156.codfw.wmnet with reason: Maintenance |
[production] |
| 01:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149 (T321130)', diff saved to https://phabricator.wikimedia.org/P39869 and previous config saved to /var/cache/conftool/dbconfig/20221116-012649-marostegui.json |
[production] |
| 01:15 |
<sukhe@cumin2002> |
START - Cookbook sre.hosts.reimage for host cp2042.codfw.wmnet with OS bullseye |
[production] |
| 01:14 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1118', diff saved to https://phabricator.wikimedia.org/P39867 and previous config saved to /var/cache/conftool/dbconfig/20221116-011427-ladsgroup.json |
[production] |
| 01:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P39866 and previous config saved to /var/cache/conftool/dbconfig/20221116-011143-marostegui.json |
[production] |
| 01:03 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2145 (T318605)', diff saved to https://phabricator.wikimedia.org/P39865 and previous config saved to /var/cache/conftool/dbconfig/20221116-010330-ladsgroup.json |
[production] |
| 01:03 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2145.codfw.wmnet with reason: Maintenance |
[production] |
| 01:03 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2145.codfw.wmnet with reason: Maintenance |
[production] |
| 00:59 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1118 (T318605)', diff saved to https://phabricator.wikimedia.org/P39864 and previous config saved to /var/cache/conftool/dbconfig/20221116-005921-ladsgroup.json |
[production] |
| 00:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P39863 and previous config saved to /var/cache/conftool/dbconfig/20221116-005636-marostegui.json |
[production] |
| 00:53 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbprov2004.codfw.wmnet with OS bullseye |
[production] |
| 00:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149 (T321130)', diff saved to https://phabricator.wikimedia.org/P39862 and previous config saved to /var/cache/conftool/dbconfig/20221116-004130-marostegui.json |
[production] |
| 00:33 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2004.codfw.wmnet with reason: host reimage |
[production] |
| 00:29 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2004.codfw.wmnet with reason: host reimage |
[production] |
| 00:14 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host dbprov2004.codfw.wmnet with OS bullseye |
[production] |
| 00:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2149 (T321130)', diff saved to https://phabricator.wikimedia.org/P39861 and previous config saved to /var/cache/conftool/dbconfig/20221116-000645-marostegui.json |
[production] |
| 00:06 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Maintenance |
[production] |
| 00:06 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2149.codfw.wmnet with reason: Maintenance |
[production] |
| 00:05 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudmetrics[1001-1002].eqiad.wmnet |
[production] |
| 00:05 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 00:03 |
<andrew@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
|
2022-11-15
§
|
| 23:54 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.decommission for hosts cloudmetrics[1001-1002].eqiad.wmnet |
[production] |
| 23:41 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2141.codfw.wmnet with reason: Maintenance |
[production] |
| 23:41 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2141.codfw.wmnet with reason: Maintenance |
[production] |
| 23:40 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2130 (T318605)', diff saved to https://phabricator.wikimedia.org/P39860 and previous config saved to /var/cache/conftool/dbconfig/20221115-234056-ladsgroup.json |
[production] |
| 23:33 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2139.codfw.wmnet with reason: Maintenance |
[production] |
| 23:32 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2139.codfw.wmnet with reason: Maintenance |
[production] |
| 23:32 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2109 (T321130)', diff saved to https://phabricator.wikimedia.org/P39859 and previous config saved to /var/cache/conftool/dbconfig/20221115-233253-marostegui.json |
[production] |
| 23:26 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1118 (T318605)', diff saved to https://phabricator.wikimedia.org/P39858 and previous config saved to /var/cache/conftool/dbconfig/20221115-232600-ladsgroup.json |
[production] |
| 23:25 |
<brennen@deploy1002> |
Finished scap: Backport for [[gerrit:856582|Feed: Use DerivativeContext and not clone main RequestContext (T323153)]] (duration: 06m 26s) |
[production] |
| 23:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2130', diff saved to https://phabricator.wikimedia.org/P39857 and previous config saved to /var/cache/conftool/dbconfig/20221115-232550-ladsgroup.json |
[production] |
| 23:25 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1118.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:25 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1118.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1107 (T318605)', diff saved to https://phabricator.wikimedia.org/P39856 and previous config saved to /var/cache/conftool/dbconfig/20221115-232532-ladsgroup.json |
[production] |
| 23:21 |
<ebernhardson@deploy1002> |
Finished deploy [wikimedia/discovery/analytics@7762e35]: import_cirrus_indexes: snapshot partitioning should not use dashes (duration: 02m 16s) |
[production] |
| 23:19 |
<brennen@deploy1002> |
brennen and tstarling: Backport for [[gerrit:856582|Feed: Use DerivativeContext and not clone main RequestContext (T323153)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 23:19 |
<brennen@deploy1002> |
Started scap: Backport for [[gerrit:856582|Feed: Use DerivativeContext and not clone main RequestContext (T323153)]] |
[production] |
| 23:18 |
<ebernhardson@deploy1002> |
Started deploy [wikimedia/discovery/analytics@7762e35]: import_cirrus_indexes: snapshot partitioning should not use dashes |
[production] |