|
2022-11-16
§
|
| 01:14 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1118', diff saved to https://phabricator.wikimedia.org/P39867 and previous config saved to /var/cache/conftool/dbconfig/20221116-011427-ladsgroup.json |
[production] |
| 01:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P39866 and previous config saved to /var/cache/conftool/dbconfig/20221116-011143-marostegui.json |
[production] |
| 01:03 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2145 (T318605)', diff saved to https://phabricator.wikimedia.org/P39865 and previous config saved to /var/cache/conftool/dbconfig/20221116-010330-ladsgroup.json |
[production] |
| 01:03 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2145.codfw.wmnet with reason: Maintenance |
[production] |
| 01:03 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2145.codfw.wmnet with reason: Maintenance |
[production] |
| 00:59 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1118 (T318605)', diff saved to https://phabricator.wikimedia.org/P39864 and previous config saved to /var/cache/conftool/dbconfig/20221116-005921-ladsgroup.json |
[production] |
| 00:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P39863 and previous config saved to /var/cache/conftool/dbconfig/20221116-005636-marostegui.json |
[production] |
| 00:53 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbprov2004.codfw.wmnet with OS bullseye |
[production] |
| 00:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149 (T321130)', diff saved to https://phabricator.wikimedia.org/P39862 and previous config saved to /var/cache/conftool/dbconfig/20221116-004130-marostegui.json |
[production] |
| 00:33 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2004.codfw.wmnet with reason: host reimage |
[production] |
| 00:29 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2004.codfw.wmnet with reason: host reimage |
[production] |
| 00:14 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host dbprov2004.codfw.wmnet with OS bullseye |
[production] |
| 00:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2149 (T321130)', diff saved to https://phabricator.wikimedia.org/P39861 and previous config saved to /var/cache/conftool/dbconfig/20221116-000645-marostegui.json |
[production] |
| 00:06 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Maintenance |
[production] |
| 00:06 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2149.codfw.wmnet with reason: Maintenance |
[production] |
| 00:05 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudmetrics[1001-1002].eqiad.wmnet |
[production] |
| 00:05 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 00:03 |
<andrew@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
|
2022-11-15
§
|
| 23:54 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.decommission for hosts cloudmetrics[1001-1002].eqiad.wmnet |
[production] |
| 23:41 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2141.codfw.wmnet with reason: Maintenance |
[production] |
| 23:41 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2141.codfw.wmnet with reason: Maintenance |
[production] |
| 23:40 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2130 (T318605)', diff saved to https://phabricator.wikimedia.org/P39860 and previous config saved to /var/cache/conftool/dbconfig/20221115-234056-ladsgroup.json |
[production] |
| 23:33 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2139.codfw.wmnet with reason: Maintenance |
[production] |
| 23:32 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2139.codfw.wmnet with reason: Maintenance |
[production] |
| 23:32 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2109 (T321130)', diff saved to https://phabricator.wikimedia.org/P39859 and previous config saved to /var/cache/conftool/dbconfig/20221115-233253-marostegui.json |
[production] |
| 23:26 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1118 (T318605)', diff saved to https://phabricator.wikimedia.org/P39858 and previous config saved to /var/cache/conftool/dbconfig/20221115-232600-ladsgroup.json |
[production] |
| 23:25 |
<brennen@deploy1002> |
Finished scap: Backport for [[gerrit:856582|Feed: Use DerivativeContext and not clone main RequestContext (T323153)]] (duration: 06m 26s) |
[production] |
| 23:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2130', diff saved to https://phabricator.wikimedia.org/P39857 and previous config saved to /var/cache/conftool/dbconfig/20221115-232550-ladsgroup.json |
[production] |
| 23:25 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1118.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:25 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1118.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1107 (T318605)', diff saved to https://phabricator.wikimedia.org/P39856 and previous config saved to /var/cache/conftool/dbconfig/20221115-232532-ladsgroup.json |
[production] |
| 23:21 |
<ebernhardson@deploy1002> |
Finished deploy [wikimedia/discovery/analytics@7762e35]: import_cirrus_indexes: snapshot partitioning should not use dashes (duration: 02m 16s) |
[production] |
| 23:19 |
<brennen@deploy1002> |
brennen and tstarling: Backport for [[gerrit:856582|Feed: Use DerivativeContext and not clone main RequestContext (T323153)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 23:19 |
<brennen@deploy1002> |
Started scap: Backport for [[gerrit:856582|Feed: Use DerivativeContext and not clone main RequestContext (T323153)]] |
[production] |
| 23:18 |
<ebernhardson@deploy1002> |
Started deploy [wikimedia/discovery/analytics@7762e35]: import_cirrus_indexes: snapshot partitioning should not use dashes |
[production] |
| 23:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2109', diff saved to https://phabricator.wikimedia.org/P39855 and previous config saved to /var/cache/conftool/dbconfig/20221115-231746-marostegui.json |
[production] |
| 23:10 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2130', diff saved to https://phabricator.wikimedia.org/P39854 and previous config saved to /var/cache/conftool/dbconfig/20221115-231043-ladsgroup.json |
[production] |
| 23:10 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1107', diff saved to https://phabricator.wikimedia.org/P39853 and previous config saved to /var/cache/conftool/dbconfig/20221115-231025-ladsgroup.json |
[production] |
| 23:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2109', diff saved to https://phabricator.wikimedia.org/P39852 and previous config saved to /var/cache/conftool/dbconfig/20221115-230240-marostegui.json |
[production] |
| 22:55 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2130 (T318605)', diff saved to https://phabricator.wikimedia.org/P39851 and previous config saved to /var/cache/conftool/dbconfig/20221115-225537-ladsgroup.json |
[production] |
| 22:55 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1107', diff saved to https://phabricator.wikimedia.org/P39850 and previous config saved to /var/cache/conftool/dbconfig/20221115-225518-ladsgroup.json |
[production] |
| 22:52 |
<ejegg> |
re-enabled civicrm dedupe jobs |
[production] |
| 22:51 |
<mutante> |
phab1004 - running public_task_dump.py |
[production] |
| 22:51 |
<ejegg> |
civicrm upgraded from d85589e8 to fa71f219 |
[production] |
| 22:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2109 (T321130)', diff saved to https://phabricator.wikimedia.org/P39849 and previous config saved to /var/cache/conftool/dbconfig/20221115-224733-marostegui.json |
[production] |
| 22:40 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1107 (T318605)', diff saved to https://phabricator.wikimedia.org/P39848 and previous config saved to /var/cache/conftool/dbconfig/20221115-224011-ladsgroup.json |
[production] |
| 22:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2109 (T321130)', diff saved to https://phabricator.wikimedia.org/P39847 and previous config saved to /var/cache/conftool/dbconfig/20221115-221247-marostegui.json |
[production] |
| 22:12 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2109.codfw.wmnet with reason: Maintenance |
[production] |
| 22:12 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2109.codfw.wmnet with reason: Maintenance |
[production] |
| 22:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2105 (T321130)', diff saved to https://phabricator.wikimedia.org/P39846 and previous config saved to /var/cache/conftool/dbconfig/20221115-221225-marostegui.json |
[production] |