|
2025-09-09
ยง
|
| 23:56 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2195', diff saved to https://phabricator.wikimedia.org/P83079 and previous config saved to /var/cache/conftool/dbconfig/20250909-235624-fceratto.json |
[production] |
| 23:53 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2176', diff saved to https://phabricator.wikimedia.org/P83078 and previous config saved to /var/cache/conftool/dbconfig/20250909-235300-fceratto.json |
[production] |
| 23:41 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2195', diff saved to https://phabricator.wikimedia.org/P83077 and previous config saved to /var/cache/conftool/dbconfig/20250909-234116-fceratto.json |
[production] |
| 23:37 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2176 (T402763)', diff saved to https://phabricator.wikimedia.org/P83076 and previous config saved to /var/cache/conftool/dbconfig/20250909-233752-fceratto.json |
[production] |
| 23:31 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db2176 (T402763)', diff saved to https://phabricator.wikimedia.org/P83075 and previous config saved to /var/cache/conftool/dbconfig/20250909-233101-fceratto.json |
[production] |
| 23:30 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2176.codfw.wmnet with reason: Maintenance |
[production] |
| 23:30 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2174 (T402763)', diff saved to https://phabricator.wikimedia.org/P83074 and previous config saved to /var/cache/conftool/dbconfig/20250909-233049-fceratto.json |
[production] |
| 23:26 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2195 (T402763)', diff saved to https://phabricator.wikimedia.org/P83073 and previous config saved to /var/cache/conftool/dbconfig/20250909-232608-fceratto.json |
[production] |
| 23:18 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db2195 (T402763)', diff saved to https://phabricator.wikimedia.org/P83072 and previous config saved to /var/cache/conftool/dbconfig/20250909-231854-fceratto.json |
[production] |
| 23:18 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2195.codfw.wmnet with reason: Maintenance |
[production] |
| 23:18 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2181 (T402763)', diff saved to https://phabricator.wikimedia.org/P83071 and previous config saved to /var/cache/conftool/dbconfig/20250909-231831-fceratto.json |
[production] |
| 23:17 |
<ejegg|afk> |
standalone SmashPig upgraded from 6f2ecf43 to 0fccf147 |
[production] |
| 23:15 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2174', diff saved to https://phabricator.wikimedia.org/P83070 and previous config saved to /var/cache/conftool/dbconfig/20250909-231542-fceratto.json |
[production] |
| 23:03 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P83069 and previous config saved to /var/cache/conftool/dbconfig/20250909-230323-fceratto.json |
[production] |
| 23:00 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2174', diff saved to https://phabricator.wikimedia.org/P83068 and previous config saved to /var/cache/conftool/dbconfig/20250909-230034-fceratto.json |
[production] |
| 22:48 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P83067 and previous config saved to /var/cache/conftool/dbconfig/20250909-224816-fceratto.json |
[production] |
| 22:45 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2174 (T402763)', diff saved to https://phabricator.wikimedia.org/P83066 and previous config saved to /var/cache/conftool/dbconfig/20250909-224527-fceratto.json |
[production] |
| 22:38 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db2174 (T402763)', diff saved to https://phabricator.wikimedia.org/P83065 and previous config saved to /var/cache/conftool/dbconfig/20250909-223835-fceratto.json |
[production] |
| 22:38 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2174.codfw.wmnet with reason: Maintenance |
[production] |
| 22:38 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2173 (T402763)', diff saved to https://phabricator.wikimedia.org/P83064 and previous config saved to /var/cache/conftool/dbconfig/20250909-223812-fceratto.json |
[production] |
| 22:33 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2181 (T402763)', diff saved to https://phabricator.wikimedia.org/P83063 and previous config saved to /var/cache/conftool/dbconfig/20250909-223308-fceratto.json |
[production] |
| 22:25 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db2181 (T402763)', diff saved to https://phabricator.wikimedia.org/P83062 and previous config saved to /var/cache/conftool/dbconfig/20250909-222514-fceratto.json |
[production] |
| 22:25 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2181.codfw.wmnet with reason: Maintenance |
[production] |
| 22:24 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2167 (T402763)', diff saved to https://phabricator.wikimedia.org/P83061 and previous config saved to /var/cache/conftool/dbconfig/20250909-222451-fceratto.json |
[production] |
| 22:23 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2173', diff saved to https://phabricator.wikimedia.org/P83060 and previous config saved to /var/cache/conftool/dbconfig/20250909-222305-fceratto.json |
[production] |
| 22:20 |
<jhathaway@cumin1002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dse-k8s-worker1014.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 22:17 |
<jhathaway@cumin1002> |
START - Cookbook sre.hosts.provision for host dse-k8s-worker1014.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 22:09 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2167', diff saved to https://phabricator.wikimedia.org/P83059 and previous config saved to /var/cache/conftool/dbconfig/20250909-220944-fceratto.json |
[production] |
| 22:07 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2173', diff saved to https://phabricator.wikimedia.org/P83058 and previous config saved to /var/cache/conftool/dbconfig/20250909-220757-fceratto.json |
[production] |
| 22:04 |
<bking@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 22:04 |
<bking@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 21:55 |
<krinkle@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1184937|beta: Remove replica instance from wmgMainStashServers (T401227)]] (duration: 12m 03s) |
[production] |
| 21:54 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2167', diff saved to https://phabricator.wikimedia.org/P83057 and previous config saved to /var/cache/conftool/dbconfig/20250909-215436-fceratto.json |
[production] |
| 21:52 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2173 (T402763)', diff saved to https://phabricator.wikimedia.org/P83056 and previous config saved to /var/cache/conftool/dbconfig/20250909-215249-fceratto.json |
[production] |
| 21:49 |
<krinkle@deploy1003> |
krinkle, bd808: Continuing with sync |
[production] |
| 21:48 |
<krinkle@deploy1003> |
krinkle, bd808: Backport for [[gerrit:1184937|beta: Remove replica instance from wmgMainStashServers (T401227)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 21:44 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db2173 (T402763)', diff saved to https://phabricator.wikimedia.org/P83055 and previous config saved to /var/cache/conftool/dbconfig/20250909-214449-fceratto.json |
[production] |
| 21:44 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2173.codfw.wmnet with reason: Maintenance |
[production] |
| 21:44 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2170 (T402763)', diff saved to https://phabricator.wikimedia.org/P83054 and previous config saved to /var/cache/conftool/dbconfig/20250909-214425-fceratto.json |
[production] |
| 21:43 |
<krinkle@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1184937|beta: Remove replica instance from wmgMainStashServers (T401227)]] |
[production] |
| 21:39 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2167 (T402763)', diff saved to https://phabricator.wikimedia.org/P83053 and previous config saved to /var/cache/conftool/dbconfig/20250909-213928-fceratto.json |
[production] |
| 21:31 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db2167 (T402763)', diff saved to https://phabricator.wikimedia.org/P83052 and previous config saved to /var/cache/conftool/dbconfig/20250909-213134-fceratto.json |
[production] |
| 21:31 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2167.codfw.wmnet with reason: Maintenance |
[production] |
| 21:31 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2166 (T402763)', diff saved to https://phabricator.wikimedia.org/P83051 and previous config saved to /var/cache/conftool/dbconfig/20250909-213112-fceratto.json |
[production] |
| 21:29 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2170', diff saved to https://phabricator.wikimedia.org/P83050 and previous config saved to /var/cache/conftool/dbconfig/20250909-212918-fceratto.json |
[production] |
| 21:24 |
<jhathaway@cumin1002> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-worker1014.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 21:21 |
<jhathaway@cumin1002> |
START - Cookbook sre.hosts.provision for host dse-k8s-worker1014.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 21:16 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2166', diff saved to https://phabricator.wikimedia.org/P83049 and previous config saved to /var/cache/conftool/dbconfig/20250909-211605-fceratto.json |
[production] |
| 21:14 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2170', diff saved to https://phabricator.wikimedia.org/P83048 and previous config saved to /var/cache/conftool/dbconfig/20250909-211410-fceratto.json |
[production] |
| 21:11 |
<jhathaway@cumin1002> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-worker1014.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART |
[production] |