|
2026-08-05
ยง
|
| 08:11 |
<jynus@cumin1003> |
START - Cookbook sre.dns.netbox |
[production] |
| 08:07 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2249', diff saved to https://phabricator.wikimedia.org/P95911 and previous config saved to /var/cache/conftool/dbconfig/20260805-080737-ladsgroup.json |
[production] |
| 08:05 |
<jynus@cumin1003> |
START - Cookbook sre.hosts.decommission for hosts db1150.eqiad.wmnet |
[production] |
| 08:02 |
<marostegui> |
Depool clouddb1020 (s5,s8) T434048 |
[production] |
| 08:02 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1020.eqiad.wmnet,service=s8 |
[production] |
| 08:02 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1020.eqiad.wmnet,service=s5 |
[production] |
| 08:01 |
<marostegui> |
Depool clouddb1018 (s2,s7) T434048 |
[production] |
| 08:01 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1018.eqiad.wmnet,service=s7 |
[production] |
| 08:01 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1018.eqiad.wmnet,service=s2 |
[production] |
| 08:01 |
<marostegui> |
Depool clouddb1017 (s1) T434048 |
[production] |
| 08:01 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1017.eqiad.wmnet,service=s1 |
[production] |
| 07:59 |
<marostegui> |
Depool clouddb1016 (s5,s8) T434048 |
[production] |
| 07:59 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1016.eqiad.wmnet,service=s8 |
[production] |
| 07:59 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1016.eqiad.wmnet,service=s5 |
[production] |
| 07:57 |
<marostegui> |
Depool clouddb1015 (s4,s6) T434048 |
[production] |
| 07:57 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1015.eqiad.wmnet,service=s6 |
[production] |
| 07:57 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1015.eqiad.wmnet,service=s4 |
[production] |
| 07:56 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2249 (T433990)', diff saved to https://phabricator.wikimedia.org/P95910 and previous config saved to /var/cache/conftool/dbconfig/20260805-075650-ladsgroup.json |
[production] |
| 07:54 |
<marostegui> |
Depool clouddb1014 (s2,s7) T434048 |
[production] |
| 07:54 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1014.eqiad.wmnet,service=s7 |
[production] |
| 07:54 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1014.eqiad.wmnet,service=s2 |
[production] |
| 07:53 |
<marostegui> |
Depool clouddb1013:s1 T434048 |
[production] |
| 07:53 |
<marostegui> |
Depool clouddb1013:s1 T409557 |
[production] |
| 07:53 |
<marostegui@cumin1003> |
conftool action : set/pooled=no; selector: name=clouddb1013.eqiad.wmnet,service=s1 |
[production] |
| 07:25 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Depooling db2249 (T433990)', diff saved to https://phabricator.wikimedia.org/P95909 and previous config saved to /var/cache/conftool/dbconfig/20260805-072529-ladsgroup.json |
[production] |
| 07:24 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2249.codfw.wmnet with reason: Maintenance |
[production] |
| 07:24 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2231 (T433990)', diff saved to https://phabricator.wikimedia.org/P95908 and previous config saved to /var/cache/conftool/dbconfig/20260805-072426-ladsgroup.json |
[production] |
| 07:21 |
<slyngshede@dns1004> |
END - running authdns-update |
[production] |
| 07:19 |
<slyngshede@dns1004> |
START - running authdns-update |
[production] |
| 07:13 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2231', diff saved to https://phabricator.wikimedia.org/P95906 and previous config saved to /var/cache/conftool/dbconfig/20260805-071340-ladsgroup.json |
[production] |
| 07:02 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2231', diff saved to https://phabricator.wikimedia.org/P95905 and previous config saved to /var/cache/conftool/dbconfig/20260805-070253-ladsgroup.json |
[production] |
| 06:52 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2231 (T433990)', diff saved to https://phabricator.wikimedia.org/P95904 and previous config saved to /var/cache/conftool/dbconfig/20260805-065206-ladsgroup.json |
[production] |
| 06:45 |
<kevinbazira@deploy1003> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . |
[production] |
| 06:22 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Depooling db2231 (T433990)', diff saved to https://phabricator.wikimedia.org/P95903 and previous config saved to /var/cache/conftool/dbconfig/20260805-062240-ladsgroup.json |
[production] |
| 06:21 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2231.codfw.wmnet with reason: Maintenance |
[production] |
| 06:21 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2215 (T433990)', diff saved to https://phabricator.wikimedia.org/P95902 and previous config saved to /var/cache/conftool/dbconfig/20260805-062137-ladsgroup.json |
[production] |
| 06:10 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2215', diff saved to https://phabricator.wikimedia.org/P95901 and previous config saved to /var/cache/conftool/dbconfig/20260805-061051-ladsgroup.json |
[production] |
| 06:00 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2215', diff saved to https://phabricator.wikimedia.org/P95900 and previous config saved to /var/cache/conftool/dbconfig/20260805-060004-ladsgroup.json |
[production] |
| 05:49 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2215 (T433990)', diff saved to https://phabricator.wikimedia.org/P95899 and previous config saved to /var/cache/conftool/dbconfig/20260805-054918-ladsgroup.json |
[production] |
| 05:19 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Depooling db2215 (T433990)', diff saved to https://phabricator.wikimedia.org/P95898 and previous config saved to /var/cache/conftool/dbconfig/20260805-051939-ladsgroup.json |
[production] |
| 05:18 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2215.codfw.wmnet with reason: Maintenance |
[production] |
| 04:30 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2201.codfw.wmnet with reason: Maintenance |
[production] |
| 03:40 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2197.codfw.wmnet with reason: Maintenance |
[production] |
| 03:40 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2196 (T433990)', diff saved to https://phabricator.wikimedia.org/P95897 and previous config saved to /var/cache/conftool/dbconfig/20260805-034036-ladsgroup.json |
[production] |
| 03:29 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2196', diff saved to https://phabricator.wikimedia.org/P95896 and previous config saved to /var/cache/conftool/dbconfig/20260805-032948-ladsgroup.json |
[production] |
| 03:19 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2196', diff saved to https://phabricator.wikimedia.org/P95895 and previous config saved to /var/cache/conftool/dbconfig/20260805-031902-ladsgroup.json |
[production] |
| 03:08 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2196 (T433990)', diff saved to https://phabricator.wikimedia.org/P95894 and previous config saved to /var/cache/conftool/dbconfig/20260805-030815-ladsgroup.json |
[production] |
| 02:34 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Depooling db2196 (T433990)', diff saved to https://phabricator.wikimedia.org/P95893 and previous config saved to /var/cache/conftool/dbconfig/20260805-023413-ladsgroup.json |
[production] |
| 02:33 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2196.codfw.wmnet with reason: Maintenance |
[production] |
| 02:33 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2186 (T433990)', diff saved to https://phabricator.wikimedia.org/P95892 and previous config saved to /var/cache/conftool/dbconfig/20260805-023310-ladsgroup.json |
[production] |