|
2025-01-30
§
|
| 11:39 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2029.codfw.wmnet |
[production] |
| 11:28 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2224', diff saved to https://phabricator.wikimedia.org/P72853 and previous config saved to /var/cache/conftool/dbconfig/20250130-112853-marostegui.json |
[production] |
| 11:22 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2191', diff saved to https://phabricator.wikimedia.org/P72850 and previous config saved to /var/cache/conftool/dbconfig/20250130-112252-marostegui.json |
[production] |
| 11:12 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2172', diff saved to https://phabricator.wikimedia.org/P72849 and previous config saved to /var/cache/conftool/dbconfig/20250130-111244-marostegui.json |
[production] |
| 09:39 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1183 T385147', diff saved to https://phabricator.wikimedia.org/P72841 and previous config saved to /var/cache/conftool/dbconfig/20250130-093927-marostegui.json |
[production] |
| 08:45 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2095,2175,2186].codfw.wmnet |
[production] |
| 08:45 |
<jayme@cumin1002> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2095,2175,2186].codfw.wmnet |
[production] |
| 07:42 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1161 T384994', diff saved to https://phabricator.wikimedia.org/P72833 and previous config saved to /var/cache/conftool/dbconfig/20250130-074228-marostegui.json |
[production] |
|
2025-01-29
§
|
| 18:26 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2030.codfw.wmnet |
[production] |
| 17:50 |
<hnowlan@cumin2002> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2242-2243].codfw.wmnet |
[production] |
| 17:50 |
<hnowlan@cumin2002> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2242-2243].codfw.wmnet |
[production] |
| 16:59 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1159 T384994', diff saved to https://phabricator.wikimedia.org/P72787 and previous config saved to /var/cache/conftool/dbconfig/20250129-165951-marostegui.json |
[production] |
| 15:05 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2095,2175,2186].codfw.wmnet |
[production] |
| 15:03 |
<jayme@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on wikikube-worker[2095,2175,2186].codfw.wmnet with reason: Depooled via sre.k8s.pool-depool-node |
[production] |
| 15:03 |
<jayme@cumin1002> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2095,2175,2186].codfw.wmnet |
[production] |
| 14:33 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1185 db2178 T384994', diff saved to https://phabricator.wikimedia.org/P72756 and previous config saved to /var/cache/conftool/dbconfig/20250129-143317-marostegui.json |
[production] |
| 14:14 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2030.codfw.wmnet |
[production] |
| 14:11 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2030.codfw.wmnet |
[production] |
| 14:07 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2030.codfw.wmnet |
[production] |
| 14:04 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2029.codfw.wmnet |
[production] |
| 14:02 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2029.codfw.wmnet |
[production] |
| 14:02 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2029.codfw.wmnet |
[production] |
| 13:58 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2029.codfw.wmnet |
[production] |
| 12:50 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1212 T384807', diff saved to https://phabricator.wikimedia.org/P72741 and previous config saved to /var/cache/conftool/dbconfig/20250129-125015-marostegui.json |
[production] |
| 12:02 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2211 T384994', diff saved to https://phabricator.wikimedia.org/P72738 and previous config saved to /var/cache/conftool/dbconfig/20250129-120213-marostegui.json |
[production] |
| 11:57 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1200', diff saved to https://phabricator.wikimedia.org/P72737 and previous config saved to /var/cache/conftool/dbconfig/20250129-115700-marostegui.json |
[production] |
| 11:24 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2031.codfw.wmnet |
[production] |
| 10:36 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depool es1025 T384912', diff saved to https://phabricator.wikimedia.org/P72730 and previous config saved to /var/cache/conftool/dbconfig/20250129-103652-fceratto.json |
[production] |
| 08:46 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1210 db2223 T384994', diff saved to https://phabricator.wikimedia.org/P72707 and previous config saved to /var/cache/conftool/dbconfig/20250129-084611-marostegui.json |
[production] |
| 08:14 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2031.codfw.wmnet |
[production] |
| 08:08 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2031.codfw.wmnet |
[production] |
| 07:58 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2031.codfw.wmnet |
[production] |
| 06:45 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1230 db2157 T384994', diff saved to https://phabricator.wikimedia.org/P72693 and previous config saved to /var/cache/conftool/dbconfig/20250129-064545-marostegui.json |
[production] |
| 06:12 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2156', diff saved to https://phabricator.wikimedia.org/P72691 and previous config saved to /var/cache/conftool/dbconfig/20250129-061214-marostegui.json |
[production] |
|
2025-01-28
§
|
| 19:36 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2028.codfw.wmnet |
[production] |
| 16:18 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db2177 T382842', diff saved to https://phabricator.wikimedia.org/P72640 and previous config saved to /var/cache/conftool/dbconfig/20250128-161857-marostegui.json |
[production] |
| 16:14 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2013,2036,2088].codfw.wmnet |
[production] |
| 16:14 |
<jayme@cumin1002> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2013,2036,2088].codfw.wmnet |
[production] |
| 16:06 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-ctrl1003.eqiad.wmnet |
[production] |
| 16:06 |
<jayme@cumin1002> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-ctrl1003.eqiad.wmnet |
[production] |
| 16:05 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depool db1166', diff saved to https://phabricator.wikimedia.org/P72637 and previous config saved to /var/cache/conftool/dbconfig/20250128-160518-marostegui.json |
[production] |
| 15:49 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-ctrl1003.eqiad.wmnet |
[production] |
| 15:49 |
<jayme@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-ctrl1003.eqiad.wmnet with reason: Depooled via sre.k8s.pool-depool-node |
[production] |
| 15:49 |
<jayme@cumin1002> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-ctrl1003.eqiad.wmnet |
[production] |
| 15:48 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-ctrl1002.eqiad.wmnet |
[production] |
| 15:48 |
<jayme@cumin1002> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-ctrl1002.eqiad.wmnet |
[production] |
| 15:14 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2028.codfw.wmnet |
[production] |
| 15:11 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2028.codfw.wmnet |
[production] |
| 15:10 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2028.codfw.wmnet |
[production] |
| 14:57 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2013,2036,2088].codfw.wmnet |
[production] |