|
2026-08-12
§
|
| 07:25 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3007.esams.wmnet |
[production] |
| 07:25 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti3008.esams.wmnet |
[production] |
| 07:11 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3008.esams.wmnet |
[production] |
| 07:07 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5007.eqsin.wmnet |
[production] |
| 06:54 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5007.eqsin.wmnet |
[production] |
| 06:36 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5006.eqsin.wmnet |
[production] |
| 06:23 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5006.eqsin.wmnet |
[production] |
| 06:20 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5005.eqsin.wmnet |
[production] |
| 06:06 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5005.eqsin.wmnet |
[production] |
| 06:03 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti5004.eqsin.wmnet |
[production] |
| 05:53 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti5004.eqsin.wmnet |
[production] |
|
2026-08-11
§
|
| 17:49 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7001.magru.wmnet |
[production] |
| 16:02 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7001.magru.wmnet |
[production] |
| 15:35 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4008.ulsfo.wmnet |
[production] |
| 15:18 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7004.magru.wmnet |
[production] |
| 15:11 |
<cwilliams@cumin1003> |
dbctl commit (dc=all): 'Depool db2204 T434565', diff saved to https://phabricator.wikimedia.org/P95984 and previous config saved to /var/cache/conftool/dbconfig/20260811-151126-cwilliams.json |
[production] |
| 15:10 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4008.ulsfo.wmnet |
[production] |
| 14:30 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7004.magru.wmnet |
[production] |
| 14:23 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7003.magru.wmnet |
[production] |
| 14:19 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4007.ulsfo.wmnet |
[production] |
| 14:12 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7003.magru.wmnet |
[production] |
| 14:11 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7002.magru.wmnet |
[production] |
| 14:06 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4007.ulsfo.wmnet |
[production] |
| 14:05 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4006.ulsfo.wmnet |
[production] |
| 13:58 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7002.magru.wmnet |
[production] |
| 13:56 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7001.magru.wmnet |
[production] |
| 13:52 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7001.magru.wmnet |
[production] |
| 13:51 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4006.ulsfo.wmnet |
[production] |
| 13:48 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4005.ulsfo.wmnet |
[production] |
| 13:37 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4005.ulsfo.wmnet |
[production] |
| 13:29 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2034.codfw.wmnet |
[production] |
| 12:54 |
<cwilliams@cumin1003> |
END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1157: Using as clone source |
[production] |
| 12:53 |
<cwilliams@cumin1003> |
START - Cookbook sre.mysql.depool depool db1157: Using as clone source |
[production] |
| 12:51 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depool ms2 T434288', diff saved to https://phabricator.wikimedia.org/P95977 and previous config saved to /var/cache/conftool/dbconfig/20260811-125129-marostegui.json |
[production] |
| 12:34 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2034.codfw.wmnet |
[production] |
| 12:30 |
<jmm@cumin2002> |
END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti-test2001.codfw.wmnet |
[production] |
| 12:22 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2033.codfw.wmnet |
[production] |
| 12:18 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2001.codfw.wmnet |
[production] |
| 12:04 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2033.codfw.wmnet |
[production] |
| 12:03 |
<jmm@cumin2003> |
END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti4005.ulsfo.wmnet |
[production] |
| 12:03 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4005.ulsfo.wmnet |
[production] |
| 11:57 |
<jmm@cumin2002> |
END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti-test2002.codfw.wmnet |
[production] |
| 11:50 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2002.codfw.wmnet |
[production] |
| 11:43 |
<cwilliams@cumin1003> |
dbctl commit (dc=all): 'Depool db2165 T434514', diff saved to https://phabricator.wikimedia.org/P95969 and previous config saved to /var/cache/conftool/dbconfig/20260811-114352-cwilliams.json |
[production] |
| 11:42 |
<jmm@cumin2003> |
END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti-test2003.codfw.wmnet |
[production] |
| 11:36 |
<jmm@cumin2003> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2003.codfw.wmnet |
[production] |
| 10:34 |
<marostegui@cumin1003> |
END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1179: Depool db1179.eqiad.wmnet to then clone it to db1278.eqiad.wmnet - marostegui@cumin1003 |
[production] |
| 10:18 |
<marostegui@cumin1003> |
START - Cookbook sre.mysql.depool depool db1179: Depool db1179.eqiad.wmnet to then clone it to db1278.eqiad.wmnet - marostegui@cumin1003 |
[production] |
| 09:28 |
<moritzm> |
installing node-tar security updates |
[production] |
| 09:09 |
<fceratto@cumin1003> |
END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1177: Decommission |
[production] |