4251-4300 of 10000 results (12ms)
2026-08-13 §
12:20 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1050.eqiad.wmnet [production]
12:19 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1049.eqiad.wmnet [production]
12:08 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1049.eqiad.wmnet [production]
12:01 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1038.eqiad.wmnet [production]
11:41 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1038.eqiad.wmnet [production]
11:36 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1167: Depool db1167.eqiad.wmnet to then clone it to db1281.eqiad.wmnet - marostegui@cumin1003 [production]
11:35 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db1167: Depool db1167.eqiad.wmnet to then clone it to db1281.eqiad.wmnet - marostegui@cumin1003 [production]
11:28 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1034.eqiad.wmnet [production]
11:19 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1034.eqiad.wmnet [production]
10:45 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1033.eqiad.wmnet [production]
10:35 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1033.eqiad.wmnet [production]
09:43 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1165: Depool db1165.eqiad.wmnet to then clone it to db1279.eqiad.wmnet - marostegui@cumin1003 [production]
09:42 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db1165: Depool db1165.eqiad.wmnet to then clone it to db1279.eqiad.wmnet - marostegui@cumin1003 [production]
09:09 <cwilliams@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2214: Upgrading db2214.codfw.wmnet [production]
09:09 <cwilliams@cumin1003> START - Cookbook sre.mysql.depool depool db2214: Upgrading db2214.codfw.wmnet [production]
09:00 <cwilliams@cumin1003> dbctl commit (dc=all): 'Depool db2214 T434754', diff saved to https://phabricator.wikimedia.org/P96045 and previous config saved to /var/cache/conftool/dbconfig/20260813-090001-cwilliams.json [production]
08:20 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1161: Depool db1161.eqiad.wmnet to then clone it to db1275.eqiad.wmnet - marostegui@cumin1003 [production]
08:20 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db1161: Depool db1161.eqiad.wmnet to then clone it to db1275.eqiad.wmnet - marostegui@cumin1003 [production]
2026-08-12 §
17:03 <cwilliams@cumin1003> dbctl commit (dc=all): 'Depool db2192 T434635', diff saved to https://phabricator.wikimedia.org/P96030 and previous config saved to /var/cache/conftool/dbconfig/20260812-170338-cwilliams.json [production]
15:52 <jmm@cumin2003> END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti2046.codfw.wmnet [production]
14:27 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2046.codfw.wmnet [production]
14:25 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2045.codfw.wmnet [production]
14:14 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2045.codfw.wmnet [production]
14:12 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2030.codfw.wmnet [production]
14:00 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2030.codfw.wmnet [production]
13:59 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2029.codfw.wmnet [production]
13:48 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2029.codfw.wmnet [production]
13:41 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2028.codfw.wmnet [production]
13:27 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2028.codfw.wmnet [production]
13:20 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2027.codfw.wmnet [production]
13:12 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2027.codfw.wmnet [production]
13:02 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti6002.drmrs.wmnet [production]
12:31 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6002.drmrs.wmnet [production]
12:00 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti6004.drmrs.wmnet [production]
11:46 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6004.drmrs.wmnet [production]
11:45 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti6001.drmrs.wmnet [production]
11:33 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6001.drmrs.wmnet [production]
11:17 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti6003.drmrs.wmnet [production]
11:10 <jmm@cumin2003> END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti3005.esams.wmnet [production]
11:08 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1158: Depool db1158.eqiad.wmnet to then clone it to db1273.eqiad.wmnet - marostegui@cumin1003 [production]
11:07 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db1158: Depool db1158.eqiad.wmnet to then clone it to db1273.eqiad.wmnet - marostegui@cumin1003 [production]
11:02 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6003.drmrs.wmnet [production]
10:25 <cwilliams@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1159: Clone source for db1274 [production]
10:24 <cwilliams@cumin1003> START - Cookbook sre.mysql.depool depool db1159: Clone source for db1274 [production]
10:10 <cwilliams@cumin1003> dbctl commit (dc=all): 'Depool db2212 T434644', diff saved to https://phabricator.wikimedia.org/P96003 and previous config saved to /var/cache/conftool/dbconfig/20260812-101053-cwilliams.json [production]
09:37 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3005.esams.wmnet [production]
09:26 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti3006.esams.wmnet [production]
09:16 <jmm@cumin2003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3006.esams.wmnet [production]
08:28 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool ms3 T434288', diff saved to https://phabricator.wikimedia.org/P95993 and previous config saved to /var/cache/conftool/dbconfig/20260812-082852-marostegui.json [production]
07:35 <jmm@cumin2003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti3007.esams.wmnet [production]