6151-6200 of 10000 results (14ms)
2026-05-26 §
09:22 <jayme@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host kubestage2001.codfw.wmnet [production]
09:20 <jayme@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubestage1003.eqiad.wmnet [production]
09:15 <jayme@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubestage2001.codfw.wmnet [production]
09:14 <jayme@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host kubestage1003.eqiad.wmnet [production]
09:14 <jayme@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host kubestage2001.codfw.wmnet [production]
08:53 <cwilliams@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1226: Upgrading db1226.eqiad.wmnet [production]
08:50 <cwilliams@cumin1003> START - Cookbook sre.mysql.depool depool db1226: Upgrading db1226.eqiad.wmnet [production]
08:21 <cwilliams@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2195: Upgrading db2195.codfw.wmnet [production]
08:20 <cwilliams@cumin1003> START - Cookbook sre.mysql.depool depool db2195: Upgrading db2195.codfw.wmnet [production]
08:00 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1222: Upgrading db1222.eqiad.wmnet [production]
07:59 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db1222: Upgrading db1222.eqiad.wmnet [production]
07:46 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1222: Upgrading db1222.eqiad.wmnet [production]
07:45 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db1222: Upgrading db1222.eqiad.wmnet [production]
07:44 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1025.eqiad.wmnet [production]
07:40 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1046.eqiad.wmnet [production]
07:37 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1222: Upgrading db1222.eqiad.wmnet [production]
07:36 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db1222: Upgrading db1222.eqiad.wmnet [production]
07:26 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1046.eqiad.wmnet [production]
07:24 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1047.eqiad.wmnet [production]
07:15 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1047.eqiad.wmnet [production]
07:15 <jmm@cumin2002> END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti1026.eqiad.wmnet [production]
07:13 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1026.eqiad.wmnet [production]
07:11 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1025.eqiad.wmnet [production]
07:10 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1025.eqiad.wmnet [production]
07:04 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1048.eqiad.wmnet [production]
06:55 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1048.eqiad.wmnet [production]
06:16 <fceratto@cumin1003> dbctl commit (dc=all): 'Depool db1222 T425622', diff saved to https://phabricator.wikimedia.org/P92910 and previous config saved to /var/cache/conftool/dbconfig/20260526-061656-fceratto.json [production]
05:15 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc1014.eqiad.wmnet: Maintenance on pc4 [production]
05:15 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool pc1014.eqiad.wmnet: Maintenance on pc4 [production]
2026-05-25 §
12:59 <fceratto@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reboot [production]
12:59 <fceratto@cumin1003> START - Cookbook sre.mysql.depool depool db1162: Reboot [production]
12:54 <fceratto@cumin1003> END (FAIL) - Cookbook sre.mysql.depool (exit_code=99) depool db1162: Reboot [production]
12:54 <fceratto@cumin1003> START - Cookbook sre.mysql.depool depool db1162: Reboot [production]
05:17 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool pc1013.eqiad.wmnet: Maintenance on pc3 [production]
05:17 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool pc1013.eqiad.wmnet: Maintenance on pc3 [production]
2026-05-22 §
10:26 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1024.eqiad.wmnet [production]
09:39 <cwilliams@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2154: Upgrading db2154.codfw.wmnet [production]
09:39 <cwilliams@cumin1003> START - Cookbook sre.mysql.depool depool db2154: Upgrading db2154.codfw.wmnet [production]
09:35 <cwilliams@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1172: Upgrading db1172.eqiad.wmnet [production]
09:34 <cwilliams@cumin1003> START - Cookbook sre.mysql.depool depool db1172: Upgrading db1172.eqiad.wmnet [production]
07:31 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1024.eqiad.wmnet [production]
06:13 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1024.eqiad.wmnet [production]
06:08 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1024.eqiad.wmnet [production]
2026-05-21 §
16:18 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Depool pc2 (T421705)', diff saved to https://phabricator.wikimedia.org/P92807 and previous config saved to /var/cache/conftool/dbconfig/20260521-161808-ladsgroup.json [production]
15:21 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1045.eqiad.wmnet [production]
15:11 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1045.eqiad.wmnet [production]
15:10 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2034.codfw.wmnet [production]
15:10 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1037.eqiad.wmnet [production]
14:54 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2034.codfw.wmnet [production]
14:53 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1037.eqiad.wmnet [production]