9801-9850 of 10000 results (9ms)
2025-01-28 §
14:55 <jayme@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on wikikube-worker[2013,2036,2088].codfw.wmnet with reason: Depooled via sre.k8s.pool-depool-node [production]
14:55 <jayme@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2013,2036,2088].codfw.wmnet [production]
14:34 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2028.codfw.wmnet [production]
14:33 <jayme@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-ctrl1002.eqiad.wmnet [production]
14:33 <jayme@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-ctrl1002.eqiad.wmnet with reason: Depooled via sre.k8s.pool-depool-node [production]
14:33 <jayme@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-ctrl1002.eqiad.wmnet [production]
14:30 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2028.codfw.wmnet [production]
12:37 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1166 T382842', diff saved to https://phabricator.wikimedia.org/P72615 and previous config saved to /var/cache/conftool/dbconfig/20250128-123713-marostegui.json [production]
10:36 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2026.codfw.wmnet [production]
10:12 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2155 T384807', diff saved to https://phabricator.wikimedia.org/P72589 and previous config saved to /var/cache/conftool/dbconfig/20250128-101224-marostegui.json [production]
09:34 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2190', diff saved to https://phabricator.wikimedia.org/P72578 and previous config saved to /var/cache/conftool/dbconfig/20250128-093446-marostegui.json [production]
09:12 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2030', diff saved to https://phabricator.wikimedia.org/P72571 and previous config saved to /var/cache/conftool/dbconfig/20250128-091242-marostegui.json [production]
09:06 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1175', diff saved to https://phabricator.wikimedia.org/P72570 and previous config saved to /var/cache/conftool/dbconfig/20250128-090620-marostegui.json [production]
09:06 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2205', diff saved to https://phabricator.wikimedia.org/P72569 and previous config saved to /var/cache/conftool/dbconfig/20250128-090601-marostegui.json [production]
09:00 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2036', diff saved to https://phabricator.wikimedia.org/P72565 and previous config saved to /var/cache/conftool/dbconfig/20250128-090000-marostegui.json [production]
08:24 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2026.codfw.wmnet [production]
07:59 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2026.codfw.wmnet [production]
07:58 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2155 T382842', diff saved to https://phabricator.wikimedia.org/P72559 and previous config saved to /var/cache/conftool/dbconfig/20250128-075857-marostegui.json [production]
07:56 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2026.codfw.wmnet [production]
06:28 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1175 T384807', diff saved to https://phabricator.wikimedia.org/P72556 and previous config saved to /var/cache/conftool/dbconfig/20250128-062846-marostegui.json [production]
06:12 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2205 T384807', diff saved to https://phabricator.wikimedia.org/P72555 and previous config saved to /var/cache/conftool/dbconfig/20250128-061230-marostegui.json [production]
2025-01-27 §
18:48 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2194', diff saved to https://phabricator.wikimedia.org/P72528 and previous config saved to /var/cache/conftool/dbconfig/20250127-184839-marostegui.json [production]
16:58 <kamila@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1159-1165].eqiad.wmnet [production]
16:58 <kamila@cumin1002> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1159-1165].eqiad.wmnet [production]
16:32 <jayme@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-ctrl[1002-1003].eqiad.wmnet [production]
16:32 <jayme@cumin1002> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-ctrl[1002-1003].eqiad.wmnet [production]
16:03 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2194', diff saved to https://phabricator.wikimedia.org/P72497 and previous config saved to /var/cache/conftool/dbconfig/20250127-160300-marostegui.json [production]
15:56 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2194 T384807', diff saved to https://phabricator.wikimedia.org/P72495 and previous config saved to /var/cache/conftool/dbconfig/20250127-155613-marostegui.json [production]
15:09 <jayme@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-ctrl[1002-1003].eqiad.wmnet [production]
15:09 <jayme@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-ctrl[1002-1003].eqiad.wmnet with reason: Depooled via sre.k8s.pool-depool-node [production]
15:09 <jayme@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-ctrl[1002-1003].eqiad.wmnet [production]
14:32 <kamila@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host parse[1018-1024].eqiad.wmnet [production]
14:28 <kamila@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host parse[1018-1024].eqiad.wmnet [production]
13:27 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1157 T384807', diff saved to https://phabricator.wikimedia.org/P72480 and previous config saved to /var/cache/conftool/dbconfig/20250127-132710-marostegui.json [production]
13:07 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2025.codfw.wmnet [production]
12:18 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2227 T384807', diff saved to https://phabricator.wikimedia.org/P72470 and previous config saved to /var/cache/conftool/dbconfig/20250127-121843-marostegui.json [production]
11:40 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2025.codfw.wmnet [production]
11:37 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2025.codfw.wmnet [production]
11:35 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2025.codfw.wmnet [production]
11:14 <jmm@cumin2002> END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti2025.codfw.wmnet [production]
11:14 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2025.codfw.wmnet [production]
10:43 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2025.codfw.wmnet [production]
10:42 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2025.codfw.wmnet [production]
10:26 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es1024 T384820', diff saved to https://phabricator.wikimedia.org/P72457 and previous config saved to /var/cache/conftool/dbconfig/20250127-102657-marostegui.json [production]
10:07 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2020.codfw.wmnet [production]
08:48 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1223', diff saved to https://phabricator.wikimedia.org/P72446 and previous config saved to /var/cache/conftool/dbconfig/20250127-084857-marostegui.json [production]
08:02 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2020.codfw.wmnet [production]
2025-01-26 §
21:39 <mvernon@cumin2002> dbctl commit (dc=all): 'Depool db2182', diff saved to https://phabricator.wikimedia.org/P72438 and previous config saved to /var/cache/conftool/dbconfig/20250126-213912-mvernon.json [production]
2025-01-25 §
08:34 <fabfur@cumin1002> END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool site magru [reason: depool magru to check for cr issues, no task ID specified] [production]
08:34 <fabfur@cumin1002> START - Cookbook sre.dns.admin DNS admin: depool site magru [reason: depool magru to check for cr issues, no task ID specified] [production]