8651-8700 of 10000 results (9ms)
2025-08-11 §
15:06 <klausman@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host ml-serve2003.codfw.wmnet [production]
14:36 <klausman@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host ml-serve2003.codfw.wmnet [production]
14:36 <klausman@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host ml-serve2003.codfw.wmnet [production]
14:35 <klausman@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host ml-serve2002.codfw.wmnet [production]
14:35 <klausman@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host ml-serve2002.codfw.wmnet [production]
14:20 <klausman@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host ml-serve2002.codfw.wmnet [production]
14:20 <klausman@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host ml-serve2002.codfw.wmnet [production]
14:20 <klausman@cumin1003> END (ERROR) - Cookbook sre.k8s.pool-depool-node (exit_code=97) depool for host ml-serve2002.codfw.wmnet [production]
14:10 <klausman@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host ml-serve2002.codfw.wmnet [production]
14:08 <klausman@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host ml-serve2001.codfw.wmnet [production]
14:08 <klausman@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host ml-serve2001.codfw.wmnet [production]
13:53 <klausman@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host ml-serve2001.codfw.wmnet [production]
13:53 <klausman@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host ml-serve2001.codfw.wmnet [production]
2025-07-29 §
06:36 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1202 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P80196 and previous config saved to /var/cache/conftool/dbconfig/20250729-063657-marostegui.json [production]
2025-07-28 §
10:02 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2162 T400599', diff saved to https://phabricator.wikimedia.org/P80061 and previous config saved to /var/cache/conftool/dbconfig/20250728-100243-marostegui.json [production]
09:04 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2218 T400591', diff saved to https://phabricator.wikimedia.org/P80039 and previous config saved to /var/cache/conftool/dbconfig/20250728-090407-marostegui.json [production]
07:16 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2220 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P80013 and previous config saved to /var/cache/conftool/dbconfig/20250728-071643-marostegui.json [production]
06:30 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2038 T400436', diff saved to https://phabricator.wikimedia.org/P80008 and previous config saved to /var/cache/conftool/dbconfig/20250728-063039-root.json [production]
2025-07-25 §
22:37 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2196 T400514', diff saved to https://phabricator.wikimedia.org/P79997 and previous config saved to /var/cache/conftool/dbconfig/20250725-223736-marostegui.json [production]
08:24 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2221 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79899 and previous config saved to /var/cache/conftool/dbconfig/20250725-082430-marostegui.json [production]
06:14 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2222 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79894 and previous config saved to /var/cache/conftool/dbconfig/20250725-061426-marostegui.json [production]
06:01 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2037 T400436', diff saved to https://phabricator.wikimedia.org/P79893 and previous config saved to /var/cache/conftool/dbconfig/20250725-060103-root.json [production]
05:54 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2038 T400435', diff saved to https://phabricator.wikimedia.org/P79890 and previous config saved to /var/cache/conftool/dbconfig/20250725-055449-root.json [production]
2025-07-24 §
16:34 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2036 T399927', diff saved to https://phabricator.wikimedia.org/P79855 and previous config saved to /var/cache/conftool/dbconfig/20250724-163439-root.json [production]
07:13 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1227 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79791 and previous config saved to /var/cache/conftool/dbconfig/20250724-071300-marostegui.json [production]
2025-07-23 §
18:52 <inflatador> depool eqiad in preparation for rolling restart T399162 [production]
2025-07-22 §
10:12 <cgoubert@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1243.eqiad.wmnet [production]
10:12 <cgoubert@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1243.eqiad.wmnet [production]
10:00 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1253 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79566 and previous config saved to /var/cache/conftool/dbconfig/20250722-100040-marostegui.json [production]
09:54 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2208 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79565 and previous config saved to /var/cache/conftool/dbconfig/20250722-095402-marostegui.json [production]
06:54 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2035 T399927', diff saved to https://phabricator.wikimedia.org/P79550 and previous config saved to /var/cache/conftool/dbconfig/20250722-065454-root.json [production]
00:26 <bking@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on cirrussearch1081.eqiad.wmnet with reason: bad cluster node/MSS issue [production]
2025-07-21 §
11:50 <XioNoX> depool ncredir7004 for ganeti7002 bird upgrade [production]
11:30 <XioNoX> depool and move ncredir7003 to ganeti7003 [production]
09:51 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2182 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79503 and previous config saved to /var/cache/conftool/dbconfig/20250721-095112-marostegui.json [production]
09:42 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1194 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79499 and previous config saved to /var/cache/conftool/dbconfig/20250721-094221-marostegui.json [production]
08:57 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1191 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79491 and previous config saved to /var/cache/conftool/dbconfig/20250721-085719-marostegui.json [production]
08:53 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2168 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79490 and previous config saved to /var/cache/conftool/dbconfig/20250721-085333-marostegui.json [production]
08:09 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2209 T399930', diff saved to https://phabricator.wikimedia.org/P79479 and previous config saved to /var/cache/conftool/dbconfig/20250721-080951-marostegui.json [production]
07:52 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2159 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79472 and previous config saved to /var/cache/conftool/dbconfig/20250721-075256-marostegui.json [production]
07:50 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1174 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79470 and previous config saved to /var/cache/conftool/dbconfig/20250721-075025-marostegui.json [production]
07:20 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es1037', diff saved to https://phabricator.wikimedia.org/P79465 and previous config saved to /var/cache/conftool/dbconfig/20250721-072037-root.json [production]
06:47 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es1039', diff saved to https://phabricator.wikimedia.org/P79453 and previous config saved to /var/cache/conftool/dbconfig/20250721-064755-root.json [production]
06:16 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1170 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79447 and previous config saved to /var/cache/conftool/dbconfig/20250721-061606-marostegui.json [production]
06:09 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2150 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79446 and previous config saved to /var/cache/conftool/dbconfig/20250721-060923-marostegui.json [production]
2025-07-18 §
13:15 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1212 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79392 and previous config saved to /var/cache/conftool/dbconfig/20250718-131554-marostegui.json [production]
12:19 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1198 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79381 and previous config saved to /var/cache/conftool/dbconfig/20250718-121901-marostegui.json [production]
08:38 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1189 for migration to mariadb 10.11', diff saved to https://phabricator.wikimedia.org/P79360 and previous config saved to /var/cache/conftool/dbconfig/20250718-083831-marostegui.json [production]
2025-07-17 §
22:57 <mutante> [cumin1002:~] $ sudo dbctl instance db1229 depool [production]
14:38 <cmooney@cumin1003> END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool site eqsin [reason: depool eqsin to test backhaul cct packet loss, T399221] [production]