8001-8050 of 10000 results (8ms)
2026-02-03 §
09:41 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db1223 T416298', diff saved to https://phabricator.wikimedia.org/P88457 and previous config saved to /var/cache/conftool/dbconfig/20260203-094116-marostegui.json [production]
06:11 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db2192 T415900', diff saved to https://phabricator.wikimedia.org/P88447 and previous config saved to /var/cache/conftool/dbconfig/20260203-061142-marostegui.json [production]
06:00 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db2214 T415862', diff saved to https://phabricator.wikimedia.org/P88443 and previous config saved to /var/cache/conftool/dbconfig/20260203-060000-marostegui.json [production]
2026-02-02 §
09:35 <dpogorzelski@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-cluster (exit_code=0) depool all services in codfw/ml-staging-codfw: maintenance [production]
09:35 <dpogorzelski@cumin1003> START - Cookbook sre.k8s.pool-depool-cluster depool all services in codfw/ml-staging-codfw: maintenance [production]
09:33 <dpogorzelski@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-cluster (exit_code=0) depool all services in codfw/ml-staging-codfw: maintenance [production]
09:33 <dpogorzelski@cumin1003> START - Cookbook sre.k8s.pool-depool-cluster depool all services in codfw/ml-staging-codfw: maintenance [production]
09:03 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db2249 T415358', diff saved to https://phabricator.wikimedia.org/P88364 and previous config saved to /var/cache/conftool/dbconfig/20260202-090328-marostegui.json [production]
06:25 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db1222 T415983', diff saved to https://phabricator.wikimedia.org/P88360 and previous config saved to /var/cache/conftool/dbconfig/20260202-062554-marostegui.json [production]
06:13 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db2161 T415748', diff saved to https://phabricator.wikimedia.org/P88357 and previous config saved to /var/cache/conftool/dbconfig/20260202-061310-marostegui.json [production]
05:57 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db1193 T416107', diff saved to https://phabricator.wikimedia.org/P88353 and previous config saved to /var/cache/conftool/dbconfig/20260202-055755-marostegui.json [production]
2026-01-29 §
14:30 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db1210 T415893', diff saved to https://phabricator.wikimedia.org/P88143 and previous config saved to /var/cache/conftool/dbconfig/20260129-143030-marostegui.json [production]
12:57 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db1209 T415879', diff saved to https://phabricator.wikimedia.org/P88127 and previous config saved to /var/cache/conftool/dbconfig/20260129-125739-marostegui.json [production]
06:58 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db1201 T415861', diff saved to https://phabricator.wikimedia.org/P88076 and previous config saved to /var/cache/conftool/dbconfig/20260129-065838-marostegui.json [production]
2026-01-28 §
06:50 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db2212 T415171', diff saved to https://phabricator.wikimedia.org/P87980 and previous config saved to /var/cache/conftool/dbconfig/20260128-065037-marostegui.json [production]
2026-01-27 §
08:25 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db1163 T415238', diff saved to https://phabricator.wikimedia.org/P87952 and previous config saved to /var/cache/conftool/dbconfig/20260127-082542-marostegui.json [production]
07:18 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.newdepool (exit_code=0) depool db2248: Reimage [production]
07:18 <marostegui@cumin1003> START - Cookbook sre.mysql.newdepool depool db2248: Reimage [production]
2026-01-26 §
09:26 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) db1224 - Depool db1224.eqiad.wmnet to then clone it to db1264.eqiad.wmnet - marostegui@cumin1003 [production]
09:25 <marostegui@cumin1003> START - Cookbook sre.mysql.depool db1224 - Depool db1224.eqiad.wmnet to then clone it to db1264.eqiad.wmnet - marostegui@cumin1003 [production]
08:48 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db1264 T415358', diff saved to https://phabricator.wikimedia.org/P87925 and previous config saved to /var/cache/conftool/dbconfig/20260126-084852-marostegui.json [production]
2026-01-25 §
23:55 <sukhe@cumin1003> END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool site esams [reason: no reason specified, ] [production]
23:54 <sukhe@cumin1003> START - Cookbook sre.dns.admin DNS admin: depool site esams [reason: no reason specified, ] [production]
2026-01-20 §
06:26 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db2179 T414543', diff saved to https://phabricator.wikimedia.org/P87779 and previous config saved to /var/cache/conftool/dbconfig/20260120-062653-marostegui.json [production]
2026-01-19 §
11:37 <marostegui@cumin1003> dbctl commit (dc=all): 'Depool db1160 T414542', diff saved to https://phabricator.wikimedia.org/P87748 and previous config saved to /var/cache/conftool/dbconfig/20260119-113722-marostegui.json [production]
2026-01-16 §
08:50 <tappof> depool titan2001, cleaning up block 01K88XDMJ9S0T2DR5K00VG9CFE (T410152) [production]
2026-01-15 §
19:50 <jasmine@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2003-2004,2007-2010,2019-2032,2040,2043,2045,2048].codfw.wmnet [production]
19:49 <jasmine@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2003-2004,2007-2010,2019-2032,2040,2043,2045,2048].codfw.wmnet [production]
17:10 <mutante> [cumin2002:~] $ sudo cumin -b 15 'tcp-proxy*' 'rm /lib/systemd/system/prometheus-node-textfile-check-nft*' [production]
2026-01-14 §
15:43 <cdobbins@cumin1003> END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool site ulsfo [reason: switch work, T408510] [production]
15:43 <cdobbins@cumin1003> START - Cookbook sre.dns.admin DNS admin: depool site ulsfo [reason: switch work, T408510] [production]
09:24 <tappof> Depool titan1002; disable Puppet and enable debug log level (T411273) [production]
2026-01-13 §
14:38 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.newdepool (exit_code=0) depool db2196: Schema change [production]
14:38 <marostegui@cumin1003> START - Cookbook sre.mysql.newdepool depool db2196: Schema change [production]
07:10 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.newdepool (exit_code=0) depool db2143: Schema change [production]
07:10 <marostegui@cumin1003> START - Cookbook sre.mysql.newdepool depool db2143: Schema change [production]
07:05 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.newdepool (exit_code=0) depool db2144: Schema change [production]
07:05 <marostegui@cumin1003> START - Cookbook sre.mysql.newdepool depool db2144: Schema change [production]
06:48 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.newdepool (exit_code=0) depool db2142: Schema change [production]
06:48 <marostegui@cumin1003> START - Cookbook sre.mysql.newdepool depool db2142: Schema change [production]
2026-01-09 §
09:50 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.newdepool (exit_code=0) depool es1049: test [production]
09:50 <marostegui@cumin1003> START - Cookbook sre.mysql.newdepool depool es1049: test [production]
2026-01-08 §
06:05 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.newdepool (exit_code=0) depool db2144: test [production]
06:05 <marostegui@cumin1003> START - Cookbook sre.mysql.newdepool depool db2144: test [production]
06:02 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.newdepool (exit_code=0) depool pc1013: test [production]
06:02 <marostegui@cumin1003> START - Cookbook sre.mysql.newdepool depool pc1013: test [production]
05:51 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) db2231 - Depool db2231.codfw.wmnet to then clone it to db2249.codfw.wmnet - marostegui@cumin1003 [production]
05:51 <marostegui@cumin1003> START - Cookbook sre.mysql.depool db2231 - Depool db2231.codfw.wmnet to then clone it to db2249.codfw.wmnet - marostegui@cumin1003 [production]
2026-01-05 §
08:59 <Emperor> depool / restart swift / repool ms-fe1014 T360913 [production]
08:58 <Emperor> depool / restart swift / repool ms-fe1010 T360913 [production]