9101-9150 of 10000 results (10ms)
2025-06-04 §
11:09 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2193 T395989', diff saved to https://phabricator.wikimedia.org/P77031 and previous config saved to /var/cache/conftool/dbconfig/20250604-110955-marostegui.json [production]
10:35 <jmm@cumin1003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4007.ulsfo.wmnet [production]
10:25 <jmm@cumin1003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4007.ulsfo.wmnet [production]
10:23 <jmm@cumin1003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4006.ulsfo.wmnet [production]
10:09 <jmm@cumin1003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4006.ulsfo.wmnet [production]
10:00 <vgutierrez> depool lvs1013 before switching to katran - T395228 [production]
09:42 <jmm@cumin1003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4005.ulsfo.wmnet [production]
09:33 <jmm@cumin1003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4005.ulsfo.wmnet [production]
09:19 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool pc1 T395983', diff saved to https://phabricator.wikimedia.org/P77006 and previous config saved to /var/cache/conftool/dbconfig/20250604-091921-marostegui.json [production]
09:10 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2044', diff saved to https://phabricator.wikimedia.org/P77002 and previous config saved to /var/cache/conftool/dbconfig/20250604-091041-marostegui.json [production]
09:02 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2041', diff saved to https://phabricator.wikimedia.org/P76999 and previous config saved to /var/cache/conftool/dbconfig/20250604-090226-marostegui.json [production]
08:10 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2224 T395989', diff saved to https://phabricator.wikimedia.org/P76990 and previous config saved to /var/cache/conftool/dbconfig/20250604-081058-marostegui.json [production]
07:01 <jmm@cumin1003> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7001.magru.wmnet [production]
06:57 <jmm@cumin1003> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7001.magru.wmnet [production]
06:04 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es1035 T395982', diff saved to https://phabricator.wikimedia.org/P76972 and previous config saved to /var/cache/conftool/dbconfig/20250604-060413-marostegui.json [production]
2025-06-03 §
11:50 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2038 T395785', diff saved to https://phabricator.wikimedia.org/P76927 and previous config saved to /var/cache/conftool/dbconfig/20250603-115026-marostegui.json [production]
11:39 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es1039 T395647', diff saved to https://phabricator.wikimedia.org/P76922 and previous config saved to /var/cache/conftool/dbconfig/20250603-113946-marostegui.json [production]
05:27 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es1037 T395867', diff saved to https://phabricator.wikimedia.org/P76876 and previous config saved to /var/cache/conftool/dbconfig/20250603-052719-marostegui.json [production]
04:52 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2037 T395420', diff saved to https://phabricator.wikimedia.org/P76873 and previous config saved to /var/cache/conftool/dbconfig/20250603-045251-marostegui.json [production]
04:31 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2040 T395771', diff saved to https://phabricator.wikimedia.org/P76869 and previous config saved to /var/cache/conftool/dbconfig/20250603-043151-marostegui.json [production]
2025-06-02 §
17:05 <jasmine@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1026-1028].eqiad.wmnet [production]
17:04 <jasmine@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1026-1028].eqiad.wmnet [production]
16:22 <sukhe> sudo cumin -b1 -s60 'A:cp and not P{cp7001*}' "depool cdn && sleep 10 && run-puppet-agent --enable 'merging CR 1091330' && systemctl restart trafficserver.service && sleep 10 && pool cdn" [production]
09:44 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2039 T395647', diff saved to https://phabricator.wikimedia.org/P76798 and previous config saved to /var/cache/conftool/dbconfig/20250602-094402-marostegui.json [production]
07:06 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es1040 T395647', diff saved to https://phabricator.wikimedia.org/P76789 and previous config saved to /var/cache/conftool/dbconfig/20250602-070602-marostegui.json [production]
05:39 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1169 T395663', diff saved to https://phabricator.wikimedia.org/P76781 and previous config saved to /var/cache/conftool/dbconfig/20250602-053905-marostegui.json [production]
05:01 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2036 T395771', diff saved to https://phabricator.wikimedia.org/P76779 and previous config saved to /var/cache/conftool/dbconfig/20250602-050150-marostegui.json [production]
2025-05-30 §
08:21 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2040 T395647', diff saved to https://phabricator.wikimedia.org/P76703 and previous config saved to /var/cache/conftool/dbconfig/20250530-082144-marostegui.json [production]
08:18 <fceratto@cumin1002> dbctl commit (dc=all): 'Depool db2187 for reimaging, see T394884', diff saved to https://phabricator.wikimedia.org/P76702 and previous config saved to /var/cache/conftool/dbconfig/20250530-081804-fceratto.json [production]
2025-05-29 §
10:07 <fceratto@cumin1002> dbctl commit (dc=all): 'Depool es2039 T395294', diff saved to https://phabricator.wikimedia.org/P76665 and previous config saved to /var/cache/conftool/dbconfig/20250529-100704-fceratto.json [production]
2025-05-28 §
16:18 <stevemunene@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on an-worker1119.eqiad.wmnet with reason: Repair data node volume failure [production]
13:31 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: new ncredir node for magru03 - jmm@cumin2002" [production]
13:31 <jmm@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: new ncredir node for magru03 - jmm@cumin2002" [production]
10:15 <marostegui@cumin1002> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) db2242 - Depool db2242.codfw.wmnet to then clone it to db2187.codfw.wmnet - marostegui@cumin1002 [production]
10:15 <marostegui@cumin1002> START - Cookbook sre.mysql.depool db2242 - Depool db2242.codfw.wmnet to then clone it to db2187.codfw.wmnet - marostegui@cumin1002 [production]
09:56 <moritzm> installing node-serialize-javascript security updates [production]
09:05 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es2035', diff saved to https://phabricator.wikimedia.org/P76551 and previous config saved to /var/cache/conftool/dbconfig/20250528-090528-marostegui.json [production]
08:33 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool es1038 T394469', diff saved to https://phabricator.wikimedia.org/P76550 and previous config saved to /var/cache/conftool/dbconfig/20250528-083338-marostegui.json [production]
2025-05-27 §
19:11 <inflatador> bking@cumin2002 depool cirrussearch106[0-6] T394350 [production]
16:48 <jasmine@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1022-1025].eqiad.wmnet [production]
16:43 <jasmine@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1022-1025].eqiad.wmnet [production]
16:21 <klausman@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host ml-serve1007.eqiad.wmnet [production]
16:21 <klausman@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host ml-serve1007.eqiad.wmnet [production]
15:23 <klausman@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host ml-serve1007.eqiad.wmnet [production]
15:13 <klausman@cumin1003> START - Cookbook sre.k8s.pool-depool-node depool for host ml-serve1007.eqiad.wmnet [production]
15:12 <klausman@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host ml-serve1008.eqiad.wmnet [production]
15:12 <klausman@cumin1003> START - Cookbook sre.k8s.pool-depool-node pool for host ml-serve1008.eqiad.wmnet [production]
14:58 <elukey@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host ml-serve1006.eqiad.wmnet [production]
14:58 <elukey@cumin1002> START - Cookbook sre.k8s.pool-depool-node pool for host ml-serve1006.eqiad.wmnet [production]
13:52 <klausman@cumin1003> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host ml-serve1008.eqiad.wmnet [production]