2401-2450 of 7171 results (26ms)
2024-02-09 §
18:03 <andrewbogott> updated the default security group, removing the 0.0.0.0/0 rule allowing port 22 access everywhere, replaced it with a 172.16.0.0/21 rule [tools]
13:06 <taavi> reboot tools-sgecron-2 due to high load [tools]
10:34 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component image-config [tools]
10:34 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.component.deploy for component image-config [tools]
09:56 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster [tools]
09:56 <taavi@cloudcumin1001> Added a new k8s worker-nfs tools-k8s-worker-nfs-14.tools.eqiad1.wikimedia.cloud to the cluster [tools]
09:47 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster [tools]
09:47 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-51 [tools]
09:46 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-51 [tools]
09:46 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-50 [tools]
09:46 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-50 [tools]
08:56 <dcaro> restart tools-k8s-worker-50 due to D some stuck processes [tools]
2024-02-08 §
13:03 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0) [tools]
13:03 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors [tools]
09:46 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster [tools]
09:46 <taavi@cloudcumin1001> Added a new k8s worker-nfs tools-k8s-worker-nfs-13.tools.eqiad1.wikimedia.cloud to the cluster [tools]
09:35 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster [tools]
09:34 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-49 [tools]
09:33 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-49 [tools]
09:33 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-48 [tools]
09:33 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-48 [tools]
09:32 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster [tools]
09:32 <taavi@cloudcumin1001> Added a new k8s worker-nfs tools-k8s-worker-nfs-12.tools.eqiad1.wikimedia.cloud to the cluster [tools]
09:23 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster [tools]
09:22 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-47 [tools]
09:22 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-47 [tools]
09:22 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-46 [tools]
09:21 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-46 [tools]
09:21 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster [tools]
09:21 <taavi@cloudcumin1001> Added a new k8s worker-nfs tools-k8s-worker-nfs-11.tools.eqiad1.wikimedia.cloud to the cluster [tools]
09:13 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster [tools]
09:11 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-45 [tools]
09:11 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-45 [tools]
09:10 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-44 [tools]
09:10 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-44 [tools]
09:10 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster [tools]
09:09 <taavi@cloudcumin1001> Added a new k8s worker-nfs tools-k8s-worker-nfs-10.tools.eqiad1.wikimedia.cloud to the cluster [tools]
09:00 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster [tools]
08:59 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster [tools]
08:58 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster [tools]
08:58 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-43 [tools]
08:57 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-43 [tools]
08:57 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-42 [tools]
08:56 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-42 [tools]
2024-02-07 §
21:33 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for all workers [tools]
18:00 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for all workers [tools]
17:58 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-9 [tools]
17:58 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-9 [tools]
17:24 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for all workers [tools]
17:23 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for all workers [tools]