2024-03-18
§
|
11:01 |
<aborrero@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics |
[tools] |
11:00 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics |
[tools] |
11:00 |
<aborrero@cloudcumin1001> |
END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component jobs-api |
[tools] |
11:00 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api |
[tools] |
11:00 |
<aborrero@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-7 from 1.23.17 to 1.24.17 (T359638) |
[tools] |
10:53 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-7 from 1.23.17 to 1.24.17 (T359638) |
[tools] |
10:53 |
<aborrero@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-7 from 1.23.17 to 1.24.17 (T359638) |
[tools] |
10:53 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-7 from 1.23.17 to 1.24.17 (T359638) |
[tools] |
10:47 |
<aborrero@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster tools upgrade from 1.23.17 to 1.24.17 (T307651) |
[tools] |
10:46 |
<aborrero@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster tools upgrade from 1.23.17 to 1.24.17 (T307651) |
[tools] |
10:04 |
<taavi@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-bastion-12.tools.eqiad1.wikimedia.cloud |
[tools] |
10:03 |
<taavi@cloudcumin1001> |
START - Cookbook wmcs.vps.refresh_puppet_certs on tools-bastion-12.tools.eqiad1.wikimedia.cloud |
[tools] |
09:27 |
<taavi> |
deleted shutdown grid engine VMs T314664 |
[tools] |
2024-03-14
§
|
17:26 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.apt.copy_to_main_repo (exit_code=0) for package 'misctools' version '1.48' |
[tools] |
17:26 |
<taavi@cloudcumin1001> |
START - Cookbook wmcs.toolforge.apt.copy_to_main_repo for package 'misctools' version '1.48' |
[tools] |
15:16 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-docker-imagebuilder-01 |
[tools] |
15:16 |
<taavi@cloudcumin1001> |
START - Cookbook wmcs.vps.remove_instance for instance tools-docker-imagebuilder-01 |
[tools] |
15:11 |
<taavi@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.vps.remove_instance (exit_code=99) for instance tools-docker-imagebuilder-01 |
[tools] |
15:11 |
<taavi@cloudcumin1001> |
START - Cookbook wmcs.vps.remove_instance for instance tools-docker-imagebuilder-01 |
[tools] |
15:10 |
<taavi@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.vps.remove_instance (exit_code=99) for instance tools-docker-imagebuilder-01 |
[tools] |
15:09 |
<taavi@cloudcumin1001> |
START - Cookbook wmcs.vps.remove_instance for instance tools-docker-imagebuilder-01 |
[tools] |
11:02 |
<taavi> |
stop grid related VMs T314664 |
[tools] |
11:01 |
<taavi> |
disable grid access for remaining tools still running on the grid T314664 |
[tools] |
2024-03-06
§
|
10:48 |
<taavi@cloudcumin1001> |
START - Cookbook wmcs.toolforge.remove_grid_node for tools-sgeweblight-10-32 |
[tools] |
10:47 |
<taavi@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.toolforge.remove_grid_node (exit_code=1) for tools-sgeweblight-10-17, tools-sgeweblight-10-32 |
[tools] |
10:47 |
<taavi@cloudcumin1001> |
START - Cookbook wmcs.toolforge.remove_grid_node for tools-sgeweblight-10-17, tools-sgeweblight-10-32 |
[tools] |
10:34 |
<taavi> |
rebuilding all docker images for https://gerrit.wikimedia.org/r/c/operations/docker-images/toollabs-images/+/1005952 (T293552) + normal package updates |
[tools] |
09:43 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0) |
[tools] |
09:43 |
<taavi@cloudcumin1001> |
START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors |
[tools] |
09:42 |
<taavi> |
reboot tools-sgeexec-10-20, -21, -23, sgeweblight-10-32 due to stuck nfs procs |
[tools] |