|
2023-12-08
§
|
| 12:00 |
<wm-bot2> |
dcaro@urcuchillay END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on etcd-discovery-1.cloudinfra-codfw1dev.codfw1dev.wikimedia.cloud (T353055) |
[admin] |
| 11:58 |
<wm-bot2> |
dcaro@urcuchillay START - Cookbook wmcs.vps.refresh_puppet_certs on etcd-discovery-1.cloudinfra-codfw1dev.codfw1dev.wikimedia.cloud (T353055) |
[admin] |
| 11:57 |
<wm-bot2> |
dcaro@urcuchillay END (ERROR) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=97) on etcd-discovery-1.cloudinfra-codfw1dev.codfw1dev.wikimedia.cloud |
[admin] |
| 11:57 |
<wm-bot2> |
dcaro@urcuchillay START - Cookbook wmcs.vps.refresh_puppet_certs on etcd-discovery-1.cloudinfra-codfw1dev.codfw1dev.wikimedia.cloud |
[admin] |
| 09:38 |
<wm-bot2> |
dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) (T345084) |
[admin] |
| 09:32 |
<dcaro> |
restarting nova and keystone as they are getting too slow (T345084) |
[admin] |
| 09:32 |
<wm-bot2> |
dcaro@urcuchillay START - Cookbook wmcs.openstack.restart_openstack (T345084) |
[admin] |
| 09:32 |
<wm-bot2> |
dcaro@urcuchillay END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) (T345084) |
[admin] |
| 09:31 |
<wm-bot2> |
dcaro@urcuchillay START - Cookbook wmcs.openstack.restart_openstack (T345084) |
[admin] |
|
2023-12-02
§
|
| 12:27 |
<taavi> |
powercycle cloudvirt1063 T352595 |
[admin] |
| 11:28 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster |
[admin] |
| 11:28 |
<taavi@cloudcumin1001> |
Added a new k8s worker tools-k8s-worker-97.tools.eqiad1.wikimedia.cloud to the cluster |
[admin] |
| 11:02 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster |
[admin] |
| 11:02 |
<taavi@cloudcumin1001> |
Added a new k8s worker tools-k8s-worker-96.tools.eqiad1.wikimedia.cloud to the cluster |
[admin] |
| 10:50 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster |
[admin] |
| 10:50 |
<taavi@cloudcumin1001> |
Added a new k8s worker tools-k8s-worker-95.tools.eqiad1.wikimedia.cloud to the cluster |
[admin] |
| 00:21 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster |
[admin] |
| 00:21 |
<taavi@cloudcumin1001> |
Added a new k8s worker tools-k8s-worker-94.tools.eqiad1.wikimedia.cloud to the cluster |
[admin] |
| 00:18 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster |
[admin] |
| 00:18 |
<taavi@cloudcumin1001> |
Added a new k8s worker tools-k8s-worker-93.tools.eqiad1.wikimedia.cloud to the cluster |
[admin] |
| 00:15 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster |
[admin] |
| 00:15 |
<taavi@cloudcumin1001> |
Added a new k8s worker tools-k8s-worker-92.tools.eqiad1.wikimedia.cloud to the cluster |
[admin] |
| 00:06 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster |
[admin] |
| 00:06 |
<taavi@cloudcumin1001> |
Added a new k8s worker tools-k8s-worker-91.tools.eqiad1.wikimedia.cloud to the cluster |
[admin] |
| 00:05 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster |
[admin] |
| 00:05 |
<taavi@cloudcumin1001> |
Added a new k8s worker tools-k8s-worker-90.tools.eqiad1.wikimedia.cloud to the cluster |
[admin] |
| 00:01 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster |
[admin] |
| 00:01 |
<taavi@cloudcumin1001> |
Added a new k8s worker tools-k8s-worker-89.tools.eqiad1.wikimedia.cloud to the cluster |
[admin] |
|
2023-12-01
§
|
| 17:43 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) |
[admin] |
| 17:37 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.restart_openstack |
[admin] |
| 17:01 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) |
[admin] |
| 16:57 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.restart_openstack |
[admin] |
| 16:48 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) |
[admin] |
| 16:46 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.restart_openstack |
[admin] |
| 16:19 |
<wm-bot2> |
fran@wmf3169 END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T351171) |
[admin] |
| 16:19 |
<wm-bot2> |
fran@wmf3169 START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T351171) |
[admin] |
| 15:48 |
<andrewbogott> |
reimaging cloudcontrol1005 due to widespread misbehavior |
[admin] |
| 14:24 |
<wm-bot2> |
fran@wmf3169 END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T351171) |
[admin] |
| 14:20 |
<wm-bot2> |
fran@wmf3169 START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T351171) |
[admin] |
| 14:19 |
<fnegri@cloudcumin1001> |
END (ERROR) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=97) (T351171) |
[admin] |
| 14:18 |
<fnegri@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T351171) |
[admin] |
| 13:55 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) |
[admin] |
| 13:54 |
<taavi@cloudcumin1001> |
START - Cookbook wmcs.openstack.restart_openstack |
[admin] |
| 11:57 |
<taavi@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) |
[admin] |
| 11:56 |
<taavi@cloudcumin1001> |
START - Cookbook wmcs.openstack.restart_openstack |
[admin] |
| 11:55 |
<taavi> |
restart neutron-rpc-server.service on eqiad1 cloudcontrols |
[admin] |