851-900 of 4567 results (32ms)
2023-04-03 §
15:14 <wm-bot2> rebooted k8s node tools-k8s-worker-76 - cookbook ran by taavi@runko [tools]
15:12 <wm-bot2> rebooted k8s node tools-k8s-worker-77 - cookbook ran by taavi@runko [tools]
15:10 <wm-bot2> rebooted k8s node tools-k8s-worker-78 - cookbook ran by taavi@runko [tools]
15:08 <wm-bot2> rebooted k8s node tools-k8s-worker-79 - cookbook ran by taavi@runko [tools]
15:06 <wm-bot2> rebooted k8s node tools-k8s-worker-80 - cookbook ran by taavi@runko [tools]
14:59 <wm-bot2> rebooted k8s node tools-k8s-worker-81 - cookbook ran by taavi@runko [tools]
14:41 <wm-bot2> rebooted k8s node tools-k8s-worker-82 - cookbook ran by taavi@runko [tools]
14:38 <wm-bot2> rebooting the whole tools k8s cluster (58 nodes) - cookbook ran by taavi@runko [tools]
14:12 <andrewbogott> test log to see if stashbot is back working [tools]
13:19 <andrewbogott> forcing puppet run on all toolforge VMs [tools]
08:28 <taavi> stop exim4.service on tools-sgecron-2 T333477 [tools]
06:52 <taavi> stop jobs-framework-emailer to prevent spam due to NFS being read-only T333477 [tools]
2023-03-29 §
16:07 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/registry-admission-webhook (dc26f52) - cookbook ran by raymond@ubuntu [tools]
15:21 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/registry-admission:24115c7 from https://gerrit.wikimedia.org/r/labs/tools/registry-admission-webhook (24115c7) - cookbook ran by raymond@ubuntu [tools]
2023-03-28 §
19:43 <wm-bot2> deployed kubernetes component https://github.com/toolforge/buildpack-admission-controller (e1b9815) - cookbook ran by raymond@ubuntu [tools]
2023-03-27 §
22:51 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/toolforge-buildpack-admission-controller:70d550a from https://github.com/toolforge/buildpack-admission-controller (70d550a) - cookbook ran by raymond@ubuntu [tools]
2023-03-26 §
20:28 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by taavi@runko [tools]
2023-03-24 §
14:13 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by arturo@endurance [tools]
2023-03-21 §
08:11 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by taavi@runko [tools]
2023-03-20 §
13:39 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by taavi@runko [tools]
10:57 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by arturo@endurance [tools]
2023-03-19 §
09:32 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by taavi@runko [tools]
2023-03-17 §
15:56 <andrewbogott> truncating .out, .err, and .log files to 10MB in anticipation of moving the NFS volumes [tools]
2023-03-13 §
09:50 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/toolforge-buildpack-admission-controller:f90bd8f from https://github.com/toolforge/buildpack-admission-controller (f90bd8f) - cookbook ran by dcaro@vulcanus [tools]
2023-03-12 §
13:40 <taavi> restart haproxy on tools-k8s-haproxy-3 [tools]
2023-03-11 §
18:38 <wm-bot2> removing grid node tools-sgeexec-10-11.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
18:36 <wm-bot2> removing grid node tools-sgeexec-10-11.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
18:34 <wm-bot2> removing grid node tools-sgeexec-10-11.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
18:31 <taavi> reboot misbehaving tools-sgeexec-10-11 [tools]
2023-03-10 §
16:36 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/maintain-kubeusers (8b42b15) - cookbook ran by taavi@runko [tools]
2023-03-09 §
10:13 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/maintain-kubeusers (53e7f81) - cookbook ran by taavi@runko [tools]
10:04 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/maintain-kubeusers:834807c from https://gerrit.wikimedia.org/r/labs/tools/maintain-kubeusers (834807c) - cookbook ran by taavi@runko [tools]
2023-03-08 §
22:31 <bd808> Live hacked user-maintainer clusterrole to work around breakage in T331572 [tools]
2023-03-07 §
11:34 <wm-bot2> Increased quotas by 2 volumes - cookbook ran by fran@wmf3169 [tools]
11:09 <wm-bot2> Increased quotas by 6 snapshots - cookbook ran by fran@wmf3169 [tools]
11:07 <wm-bot2> Increased quotas by 4000 gigabytes - cookbook ran by fran@wmf3169 [tools]
2023-03-06 §
12:51 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/registry-admission-webhook (6688477) - cookbook ran by taavi@runko [tools]
12:33 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/registry-admission:e916fee from https://gerrit.wikimedia.org/r/labs/tools/registry-admission-webhook (e916fee) - cookbook ran by taavi@runko [tools]
12:16 <arturo> delete calico deployment, redeploy from https://gitlab.wikimedia.org/repos/cloud/toolforge/calico (T328539) [tools]
2023-03-05 §
15:43 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/maintain-kubeusers (3e04025) - cookbook ran by taavi@runko [tools]
2023-03-02 §
11:32 <arturo> aborrero@tools-k8s-control-2:~$ sudo -i kubectl apply -f /etc/kubernetes/toolforge-tool-roles.yaml (https://gerrit.wikimedia.org/r/c/operations/puppet/+/889836) [tools]
2023-03-01 §
13:18 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (13eda9d) - cookbook ran by taavi@runko [tools]
2023-02-28 §
17:19 <wm-bot2> deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway (9252af7) - cookbook ran by taavi@runko [tools]
17:04 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (e46da83) - cookbook ran by taavi@runko [tools]
2023-02-23 §
18:07 <wm-bot2> deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway (efb60b3) - cookbook ran by taavi@runko [tools]
09:33 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/buildpack-admission:b34e2f8 from https://github.com/toolforge/buildpack-admission-controller.git (b34e2f8) - cookbook ran by taavi@runko [tools]
2023-02-21 §
09:37 <arturo> hard-reboot tools-sgeexec-10-11 (unresponsive to ssh) [tools]
2023-02-20 §
11:24 <taavi> redeploy volume-admission with helm and cert-manager certificates T329530 T292238 [tools]
11:15 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/volume-admission:7fd13ac from https://gerrit.wikimedia.org/r/cloud/toolforge/volume-admission-controller (ede8bd0) - cookbook ran by taavi@runko [tools]
11:05 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/toolforge-volume-admission-controller:7fd13ac from https://gerrit.wikimedia.org/r/cloud/toolforge/volume-admission-controller (7fd13ac) - cookbook ran by taavi@runko [tools]