351-400 of 3988 results (29ms)
2023-02-10 §
15:45 <taavi> reboot tools-k8s-worker-82 to troubleshoot network issues [tools]
12:44 <wm-bot2> Added a new k8s worker tools-k8s-worker-82.tools.eqiad1.wikimedia.cloud to the worker pool (T329357) - cookbook ran by taavi@runko [tools]
12:31 <wm-bot2> Adding a new k8s worker node (T329357) - cookbook ran by taavi@runko [tools]
12:29 <wm-bot2> Added a new k8s worker tools-k8s-worker-81.tools.eqiad1.wikimedia.cloud to the worker pool (T329357) - cookbook ran by taavi@runko [tools]
12:15 <wm-bot2> Adding a new k8s worker node (T329357) - cookbook ran by taavi@runko [tools]
11:53 <wm-bot2> Adding a new k8s worker node (T329357) - cookbook ran by taavi@runko [tools]
11:44 <wm-bot2> removing grid node tools-sgeweblight-10-23.tools.eqiad1.wikimedia.cloud (T329357) - cookbook ran by taavi@runko [tools]
11:42 <wm-bot2> removing grid node tools-sgeexec-10-5.tools.eqiad1.wikimedia.cloud (T329357) - cookbook ran by taavi@runko [tools]
11:39 <wm-bot2> removing grid node tools-sgeweblight-10-19.tools.eqiad1.wikimedia.cloud (T329357) - cookbook ran by taavi@runko [tools]
11:26 <wm-bot2> removing grid node tools-sgeweblight-10-12.tools.eqiad1.wikimedia.cloud (T329357) - cookbook ran by taavi@runko [tools]
11:24 <wm-bot2> removing grid node tools-sgeexec-10-1.tools.eqiad1.wikimedia.cloud (T329357) - cookbook ran by taavi@runko [tools]
2023-02-01 §
16:03 <taavi> deployed tools-webservice 0.89 [tools]
15:43 <wm-bot2> deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/image-config (372037f) - cookbook ran by taavi@runko [tools]
2023-01-26 §
15:05 <taavi> drain and reboot tools-k8s-worker-74 which seems to have some issues with nfs [tools]
14:37 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (307f302) - cookbook ran by taavi@runko [tools]
14:30 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/toolforge-jobs-framework-api:05966c6 from https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (05966c6) - cookbook ran by taavi@runko [tools]
2023-01-24 §
12:04 <taavi> deploying toolforge-jobs-framework-cli v10 T327775 [tools]
10:07 <taavi> publish toolforge-jobs-framework-cli v9 [tools]
2023-01-23 §
11:31 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (d5ae229) - cookbook ran by taavi@runko [tools]
11:23 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/toolforge-jobs-framework-api:d085c50 from https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (d085c50) - cookbook ran by taavi@runko [tools]
11:17 <wm-bot2> deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/image-config (864171a) - cookbook ran by taavi@runko [tools]
2023-01-20 §
23:24 <andrewbogott> truncating logfiles with find . -name '*.err' -size +1G -exec truncate --size=100M {} \; [tools]
21:24 <andrewbogott> truncating logfiles with find . -name '*.out' -size +1G -exec truncate --size=100M {} \; [tools]
01:06 <andrewbogott> truncating logfiles with find . -name '*.log' -size +1G -exec truncate --size=100M {} \; [tools]
2023-01-19 §
11:46 <arturo> `aborrero@tools-k8s-control-1:~$ sudo -i kubectl delete clusterrolebinding jobs-api-psp` (cleanup unused stuff) [tools]
2023-01-18 §
15:42 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (0ad4c66) - cookbook ran by arturo@nostromo [tools]
15:29 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/toolforge-jobs-framework-api:54cc15e from https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (54cc15e) - cookbook ran by arturo@nostromo [tools]
2023-01-17 §
13:55 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (8cf38a1) - cookbook ran by arturo@endurance [tools]
13:51 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (0d0a882) - cookbook ran by arturo@endurance [tools]
13:34 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/toolforge-jobs-framework-api:3a58c1d from https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (3a58c1d) - cookbook ran by arturo@endurance [tools]
2023-01-10 §
11:55 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (8e0a2f9) - cookbook ran by arturo@endurance [tools]
11:52 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/toolforge-jobs-framework-api:9514b00 from https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (8e0a2f9) - cookbook ran by arturo@endurance [tools]
11:36 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (0243967) - cookbook ran by arturo@endurance [tools]
2023-01-03 §
17:17 <andrewbogott> find -name '*.log' -size +1G -exec truncate --size=1G {} \; [tools]
2022-12-20 §
09:07 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by arturo@nostromo [tools]
2022-12-12 §
14:36 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by dcaro@vulcanus [tools]
2022-12-09 §
07:20 <taavi> change the canonical tools-mail external hostname to use mail.tools.wmcloud.org and add valid spf to toolforge.org T324809 [tools]
2022-12-05 §
11:06 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by dcaro@vulcanus [tools]
2022-11-30 §
10:39 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (bc3529d) - cookbook ran by arturo@nostromo [tools]
10:17 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/toolforge-jobs-framework-api:c360d54 from https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (c360d54) - cookbook ran by arturo@nostromo [tools]
2022-11-29 §
19:52 <taavi> clear puppet failure emails from exim queues [tools]
2022-11-09 §
08:58 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by arturo@nostromo [tools]
2022-11-05 §
19:28 <andrewbogott> cleaning up nfs share with root@labstore1004:/srv/tools/shared/tools# find -name '*.err' -size +1G -exec truncate --size=1G {} \; [tools]
13:26 <andrewbogott> cleaning up nfs share with root@labstore1004:/srv/tools/shared/tools# find -name '*.log' -size +1G -exec truncate --size=1G {} \; [tools]
2022-11-04 §
20:41 <andrewbogott> cleaning up nfs share with root@labstore1004:/srv/tools/shared/tools# find -name '*.err' -not -newermt "Nov 1, 2021" -exec rm {} \; [tools]
14:02 <andrewbogott> cleaning up nfs share with root@labstore1004:/srv/tools/shared/tools# find -name '*.log' -not -newermt "Nov 1, 2021" -exec rm {} \; [tools]
12:20 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (d464be4) (T304900) - cookbook ran by arturo@nostromo [tools]
12:12 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/toolforge-jobs-framework-api:2b800f5 from https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (2b800f5) (T304900) - cookbook ran by arturo@nostromo [tools]
2022-11-01 §
09:37 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master (T322110) - cookbook ran by dcaro@vulcanus [tools]
2022-10-26 §
08:45 <dcaro> depooling and rebooting tools-sgeexec-10-22 to get nfs scratch working again [tools]