1-50 of 3563 results (22ms)
2022-10-09 §
17:29 <taavi> kill 10 idle tmux sessions of user 'hoi' on tools-sgebastion-10 T320352 [tools]
2022-10-07 §
13:02 <taavi> taavi@cloudcontrol1005 ~ $ sudo mark_tool --disable oncall # T320240 [tools]
2022-10-06 §
00:39 <bd808> Image rebuild failing with debian apt repo signature issue. Will investigate tomorrow. (T316554) [tools]
00:36 <bd808> Rebuilding all Toolforge docker images to pick up bug and security fix packages. (T316554) [tools]
00:04 <bd808> Building new php74-sssd-base & web images (T310435) [tools]
2022-10-03 §
14:36 <wm-bot2> build & push docker image docker-registry.tools.wmflabs.org/volume-admission:latest from https://gerrit.wikimedia.org/r/cloud/toolforge/volume-admission-controller (8da432b) - cookbook ran by taavi@runko [tools]
2022-09-28 §
21:23 <lucaswerkmeister> on tools-sgebastion-10: run-puppet-agent # T318858 [tools]
21:22 <lucaswerkmeister> on tools-sgebastion-10: apt remove emacs-common emacs-bin-common # fix package conflict, T318858 [tools]
21:15 <lucaswerkmeister> added root SSH key for myself, manually ran puppet on tools-sgebastion-10 to apply it (seemingly successfully) [tools]
2022-09-22 §
12:30 <taavi> add TheresNoTime to the 'toollabs-trusted' gerrit group T317438 [tools]
12:27 <taavi> add TheresNoTime as a project admin and to the roots sudo policy T317438 [tools]
2022-09-10 §
07:38 <wm-bot2> removing instance tools-prometheus-03 - cookbook ran by taavi@runko [tools]
2022-09-07 §
10:22 <dcaro> Pushing the new toolforge builder image based on the new 0.8 buildpacks (T316854) [tools]
2022-09-06 §
08:06 <dcaro_away> Published new toolforge-bullseye0-run and toolforge-bullseye0-build images for the toolforge buildpack builder (T316854) [tools]
2022-08-25 §
10:40 <taavi> tagged new version of the python39-web container with a shell implementation of webservice-runner T293552 [tools]
2022-08-24 §
12:20 <wm-bot2> deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/ingress-nginx (eba66bc) - cookbook ran by taavi@runko [tools]
12:20 <taavi> upgrading ingress-nginx to v1.3 [tools]
2022-08-20 §
07:44 <dcaro_away> all k8s nodes ready now \o/ (T315718) [tools]
07:43 <dcaro_away> rebooted tools-k8s-control-2, seemed stuck trying to wait for tools home (nfs?), after reboot came back up (T315718) [tools]
07:41 <dcaro_away> cloudvirt1023 down took out 3 workers, 1 control, and a grid exec and a weblight, they are taking long to restart, looking (T315718) [tools]
2022-08-18 §
14:45 <andrewbogott> adding lucaswerkmeister as projectadmin (T314527) [tools]
14:43 <andrewbogott> removing some inactive projectadmins: rush, petrb, mdipietro, jeh, krenair [tools]
2022-08-17 §
16:34 <taavi> kubectl sudo delete cm -n tool-wdml maintain-kubeusers # T315459 [tools]
08:30 <taavi> failing the grid from the shadow back to the master, some disruption expected [tools]
2022-08-16 §
17:28 <taavi> fail over docker-registry, tools-docker-registry-06->docker-registry-05 [tools]
2022-08-11 §
16:57 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by taavi@runko [tools]
16:55 <taavi> restart puppetdb on tools-puppetdb-1, crashed during the ceph issues [tools]
2022-08-05 §
15:08 <wm-bot2> removing grid node tools-sgewebgen-10-1.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
15:05 <wm-bot2> removing grid node tools-sgeexec-10-12.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
15:00 <wm-bot2> created node tools-sgewebgen-10-3.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by taavi@runko [tools]
2022-08-03 §
15:51 <dhinus> recreated jobs-api pods to pick up new ConfigMap [tools]
15:02 <wm-bot2> deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (c47ac41) - cookbook ran by fran@MacBook-Pro.station [tools]
2022-07-20 §
19:31 <taavi> reboot toolserver-proxy-01 to free up disk space probably held by stale file handles [tools]
08:06 <wm-bot2> removing grid node tools-sgeexec-10-6.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
2022-07-19 §
17:53 <wm-bot2> created node tools-sgeexec-10-21.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by taavi@runko [tools]
17:00 <wm-bot2> removing grid node tools-sgeexec-10-3.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
16:58 <wm-bot2> removing grid node tools-sgeexec-10-4.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
16:24 <wm-bot2> created node tools-sgeexec-10-20.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by taavi@runko [tools]
15:59 <taavi> tag current maintain-kubernetes :beta image as: :latest [tools]
2022-07-17 §
15:52 <wm-bot2> removing grid node tools-sgeexec-10-10.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
15:43 <wm-bot2> removing grid node tools-sgeexec-10-2.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
13:26 <wm-bot2> created node tools-sgeexec-10-16.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by taavi@runko [tools]
2022-07-14 §
13:48 <taavi> rebooting tools-sgeexec-10-2 [tools]
2022-07-13 §
12:09 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by dcaro@vulcanus [tools]
2022-07-11 §
16:06 <wm-bot2> Increased quotas by {self.increases} (T312692) - cookbook ran by nskaggs@x1carbon [tools]
2022-07-07 §
07:34 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master - cookbook ran by dcaro@vulcanus [tools]
2022-06-28 §
17:34 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master (T311538) - cookbook ran by dcaro@vulcanus [tools]
15:51 <taavi> add 4096G cinder quota T311509 [tools]
2022-06-27 §
18:14 <taavi> restart calico, appears to have got stuck after the ca replacement operation [tools]
18:02 <taavi> switchover active cron server to tools-sgecron-2 T284767 [tools]