4051-4100 of 7208 results (31ms)
2021-07-26 §
17:37 <bstorm> repooled the whole set of ingress workers after upgrades T280340 [tools]
16:37 <bstorm> removing tools-k8s-ingress-4 from active ingress nodes at the proxy T280340 [tools]
2021-07-23 §
07:15 <majavah> restart nginx on tools-static-14 to see if it helps with fontcdn issues [tools]
2021-07-22 §
23:35 <bstorm> deleted tools-sgebastion-09 since it has been shut off since March anyway [tools]
15:32 <arturo> re-deploying toolforge-jobs-framework-api [tools]
15:30 <arturo> pushed new docker image on the registry for toolforge-jobs-framework-api 4d8235b879adbac9122a968b4335cf2bafee2b61 (T287077) [tools]
2021-07-21 §
20:01 <bstorm> deployed new maintain-kubeusers to toolforge T285011 [tools]
19:55 <bstorm> deployed new rbac for maintain-kubeusers changes T285011 [tools]
17:10 <majavah> deploying calico v3.18.4 T280342 [tools]
14:35 <majavah> updating systemd on toolforge stretch bastions T287036 [tools]
11:59 <arturo> deploying jobs-framework-api 07346d715d17585db9c16dd152cc91ef0bea33c3 (T286108) [tools]
11:04 <arturo> enabling TTLAfterFinished feature gate on kubeadm live configmap (T286108) [tools]
11:01 <arturo> enabling TTLAfterFinished feature gate on static pod manifests on /etc/kubernetes/manifests/kube-{apiserver,controller-manager}.yaml in all 3 control nodes (T286108) [tools]
2021-07-20 §
18:42 <majavah> deploying systemd security tools on toolforge public stretch machines T287004 [tools]
17:45 <arturo> pushed new toolforge-jobs-framework-api docker image into the registry (3a6ae38d51202c5c765c8d800cb8380e2a20b998) (T286126 [tools]
17:37 <arturo> added toolforge-jobs-framework-cli v3 to aptly buster-tools and buster-toolsbeta [tools]
13:25 <majavah> apply buster systemd security updates [tools]
2021-07-19 §
23:24 <bstorm> applied matchPolicy: equivalent to tools ingress validation controller T280360 [tools]
16:43 <bstorm> cleared queue error state caused by excessive resource use by topicmatcher T282474 [tools]
2021-07-16 §
14:04 <arturo> deployed jobs-framework-api 42b7a885a5bc1bf00c300e8d77bd92e1430a8327 (T286132) [tools]
11:57 <arturo> added toollabs-webservice_0.75_all to jessie-tools aptly repo (T286003) [tools]
11:52 <arturo> created `jessie-tools` aptly repository on tools-services-05 (T286003) [tools]
2021-07-15 §
16:12 <arturo> deploy toolforge-jobs-framework-api git version d85d93ee1c5d4be6a526cf83e806b2679dde3875 (T285944, T286107, T285979, T286485, T286107) [tools]
15:55 <arturo> added toolforge-jobs-framework-cli_2_all.deb to buster-{tools,toolsbeta} (T285944) [tools]
2021-07-14 §
23:29 <bstorm> mounted nfs on tools-services-05 and backing up aptly to NFS dir T286003 [tools]
09:17 <majavah> copying calico 3.18.4 images from docker hub to docker-registry.tools.wmflabs.org T280342 [tools]
2021-07-12 §
16:56 <bstorm> deleted job 4720371 due to LDAP failure [tools]
16:51 <bstorm> cleared the E state from two job queues [tools]
2021-07-02 §
18:46 <bstorm> cleared error state for tools-sgeexec-0940.tools.eqiad.wmflabs [tools]
2021-07-01 §
22:08 <bstorm> releasing webservice 0.75 [tools]
17:03 <andrewbogott> rebooting tools-k8s-worker-[31,33,35,44,49,51,57-58,70].tools.eqiad1.wikimedia.cloud [tools]
16:47 <bstorm> remounted scratch everywhere...but mostly tools T224747 [tools]
15:47 <arturo> rebased labs/private.git [tools]
11:04 <arturo> added toolforge-jobs-framework-cli_1_all.deb to aptly buster-tools,buster-toolsbeta [tools]
10:34 <arturo> refreshed jobs-api deployment [tools]
2021-06-29 §
21:58 <bstorm> clearing one errored queue and a stack of discarded jobs [tools]
20:11 <majavah> toolforge kubernetes upgrade complete T280299 [tools]
17:03 <majavah> starting toolforge kubernetes 1.18 upgrade - T280299 [tools]
16:17 <arturo> deployed jobs-framework-api in the k8s cluster [tools]
15:33 <majavah> remove duplicate definitions from tools-clushmaster-02 /root/.ssh/known_hosts [tools]
15:12 <arturo> livehacking puppetmaster for T283238 [tools]
10:24 <dcaro> running puppet on the buster bastions after 20000 minutes failing... might break something [tools]
2021-06-15 §
19:02 <bstorm> cleared error status from a few queues [tools]
16:15 <majavah> deleting unused shutdown nodes: tools-checker-03 tools-k8s-haproxy-1 tools-k8s-haproxy-2 [tools]
2021-06-14 §
22:21 <bstorm> push docker-registry.tools.wmflabs.org/toolforge-python37-sssd-web:testing to test staged os.execv (and other patches) using toolsbeta toollabs-webservice version 0.75 T282975 [tools]
2021-06-13 §
08:15 <majavah> clear grid error state from tools-sgeexec-0907, tools-sgeexec-0916, tools-sgeexec-0940 [tools]
2021-06-12 §
14:39 <majavah> remove nonexistent tools-prometheus-04 and add tools-prometheus-05 to hiera key "prometheus_nodes" [tools]
13:53 <majavah> create empty bullseye-{tools,toolsbeta} repositories on tools-services-05 aptly [tools]
2021-06-10 §
17:38 <majavah> clear error state from tools-sgeexec-0907, task@tools-sgeexec-0939 [tools]
2021-06-09 §
13:57 <majavah> clear error state from exec nodes tools-sgeexec-0913, tools-sgeexec-0936, task@tools-sgeexec-0940 [tools]