801-850 of 1001 results (27ms)
2020-03-16 §
13:18 <arturo> live-hack toolsbeta-puppetmaster-02 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/578406 (T234617) [toolsbeta]
2020-03-11 §
21:32 <bstorm_> deployed jobutils_1.39 and miscutils_1.39 to toolsbeta [toolsbeta]
2020-03-09 §
13:10 <arturo> created VM `toolsbeta-legacy-redirector` (T247236) [toolsbeta]
13:08 <arturo> instance quota was full, bump it from 35 to 40 [toolsbeta]
2020-03-06 §
16:22 <bstorm_> updating maintain-kubeusers image to filter invalid tool names [toolsbeta]
2020-03-05 §
21:22 <bstorm_> updated maintain-kubeusers to the latest version for toolsbeta only to live test [toolsbeta]
2020-02-27 §
19:18 <bstorm_> upgraded toollabs-webservice to 0.64 on stretch-toolsbeta for testing [toolsbeta]
16:03 <jeh> create 3 new VMs toolsbeta-elastic7-0[1,2,3] [toolsbeta]
16:00 <jeh> increase CloudVPS quota instance count for new elasticsearch servers [toolsbeta]
2020-02-26 §
20:35 <bstorm_> hard rebooting the grid master for toolsbeta [toolsbeta]
20:20 <jeh> restart toolsbeta-sgegrid-shadow [toolsbeta]
2020-02-18 §
23:20 <bstorm_> added toolsbeta-sgegrid-master.toolsbeta.eqiad1.wikimedia.cloud and toolsbeta-sgegrid-shadow.toolsbeta.eqiad1.wikimedia.cloud to gridengine admin host lists [toolsbeta]
2020-02-10 §
21:18 <bstorm_> upgraded toollabs-webservice package for stretch toolsbeta to 0.62 T244293 T244289 T234617 T156626 [toolsbeta]
2020-02-07 §
23:07 <bstorm_> upgraded toollabs-webservice for stetch toolsbeta to 0.60 T244611 [toolsbeta]
21:09 <bstorm_> upgraded toollabs-webservice package for stretch toolsbeta to 0.59 T244293 T244289 T234617 T156626 [toolsbeta]
2020-01-23 §
03:14 <bd808> Demoted projectadmins not listed in the "roots" sudoer policy to project members just to avoid random confusion [toolsbeta]
03:06 <bd808> Added legoktm to "roots" sudoer policy [toolsbeta]
02:53 <bd808> Added legoktm as project admin [toolsbeta]
2020-01-22 §
11:59 <arturo> remove toolviews scripts from toolsbeta-proxy-{1,2}, source of cronspam [toolsbeta]
2020-01-21 §
12:49 <arturo> cleanup livehackings in toolsbeta-sgebastion-04 and toolsbeta-proxy-1 [toolsbeta]
09:40 <arturo> livehacking toolsbeta-sgebastion-04 (https://gerrit.wikimedia.org/r/c/566045 and https://gerrit.wikimedia.org/r/c/565575) and toolsbeta-proxy-1 (https://gerrit.wikimedia.org/r/c/565556) for testing T234617 [toolsbeta]
2020-01-17 §
12:51 <arturo> livehack toolsbeta-puppetmaster-02 to test https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/565556 (T234617) [toolsbeta]
10:37 <arturo> enabling puppet agent in toolsbeta-proxy-1 which was disabled without reason since 2019-12-02 (probably by me) [toolsbeta]
2020-01-16 §
23:13 <bstorm_> updated toollabs-webservice to 0.58 for stretch to test things out [toolsbeta]
12:07 <arturo> live-hack tools-webservice in tools-sgebastion-04 to test https://gerrit.wikimedia.org/r/c/565259 (T242719) [toolsbeta]
2020-01-14 §
02:15 <andrewbogott> rebooting toolsbeta-sgecron-01 and toolsbeta-test-k8s-etcd-3 to get nfs unstuch [toolsbeta]
2020-01-13 §
16:41 <bstorm_> There was a filesystem unclean and other problems on the "old cluster" worker node 1001. Rebooting it in case that helps. [toolsbeta]
2020-01-10 §
21:05 <bstorm_> updated toollabs-webservice package to 0.55 for testing [toolsbeta]
2020-01-07 §
15:51 <bstorm_> changed kubeadm-config to use a list instead of a hash for extravols on the apiserver in the new k8s cluster T242067 [toolsbeta]
2020-01-06 §
21:42 <bstorm_> disabled rpcbind on toolsbeta-sgebastion-04 to test some things [toolsbeta]
2020-01-03 §
17:46 <bstorm_> stashed uncommitted changes on the puppetmaster because they seem to be things that are already merged [toolsbeta]
11:27 <arturo> [new k8s] cadvisor is running in the metrics namespace now (T237643) [toolsbeta]
2020-01-02 §
22:37 <bstorm_> Deleting the massive number of test ingresses for tool-fourohfour so the ingress controllers aren't moving so slowly. [toolsbeta]
22:19 <bstorm_> Changed the ingress-admission ValidatingWebhookConfiguration to check extensions as well as networking API groups [toolsbeta]
2019-12-17 §
00:14 <bstorm_> Fully enabled encryption at rest for toolsbeta kubernetes [toolsbeta]
2019-12-16 §
23:03 <bstorm_> updated the kubeadm-config configmap to match the new init file [toolsbeta]
2019-12-04 §
13:02 <arturo> drop puppet prefix `toolsbeta-grid-master`, deprecated and no longer in use [toolsbeta]
12:50 <arturo> drop puppet prefix `toolsbeta-bastion`, deprecated and no longer in use [toolsbeta]
2019-12-02 §
10:38 <arturo> create wildcard DNS record for `*.toolsbeta.wmflabs.org` for use by the new k8s cluster [toolsbeta]
10:34 <arturo> manually scale nginx-ingress deployment to 5 replicas (T239405) [toolsbeta]
2019-11-25 §
10:30 <arturo> add puppet cert SANs via hiera to toolsbeta-test-k8s-etcd nodes (T238655) [toolsbeta]
2019-11-21 §
14:15 <arturo> upgrade new k8s cluster to 1.15.6 using kubeadm (plus kubelet) [toolsbeta]
2019-11-15 §
14:46 <arturo> stop live-hacks on toolsbeta-test-k8s-haproxy-1 T237643 [toolsbeta]
2019-11-14 §
10:32 <arturo> live-hacking toolsbeta-test-k8s-haproxy-1 to point to just the k8s apiserver in control-1 Turn on --v=10 in control-1 for extended debug [toolsbeta]
2019-11-08 §
19:36 <bstorm_> rebooted the proxy server just in case that fixes something. [toolsbeta]
11:58 <arturo> adding `profile::toolforge::bastion::nproc: 100` to puppet prefix `toolsbeta-sgebastion` (T236202) [toolsbeta]
11:38 <arturo> new k8s: refresh deployment for nginx-ingress with latest changes from puppet [toolsbeta]
2019-11-07 §
21:55 <bstorm_> killed pods for ingress admission controller to upgrade to new image T215531 [toolsbeta]
2019-11-06 §
22:39 <bstorm_> upgraded repo version of toollabs-webservice in toolsbeta-stretch to 0.49 -- changes for the new k8s cluster T215531 [toolsbeta]
19:09 <bstorm_> added profile::toolforge::proxies in global hiera to try and figure out why it won't let anything use redis T237443 [toolsbeta]