801-850 of 1174 results (26ms)
2021-02-26 §
20:39 <andrewbogott> rebooting all hosts [toolsbeta]
15:35 <dcaro> removed toolsbeta-test-k8s-etcd-9 with depool from kubeadmin/etcd (T274497) [toolsbeta]
11:46 <arturo> `openstack server create --os-project-id toolsbeta --image debian-10.0-buster --flavor g2.cores2.ram4.disk40 --network lan-flat-cloudinstances2b --property description='buster bastion test' toolsbeta-bastion-05` (T275865) [toolsbeta]
11:39 <arturo> created puppet prefix 'toolsbeta-bastion' to hold new configuration for buster-based bastions (T275865) [toolsbeta]
09:09 <dcaro> Playing around with cookbooks by adding/removing etcd nodes, etcd might missbehave from time to time (T274497) [toolsbeta]
2021-02-19 §
12:42 <arturo> deploying new version of the ingress admission controller [toolsbeta]
11:46 <arturo> merged https://gerrit.wikimedia.org/r/c/operations/puppet/+/662941 (T274139) which should only affect toolsbeta [toolsbeta]
10:27 <arturo> create DNS record `jobs.svc.toolsbeta.eqiad1.wikimedia.cloud` with CNAME to `k8s.toolsbeta.eqiad1.wikimedia.cloud` (T274139) [toolsbeta]
10:25 <arturo> create DNS zone `svc.toolsbeta.eqiad1.wikimedia.cloud` (T274139) [toolsbeta]
2021-02-10 §
12:34 <arturo> live-hacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/662941 (T274139) [toolsbeta]
12:23 <arturo> add `webserver` security group to toolsbeta-proxy-3 and -4 [toolsbeta]
12:20 <arturo> fix A record for `toolsbeta.wmflabs.org`, point it to 172.16.1.150 (toolsbeta-proxy-3), it was previously pointing to an old IP address [toolsbeta]
2021-02-08 §
11:48 <arturo> trying to introduce TLS support in the front proxy T274123 [toolsbeta]
2021-02-05 §
00:36 <bstorm> updated jobutils and miscutils to 1.40 in aptly for toolsbeta testing [toolsbeta]
2021-01-21 §
15:29 <bstorm> pushed the maintain-kubeusers:beta tag with the new code to the docker repo T271847 [toolsbeta]
2021-01-13 §
14:10 <dcaro> dcaro doing puppet tests, puppet runs might break [toolsbeta]
10:07 <arturo> allocate floating IP 185.15.56.84, and use it for docker-registry.toolsbeta.wmflabs.org (instance toolsbeta-docker-registry-01) (T271867) [toolsbeta]
10:05 <arturo> release and delete floating IP 185.15.56.242 (docker-registry.toolsbeta.wmflabs.org) (T271867) [toolsbeta]
2020-12-22 §
10:48 <arturo> rebase & resolve ugly git merge conflict in labs/private.git [toolsbeta]
2020-12-18 §
10:52 <arturo> live-hacking local puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/650470 (T267966) [toolsbeta]
2020-12-14 §
19:27 <bstorm> create temporary instance toolsbeta-test-io-unthrottled T267966 [toolsbeta]
19:25 <bstorm> created temporary instance toolsbeta-io-test-local T267966 [toolsbeta]
2020-12-11 §
23:31 <bstorm> increasing the output throttle for toolsbeta-test-k8s-haproxy-* nodes in order to figure out what's up with the timeouts [toolsbeta]
2020-12-10 §
08:58 <dcaro> starting a new etcd instance completely from ansible playbook (etcd-8) (T267412) [toolsbeta]
2020-12-09 §
15:30 <dcaro> Playing aronud adding a new etcd node (k8s-etcd-7) (T267412) [toolsbeta]
2020-12-04 §
11:17 <dcaro> Created a new 'standardized' security froup for k8s from ansible toolsbeta-k8s-full-connectivity (T267412) [toolsbeta]
10:12 <dcaro> Trying to create a whole new etcd member from ansible (T267412) [toolsbeta]
2020-11-23 §
14:17 <dcaro> All control nodes re-imaged (T267140) [toolsbeta]
14:08 <dcaro> Taking control-3 node out as control-6 is up and running (T267140) [toolsbeta]
11:12 <dcaro> Launching control-6, to replace control-3 (T267140) [toolsbeta]
10:45 <dcaro> Taking out control-2 node, replaced by control-5 (I saw one 503 reply on the proxy when creating control-5, fyi) (T267140) [toolsbeta]
10:32 <dcaro> Creating new control-5 node (will replace control-2) (T267140) [toolsbeta]
09:58 <dcaro> Remove control-1 node from the pool (was replaced by control-4) (T267140) [toolsbeta]
09:57 <dcaro> Remove control-1 node from the pool (was replaced by control-4) (T267195) [toolsbeta]
2020-11-18 §
11:46 <dcaro_> Modifying the security groupts to mirror tools (T267140) [toolsbeta]
10:50 <dcaro_> Adding new control-4 node to the control cluster (T267140) [toolsbeta]
2020-11-17 §
15:32 <dcaro> Creating new toolsbeta-test-k8s-control-4 node and adding it to the cluster (T267140) [toolsbeta]
12:09 <Lucas_WMDE> <dcaro> 11:59:36 UTC – toolbeta up and running again, documented on the live doc for now, apsrever had the wrong config (T267140) [toolsbeta]
10:40 <arturo> hand-edited /etc/kubernetes/manifests/kube-apiserver.yaml in all 3 k8s control nodes to account for new etcd servers (T267140) [toolsbeta]
08:58 <dcaro> etcd hosts reimaged (T267140) [toolsbeta]
08:54 <dcaro> etcd-4,5 and 6 are up and running, removing 1,2 and 3 (T267140) [toolsbeta]
2020-11-16 §
11:44 <dcaro> etcd5 member added, creating instance toolsbeta-test-k8s-etcd6 and adding to the etcd cluster (T267140) [toolsbeta]
11:27 <dcaro> Creating instance toolsbeta-test-k8s-etcd5 and adding to the etcd cluster (T267140) [toolsbeta]
2020-11-10 §
19:42 <bstorm> safelisted "argocd" namespace with namespaceSelector for registry-admission controller [toolsbeta]
18:49 <legoktm> associated floating IP to toolsbeta-docker-registry-01 and pointed DNS docker-registry.toolsbeta.wmflabs.org. at it [toolsbeta]
18:27 <legoktm> creating toolsbeta-docker-imagebuilder-01 (T267616) [toolsbeta]
17:18 <dcaro> launching instance toolsbeta-test-k8s-etcd-4 (T267140) [toolsbeta]
17:15 <dcaro> removing unused toolsbeta-k8s-etcd prefix (we use toolsbeta-test-k8s-etcd) (T267140) [toolsbeta]
14:44 <dcaro> taking down one of the test-k8s etcd nodes to reimage (T267140) [toolsbeta]
2020-11-06 §
23:44 <bstorm> toolsbeta k8s cluster fully upgraded to 1.17.13 T263284 [toolsbeta]