1151-1200 of 1486 results (24ms)
2020-11-17 §
10:40 <arturo> hand-edited /etc/kubernetes/manifests/kube-apiserver.yaml in all 3 k8s control nodes to account for new etcd servers (T267140) [toolsbeta]
08:58 <dcaro> etcd hosts reimaged (T267140) [toolsbeta]
08:54 <dcaro> etcd-4,5 and 6 are up and running, removing 1,2 and 3 (T267140) [toolsbeta]
2020-11-16 §
11:44 <dcaro> etcd5 member added, creating instance toolsbeta-test-k8s-etcd6 and adding to the etcd cluster (T267140) [toolsbeta]
11:27 <dcaro> Creating instance toolsbeta-test-k8s-etcd5 and adding to the etcd cluster (T267140) [toolsbeta]
2020-11-10 §
19:42 <bstorm> safelisted "argocd" namespace with namespaceSelector for registry-admission controller [toolsbeta]
18:49 <legoktm> associated floating IP to toolsbeta-docker-registry-01 and pointed DNS docker-registry.toolsbeta.wmflabs.org. at it [toolsbeta]
18:27 <legoktm> creating toolsbeta-docker-imagebuilder-01 (T267616) [toolsbeta]
17:18 <dcaro> launching instance toolsbeta-test-k8s-etcd-4 (T267140) [toolsbeta]
17:15 <dcaro> removing unused toolsbeta-k8s-etcd prefix (we use toolsbeta-test-k8s-etcd) (T267140) [toolsbeta]
14:44 <dcaro> taking down one of the test-k8s etcd nodes to reimage (T267140) [toolsbeta]
2020-11-06 §
23:44 <bstorm> toolsbeta k8s cluster fully upgraded to 1.17.13 T263284 [toolsbeta]
21:23 <bstorm> upgrading toolsbeta-test-k8s-control-1 to k8s 1.17.13 T263284 [toolsbeta]
15:56 <dcaro> Deleting instances proxy-1 and proxy-2, that will finish the proxy rebuild (T267140) [toolsbeta]
15:53 <dcaro> Removing proxy-1 and proxy-3 from hiera, proxy-3 stays as active and proxy-4 as backup (T267140) [toolsbeta]
13:18 <dcaro> bringin up a new proxy-4 instance as slave (T267140) [toolsbeta]
13:18 <dcaro> bringin up a new proxy-4 instance as slave [toolsbeta]
2020-11-05 §
16:40 <dcaro> Moving active proxy from proxy-1 to proxy-3 (T267140) [toolsbeta]
15:53 <dcaro> Adding toolsbeta-proxy-3 to the list of slave proxies in hiera (T267140) [toolsbeta]
2020-11-04 §
15:42 <dcaro> re-creating the toolsbeta-proxy-03, used wrong image on the first try (T267140) [toolsbeta]
15:21 <dcaro> creating new proxy instance toolsbeta-proxy-03 [toolsbeta]
15:18 <arturo> dropping project hiera config for `toollabs::checker_hosts`, `toollabs::proxy::ssl_certificate_name`, `toollabs::proxy::ssl_install_certificate` and `toollabs::proxy::web_domain`, no longer in use [toolsbeta]
15:15 <arturo> dropping project hiera config for `toollabs::proxy::proxies`, no longer in use [toolsbeta]
11:45 <dcaro> The k8s scheduler-01 fails to connect to etcd (not sure ever did), trying to fix [toolsbeta]
2020-11-03 §
16:04 <arturo> add dcaro to the toolsbeta.admin LDAP group (T266068) [toolsbeta]
15:30 <dcaro> T267121: Puppetmaster replaced, also removed old puppetdb master from hiera, testing [toolsbeta]
15:07 <dcaro> Replacing old puppetmaster 02 and 03 from hiera with 04 [toolsbeta]
10:55 <dcaro> dcaro investigating puppet errors on toolsbeta-puppetdb-02 [toolsbeta]
2020-11-02 §
13:35 <arturo> added dcaro as projectadmin & user (T266068) [toolsbeta]
2020-10-29 §
22:20 <legoktm> switched test tool over to use buildpack image (T265681) [toolsbeta]
2020-10-28 §
18:58 <andrewbogott> deleting toolsbeta-puppetmaster-03 — seems broken and unused [toolsbeta]
2020-10-22 §
16:22 <bstorm> created buildpack psp for T265557 [toolsbeta]
2020-09-10 §
09:17 <arturo> force-rebooting toolsbeta-test-haproxy-2 (unresponsive) [toolsbeta]
09:15 <arturo> livehacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/626133 (T250172) [toolsbeta]
09:00 <arturo> tainted/labeld toolsbeta-test-k8s-ingress-1 (and -2) in the k8s cluster (T250172) [toolsbeta]
08:59 <arturo> added toolsbeta-test-k8s-ingress-1 (and -2) to the k8s cluster (T250172) [toolsbeta]
2020-09-09 §
11:50 <arturo> after force-rebooting everything, the k8s cluster seems to have recovered itself. magic. [toolsbeta]
11:45 <arturo> force-rebooting the 3 k8s etcd nodes. They seem down [toolsbeta]
11:42 <arturo> actually, the whole k8s cluster seems down? the API seems down at least [toolsbeta]
11:39 <arturo> all 3 k8s control nodes seem in bad shape. Wont let me ssh in, or use the console access. Try force-rebooting them [toolsbeta]
11:27 <arturo> created 2 VMs: toolsbeta-test-k8s-ingress-1 and toolsbeta-test-k8s-ingress-2 (T250172) [toolsbeta]
11:25 <arturo> created new server group toolsbeta-k8s-ingress (T250172) [toolsbeta]
11:24 <arturo> created new puppet prefix `toolsbeta-test-k8s-ingress` (T250172) [toolsbeta]
2020-07-15 §
21:35 <bstorm> set all of toolsbeta to mount NFS 4.2 except the bastion T257945 [toolsbeta]
2020-07-14 §
22:28 <bstorm> rebooting toolsbeta-sgebastion-04 during NFS testing thing [toolsbeta]
2020-07-08 §
11:08 <arturo> live-hacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/610029 (T234617) [toolsbeta]
2020-06-26 §
12:12 <arturo> puppetmaster live-hacking with https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/608005 (T120210) [toolsbeta]
2020-06-24 §
12:55 <arturo> live-hacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/607279 (T120225) [toolsbeta]
12:23 <arturo> live-hacking puppetmaster with exim prometheus stuff (T175964) [toolsbeta]
11:31 <arturo> live-hack the puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/607320 (T175964) [toolsbeta]