1-50 of 2236 results (9ms)
2019-11-15 §
14:44 <arturo> stop live-hacks on tools-prometheus-01 T237643 [tools]
2019-11-13 §
17:20 <arturo> live-hacking tools-prometheus-01 to test some experimental configs for the new k8s cluster (T237643) [tools]
2019-11-12 §
12:52 <arturo> reboot tools-proxy-06 to reset iptables setup T238058 [tools]
2019-11-10 §
02:17 <bd808> Building new Docker images for T237836 (retrying after cleaning out old images on tools-docker-builder-06) [tools]
02:15 <bd808> Cleaned up old images on tools-docker-builder-06 using instructions from https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Kubernetes#Building_toolforge_specific_images [tools]
02:10 <bd808> Building new Docker images for T237836 [tools]
01:45 <bstorm_> deploying bugfix for webservice in tools and toolsbeta T237836 [tools]
2019-11-08 §
22:47 <bstorm_> adding rsync::server::wrap_with_stunnel: false to the tools-docker-registry-03/4 servers to unbreak puppet [tools]
18:40 <bstorm_> pushed new webservice package to the bastions T230961 [tools]
18:37 <bstorm_> pushed new webservice package supporting buster containers to repo T230961 [tools]
18:36 <bstorm_> pushed buster-sssd images to the docker repo [tools]
17:15 <phamhi> pushed new buster images with the prefix name "toolforge" [tools]
2019-11-07 §
13:27 <arturo> deployed registry-admission-webhook and ingress-admission-controller into the new k8s cluster (T236826) [tools]
13:01 <arturo> creating puppet prefix `tools-k8s-worker` and a couple of VMs `tools-k8s-worker-[1,2]` T236826 [tools]
12:57 <arturo> increasing project quota T237633 [tools]
11:54 <arturo> point `k8s.tools.eqiad1.wikimedia.cloud` to tools-k8s-haproxy-1 T236826 [tools]
11:43 <arturo> create VMs `tools-k8s-haproxy-[1,2]` T236826 [tools]
11:43 <arturo> create puppet prefix `tools-k8s-haproxy` T236826 [tools]
2019-11-06 §
22:32 <bstorm_> added rsync::server::wrap_with_stunnel: false to tools-sge-services prefix to fix puppet [tools]
21:33 <bstorm_> docker images needed for kubernetes cluster upgrade deployed T215531 [tools]
20:26 <bstorm_> building and pushing docker images needed for kubernetes cluster upgrade [tools]
16:10 <arturo> new k8s cluster control nodes are bootstrapped (T236826) [tools]
13:51 <arturo> created FQDN `k8s.tools.eqiad1.wikimedia.cloud` pointing to `tools-k8s-control-1` for the initial bootstrap (T236826) [tools]
13:50 <arturo> created 3 VMs`tools-k8s-control-[1,2,3]` (T236826) [tools]
13:43 <arturo> created `tools-k8s-control` puppet prefix T236826 [tools]
11:57 <phamhi> restarted all webservices in grid (T233347) [tools]
2019-11-05 §
23:08 <Krenair> Dropped 59a77a3, 3830802, and 83df61f from tools-puppetmaster-01:/var/lib/git/labs/private cherry-picks as these are no longer required T206235 [tools]
22:49 <Krenair> Disassociated floating IP 185.15.56.60 from tools-static-13, traffic to this host goes via the project-proxy now. DNS was already changed a few days ago. T236952 [tools]
22:35 <bstorm_> upgraded libpython3.4 libpython3.4-dbg libpython3.4-minimal libpython3.4-stdlib python3.4 python3.4-dbg python3.4-minimal to fix an old broken patch T237468 [tools]
22:12 <bstorm_> pushed docker-registry.tools.wmflabs.org/maintain-kubeusers:beta to the registry to deploy in toolsbeta [tools]
17:38 <phamhi> restarted lighttpd based webservice pods on tools-worker-103x and 1040 (T233347) [tools]
17:34 <phamhi> restarted lighttpd based webservice pods on tools-worker-102[0-9] (T233347) [tools]
17:06 <phamhi> restarted lighttpd based webservice pods on tools-worker-101[0-9] (T233347) [tools]
16:44 <phamhi> restarted lighttpd based webservice pods on tools-worker-100[1-9] (T233347) [tools]
13:55 <arturo> created 3 new VMs: `tools-k8s-etcd-[4,5,6]` T236826 [tools]
2019-11-04 §
14:45 <phamhi> Built and pushed ruby25 docker image based on buster (T230961) [tools]
14:45 <phamhi> Built and pushed golang111 docker image based on buster (T230961) [tools]
14:45 <phamhi> Built and pushed jdk11 docker image based on buster (T230961) [tools]
14:45 <phamhi> Built and pushed php73 docker image based on buster (T230961) [tools]
11:10 <phamhi> Built and pushed python37 docker image based on buster (T230961) [tools]
2019-11-01 §
21:00 <Krenair> Removed tools-checker.wmflabs.org A record to 208.80.155.229 as that target IP is in the old pre-neutron range that is no longer routed [tools]
20:57 <Krenair> Removed trusty.tools.wmflabs.org CNAME to login-trusty.tools.wmflabs.org as that target record does not exist, presumably deleted ages ago [tools]
20:56 <Krenair> Removed tools-trusty.wmflabs.org CNAME to login-trusty.tools.wmflabs.org as that target record does not exist, presumably deleted ages ago [tools]
20:38 <Krenair> Updated A record for tools-static.wmflabs.org to point towards project-proxy T236952 [tools]
2019-10-31 §
18:46 <andrewbogott> deleted and/or truncated a bunch of logfiles on tools-worker-1001. Runaway logfiles filled up the drive which prevented puppet from running. If puppet had run, it would have prevented the runaway logfiles. [tools]
13:59 <arturo> update puppet prefix `tools-k8s-etcd-` to use the `role::wmcs::toolforge::k8s::etcd` T236826 [tools]
13:41 <arturo> disabling puppet in tools-k8s-etcd- nodes to test https://gerrit.wikimedia.org/r/c/operations/puppet/+/546995 [tools]
10:15 <arturo> SSL cert replacement for tools-docker-registry and tools-k8s-master went fine apparently (T236962) [tools]
10:02 <arturo> icinga downtime toolschecker for 1h for replacing SSL certs in tools-docker-registry and tools-k8s-master (T236962) [tools]
2019-10-30 §
13:53 <arturo> replacing SSL cert in tools-proxy-x server apparently OK (merged https://gerrit.wikimedia.org/r/c/operations/puppet/+/545679) T235252 [tools]