301-350 of 2923 results (19ms)
2020-05-20 §
09:23 <arturo> `aborrero@cloud-cumin-01:~$ sudo cumin --force -x 'O{project:tools name:tools-sge[bcew].*}' 'rm /etc/apt/preferences.d/* ; run-puppet-agent'` (T247422) [tools]
2020-05-19 §
17:00 <bstorm_> deleting/restarting the paws db-proxy pod because it cannot connect to the replicas...and I'm hoping that's due to depooling and such [tools]
2020-05-13 §
18:14 <bstorm_> upgrading calico to 3.14.0 with typha enabled in Toolforge K8s T250863 [tools]
18:10 <bstorm_> set "profile::toolforge::k8s::typha_enabled: true" in tools project for calico upgrade T250863 [tools]
2020-05-09 §
00:28 <bstorm_> added nfs.* to ignored_fs_types for the prometheus::node_exporter params in project hiera T252260 [tools]
2020-05-08 §
18:17 <bd808> Building all jessie-sssd derived images (T197930) [tools]
17:29 <bd808> Building new jessie-sssd base image (T197930) [tools]
2020-05-07 §
21:51 <bstorm_> rebuilding the docker images for Toolforge k8s [tools]
19:03 <bstorm_> toollabs-webservice 0.69 now pushed to the Toolforge bastions [tools]
18:57 <bstorm_> pushing new toollabs-webservice package v0.69 to the tools repos [tools]
2020-05-06 §
21:20 <bd808> Kubectl delete node tools-k8s-worker-[16-20] (T248702) [tools]
18:24 <bd808> Updated "profile::toolforge::k8s::worker_nodes" list in "tools-k8s-haproxy" prefix puppet (T248702) [tools]
18:14 <bd808> Shutdown tools-k8s-worker-[16-20] instances (T248702) [tools]
18:04 <bd808> Draining tools-k8s-worker-[16-20] in preparation for decomm (T248702) [tools]
17:56 <bd808> Cordoned tools-k8s-worker-[16-20] in preparation for decomm (T248702) [tools]
00:01 <bd808> Joining tools-k8s-worker-60 to the k8s worker pool [tools]
00:00 <bd808> Joining tools-k8s-worker-59 to the k8s worker pool [tools]
2020-05-05 §
23:58 <bd808> Joining tools-k8s-worker-58 to the k8s worker pool [tools]
23:55 <bd808> Joining tools-k8s-worker-57 to the k8s worker pool [tools]
23:53 <bd808> Joining tools-k8s-worker-56 to the k8s worker pool [tools]
21:51 <bd808> Building 5 new k8s worker nodes (T248702) [tools]
2020-05-04 §
22:08 <bstorm_> deleting tools-elastic-01/2/3 T236606 [tools]
16:46 <arturo> removing the now unused `/etc/apt/preferences.d/toolforge_k8s_kubeadmrepo*` files (T250866) [tools]
16:43 <arturo> removing the now unused `/etc/apt/sources.list.d/toolforge-k8s-kubeadmrepo.list` file (T250866) [tools]
2020-04-29 §
22:13 <bstorm_> running a fixup script after fixing a bug T247455 [tools]
21:28 <bstorm_> running the rewrite-psp-preset.sh script across all tools T247455 [tools]
16:54 <bstorm_> deleted the maintain-kubeusers pod to start running the new image T247455 [tools]
16:52 <bstorm_> tagged docker-registry.tools.wmflabs.org/maintain-kubeusers:beta to latest to deploy to toolforge T247455 [tools]
2020-04-28 §
22:58 <bstorm_> rebuilding docker-registry.tools.wmflabs.org/maintain-kubeusers:beta T247455 [tools]
2020-04-23 §
19:22 <bd808> Increased Kubernetes services quota for bd808-test tool. [tools]
2020-04-21 §
23:06 <bstorm_> repooled tools-k8s-worker-38/52, tools-sgewebgrid-lighttpd-0918/9 and tools-sgeexec-0901 T250869 [tools]
22:09 <bstorm_> depooling tools-sgewebgrid-lighttpd-0918/9 and tools-sgeexec-0901 T250869 [tools]
22:02 <bstorm_> draining tools-k8s-worker-38 and tools-k8s-worker-52 as they are on the crashed host T250869 [tools]
2020-04-20 §
15:31 <bd808> Rebuilding Docker containers to pick up tools-webservice v0.68 (T250625) [tools]
14:47 <arturo> added joakino to tools.admin LDAP group [tools]
13:28 <jeh> shutdown elasticsearch v5 cluster running Jessie T236606 [tools]
12:46 <arturo> uploading tools-webservice v0.68 to aptly stretch-tools and update it on relevant servers (T250625) [tools]
12:06 <arturo> uploaded tools-webservice v0.68 to stretch-toolsbeta for testing [tools]
11:59 <arturo> `root@tools-sge-services-03:~# aptly db cleanup` removed 340 unreferenced packages, and 2 unreferenced files [tools]
2020-04-15 §
23:20 <bd808> Building ruby25-sssd/base and children (T141388, T250118) [tools]
20:09 <jeh> update default security group to allow prometheus01.metricsinfra.eqiad.wmflabs TCP 9100 T250206 [tools]
2020-04-14 §
18:26 <bstorm_> Deployed new code and RBAC for maintain-kubeusers T246123 [tools]
18:19 <bstorm_> updating the maintain-kubeusers:latest image T246123 [tools]
17:32 <bstorm_> updating the maintain-kubeusers:beta image on tools-docker-imagebuilder-01 T246123 [tools]
2020-04-10 §
21:33 <bd808> Rebuilding all Docker images for the Kubernetes cluster (T249843) [tools]
21:27 <bd808> Rebuilding all Docker images for the Kubernetes cluster (T249843) [tools]
19:36 <bstorm_> after testing deploying toollabs-webservice 0.67 to tools repos T249843 [tools]
14:53 <arturo> live-hacking tools-puppetmaster-02 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/587991 for T249837 [tools]
2020-04-09 §
15:13 <bd808> Rebuilding all stretch and buster Docker images. Jessie is broken at the moment due to package version mismatches [tools]
11:18 <arturo> bump nproc limit in bastions https://gerrit.wikimedia.org/r/c/operations/puppet/+/587715 (T219070) [tools]