251-300 of 3247 results (18ms)
2021-04-06 §
10:21 <arturo> aptly repo had some weirdness due to the cinder volume: hardlinks created by aptly were broken, solved with `sudo aptly publish --skip-signing repo stretch-tools -force-overwrite` [tools]
10:07 <dcaro> adding new etcd member using the cookbook wmcs.toolforge.add_etcd_node (T267082) [tools]
10:05 <arturo> installed aptly from buster-backports on tools-services-05 to see if that makes any difference with an issue when publishing repos [tools]
09:53 <dcaro> Removing etcd member tools-k8s-etcd-4.tools.eqiad.wmflabs (T267082) [tools]
08:55 <dcaro> adding new etcd member using the cookbook wmcs.toolforge.add_etcd_node (T267082) [tools]
2021-04-05 §
17:02 <bstorm> chowned the data volume for the docker registry to docker-registry:docker-registry [tools]
09:56 <arturo> make jhernandez (IRC joakino) projectadmin (T278975) [tools]
2021-04-01 §
20:43 <bstorm> cleared error state from the grid queues caused by unspecified job errors [tools]
15:53 <dcaro> Removed etcd member tools-k8s-etcd-5.tools.eqiad.wmflabs, adding a new member (T267082) [tools]
15:43 <dcaro> Removing etcd member tools-k8s-etcd-5.tools.eqiad.wmflabs (T267082) [tools]
15:36 <dcaro> Added new etcd member tools-k8s-etcd-9.tools.eqiad1.wikimedia.cloud (T267082) [tools]
15:18 <dcaro> adding new etcd member using the cookbook wmcs.toolforge.add_etcd_node (T267082) [tools]
2021-03-31 §
15:57 <arturo> rebooting `tools-mail-03` after enabling NFS (T267082, T278538) [tools]
15:57 <arturo> rebooting `tools-mail-03` after enabling NFS (T [tools]
15:04 <arturo> created MX record for `tools.wmcloud.org` pointing to `mail.tools.wmcloud.org` [tools]
15:03 <arturo> created DNS A record `mail.tools.wmcloud.org` pointing to 185.15.56.63 [tools]
14:56 <arturo> shutoff tools-mail-02 (T278538) [tools]
14:54 <arturo> point floating IP 185.15.56.63 to tools-mail-03 (T278538) [tools]
14:45 <arturo> created VM `tools-mail-03` as Debian Buster (T278538) [tools]
14:39 <arturo> relocate some of the hiera keys for email server from project-level to prefix [tools]
09:44 <dcaro> running disk performance test on etcd-4 (round2) [tools]
09:05 <dcaro> running disk performance test on etcd-8 [tools]
08:43 <dcaro> running disk performance test on etcd-4 [tools]
2021-03-30 §
16:15 <bstorm> added `labstore::traffic_shaping::egress: 800mbps` to tools-static prefix T278539 [tools]
15:44 <arturo> shutoff tools-static-12/13 (T278539) [tools]
15:41 <arturo> point horizon web proxy `tools-static.wmflabs.org` to tools-static-14 (T278539) [tools]
15:37 <arturo> add `mount_nfs: true` to tools-static prefix (T2778539) [tools]
15:26 <arturo> create VM tools-static-14 with Debian Buster image (T278539) [tools]
12:19 <arturo> introduce horizon proxy `deb-tools.wmcloud.org` (T278436) [tools]
12:15 <arturo> shutdown tools-sgebastion-09 (stretch) [tools]
11:05 <arturo> created VM `tools-sgebastion-10` as Debian Buster (T275865) [tools]
11:04 <arturo> created server group `tools-bastion` with anti-affinity policy [tools]
2021-03-28 §
19:31 <legoktm> legoktm@tools-sgebastion-08:~$ sudo qdel -f 9999704 # T278645 [tools]
2021-03-27 §
02:48 <Reedy> qdel -f 9999895 9999799 [tools]
2021-03-26 §
12:21 <arturo> shutdown tools-package-builder-02 (stretch), we keep -03 which is buster (T275864) [tools]
2021-03-25 §
19:30 <bstorm> forced deletion of all jobs stuck in a deleting state T277653 [tools]
17:46 <arturo> rebooting tools-sgeexec-* nodes to account for new grid master (T277653) [tools]
16:20 <arturo> rebuilding tools-sgegrid-master VM as debian buster (T277653) [tools]
16:18 <arturo> icinga-downtime toolschecker for 2h [tools]
16:05 <bstorm> failed over the tools grid to the shadow master T277653 [tools]
13:36 <arturo> shutdown tools-sge-services-03 (T278354) [tools]
13:33 <arturo> shutdown tools-sge-services-04 (T278354) [tools]
13:31 <arturo> point aptly clients to `tools-services-05.tools.eqiad1.wikimedia.cloud` (hiera change) (T278354) [tools]
12:58 <arturo> created VM `tools-services-05` as Debian Buster (T278354) [tools]
12:51 <arturo> create cinder volume `tools-aptly-data` (T278354) [tools]
2021-03-24 §
12:46 <arturo> shutoff the old stretch VMs `tools-docker-registry-03` and `tools-docker-registry-04` (T278303) [tools]
12:38 <arturo> associate floating IP 185.15.56.67 with `tools-docker-registry-05` and refresh FQDN docker-registry.tools.wmflabs.org accordingly (T278303) [tools]
12:33 <arturo> attach cinder volume `tools-docker-registry-data` to VM `tools-docker-registry-05` (T278303) [tools]
12:32 <arturo> snapshot cinder volume `tools-docker-registry-data` into `tools-docker-registry-data-stretch-migration` (T278303) [tools]
12:32 <arturo> bump cinder storage quota from 80G to 400G (without quota request task) [tools]