| 2021-04-06
      
      § | 
    
  | 15:16 | <bstorm> | cleared queue state since a few had "errored" for failed jobs. | [tools] | 
            
  | 12:59 | <dcaro> | Removing etcd member tools-k8s-etcd-7.tools.eqiad1.wikimedia.cloud to get an odd number  (T267082) | [tools] | 
            
  | 11:45 | <arturo> | upgrading jobutils & misctools to 1.42 everywhere | [tools] | 
            
  | 11:39 | <arturo> | cleaning up aptly: old package versions, old repos (jessie, trusty, precise) etc | [tools] | 
            
  | 10:31 | <dcaro> | Removing etcd member tools-k8s-etcd-6.tools.eqiad.wmflabs  (T267082) | [tools] | 
            
  | 10:21 | <arturo> | published jobutils & misctools 1.42 (T278748) | [tools] | 
            
  | 10:21 | <arturo> | published jobutils & misctools 1.42 | [tools] | 
            
  | 10:21 | <arturo> | aptly repo had some weirdness due to the cinder volume: hardlinks created by aptly were broken, solved with `sudo aptly publish --skip-signing repo stretch-tools -force-overwrite` | [tools] | 
            
  | 10:07 | <dcaro> | adding new etcd member using the cookbook wmcs.toolforge.add_etcd_node (T267082) | [tools] | 
            
  | 10:05 | <arturo> | installed aptly from buster-backports on tools-services-05 to see if that makes any difference with an issue when publishing repos | [tools] | 
            
  | 09:53 | <dcaro> | Removing etcd member tools-k8s-etcd-4.tools.eqiad.wmflabs  (T267082) | [tools] | 
            
  | 08:55 | <dcaro> | adding new etcd member using the cookbook wmcs.toolforge.add_etcd_node (T267082) | [tools] | 
            
  
    | 2021-03-31
      
      § | 
    
  | 15:57 | <arturo> | rebooting `tools-mail-03` after enabling NFS (T267082, T278538) | [tools] | 
            
  | 15:57 | <arturo> | rebooting `tools-mail-03` after enabling NFS (T | [tools] | 
            
  | 15:04 | <arturo> | created MX record for `tools.wmcloud.org` pointing to `mail.tools.wmcloud.org` | [tools] | 
            
  | 15:03 | <arturo> | created DNS A record `mail.tools.wmcloud.org` pointing to 185.15.56.63 | [tools] | 
            
  | 14:56 | <arturo> | shutoff tools-mail-02 (T278538) | [tools] | 
            
  | 14:54 | <arturo> | point floating IP 185.15.56.63 to tools-mail-03 (T278538) | [tools] | 
            
  | 14:45 | <arturo> | created VM `tools-mail-03` as Debian Buster (T278538) | [tools] | 
            
  | 14:39 | <arturo> | relocate some of the hiera keys for email server from project-level to prefix | [tools] | 
            
  | 09:44 | <dcaro> | running disk performance test on etcd-4 (round2) | [tools] | 
            
  | 09:05 | <dcaro> | running disk performance test on etcd-8 | [tools] | 
            
  | 08:43 | <dcaro> | running disk performance test on etcd-4 | [tools] | 
            
  
    | 2021-03-30
      
      § | 
    
  | 16:15 | <bstorm> | added `labstore::traffic_shaping::egress: 800mbps` to tools-static prefix T278539 | [tools] | 
            
  | 15:44 | <arturo> | shutoff tools-static-12/13 (T278539) | [tools] | 
            
  | 15:41 | <arturo> | point horizon web proxy `tools-static.wmflabs.org` to tools-static-14  (T278539) | [tools] | 
            
  | 15:37 | <arturo> | add `mount_nfs: true` to tools-static prefix (T2778539) | [tools] | 
            
  | 15:26 | <arturo> | create VM tools-static-14 with Debian Buster image (T278539) | [tools] | 
            
  | 12:19 | <arturo> | introduce horizon proxy `deb-tools.wmcloud.org` (T278436) | [tools] | 
            
  | 12:15 | <arturo> | shutdown tools-sgebastion-09 (stretch) | [tools] | 
            
  | 11:05 | <arturo> | created VM `tools-sgebastion-10` as Debian Buster (T275865) | [tools] | 
            
  | 11:04 | <arturo> | created server group `tools-bastion` with anti-affinity policy | [tools] |