| 2025-04-24
      
      § | 
    
  | 23:09 | <bd808> | `systemctl stop sssd; rm -rf /var/lib/sss/db/*; systemctl restart sssd` on tools-bastion-12 | [tools] | 
            
  | 23:03 | <bd808> | `sss_cache -E` on tools-bastion-12 after seeing "sudo: PAM account management error: Authentication service cannot retrieve authentication info" | [tools] | 
            
  | 18:50 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-cli | [tools] | 
            
  | 18:39 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component envvars-cli | [tools] | 
            
  | 18:38 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-cli | [tools] | 
            
  | 18:33 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component envvars-cli | [tools] | 
            
  | 18:32 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-cli | [tools] | 
            
  | 18:25 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component envvars-cli | [tools] | 
            
  | 11:51 | <taavi> | add missing ICMPv6 security group rule to 'default' group | [tools] | 
            
  | 08:02 | <taavi> | add an AAAA record for toolserver.org T392506 | [tools] | 
            
  
    | 2025-04-23
      
      § | 
    
  | 19:21 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-58 | [tools] | 
            
  | 19:16 | <andrew@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-58 | [tools] | 
            
  | 15:56 | <taavi@cloudcumin1001> | END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-legacy-redirector-3.tools.eqiad1.wikimedia.cloud | [tools] | 
            
  | 15:55 | <taavi@cloudcumin1001> | START - Cookbook wmcs.vps.refresh_puppet_certs on tools-legacy-redirector-3.tools.eqiad1.wikimedia.cloud | [tools] | 
            
  | 15:10 | <arturo> | give `tools-tofu` bot account member powers for https://gitlab.wikimedia.org/repos/cloud/toolforge/tofu-provisioning | [tools] | 
            
  | 13:50 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api | [tools] | 
            
  | 13:36 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component builds-api | [tools] | 
            
  | 11:18 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api | [tools] | 
            
  | 11:05 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component jobs-api | [tools] | 
            
  | 07:02 | <taavi> | rebooting tools-mail-4 with stuck NFS handles | [tools] | 
            
  
    | 2025-04-16
      
      § | 
    
  | 19:45 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api | [tools] | 
            
  | 19:33 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component builds-api | [tools] | 
            
  | 19:30 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api | [tools] | 
            
  | 19:27 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component builds-api | [tools] | 
            
  | 15:00 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission | [tools] | 
            
  | 14:47 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component registry-admission | [tools] |