| 2025-08-19
      
      § | 
    
  | 14:45 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api | [tools] | 
            
  | 14:45 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component builds-api | [tools] | 
            
  | 14:44 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api | [tools] | 
            
  | 14:44 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component builds-api | [tools] | 
            
  | 14:42 | <raymond-ndibe@cloudcumin1001> | END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component jobs-api | [tools] | 
            
  | 14:37 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component jobs-api | [tools] | 
            
  | 14:37 | <dcaro> | flipped the tools-harbor.wmcloud.org endpoint to point to tools-harbor-2 (T350687) | [tools] | 
            
  | 14:22 | <Raymond_Ndibe> | setting tools-harbor-1 as read-only (T350687) | [tools] | 
            
  | 13:24 | <raymond-ndibe@cloudcumin1001> | END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component jobs-api | [tools] | 
            
  | 13:23 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component jobs-api | [tools] | 
            
  | 13:21 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api | [tools] | 
            
  | 13:19 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component jobs-api | [tools] | 
            
  | 13:18 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor | [tools] | 
            
  | 13:18 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor | [tools] | 
            
  | 13:18 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-harbor | [tools] | 
            
  | 13:18 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor | [tools] | 
            
  | 09:27 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api | [tools] | 
            
  | 09:13 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component jobs-api | [tools] | 
            
  
    | 2025-08-18
      
      § | 
    
  | 21:20 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api | [tools] | 
            
  | 21:07 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component jobs-api | [tools] | 
            
  | 17:49 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T362869) | [tools] | 
            
  | 17:49 | <dcaro@cloudcumin1001> | Updating container image docker-registry.svc.toolforge.org/kube-state-metrics:v2.16.0 (T362869) | [tools] | 
            
  | 17:49 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T362869) | [tools] | 
            
  | 17:48 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T362869) | [tools] | 
            
  | 17:48 | <dcaro@cloudcumin1001> | Updating container image docker-registry.svc.toolforge.org/metrics-server:v0.7.2 (T362869) | [tools] | 
            
  | 17:48 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T362869) | [tools] | 
            
  | 17:12 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-cli | [tools] | 
            
  | 17:03 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli | [tools] | 
            
  | 17:02 | <dcaro@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api | [tools] | 
            
  | 16:54 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-harbor-2.tools.eqiad1.wikimedia.cloud (T350687) | [tools] | 
            
  | 16:53 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.vps.refresh_puppet_certs on tools-harbor-2.tools.eqiad1.wikimedia.cloud (T350687) | [tools] | 
            
  | 16:50 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component jobs-api | [tools] | 
            
  | 08:35 | <wmbot~dcaro@acme> | END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-harbor-2.tools.eqiad1.wikimedia.cloud | [tools] | 
            
  | 08:34 | <wmbot~dcaro@acme> | START - Cookbook wmcs.vps.refresh_puppet_certs on tools-harbor-2.tools.eqiad1.wikimedia.cloud | [tools] | 
            
  
    | 2025-08-14
      
      § | 
    
  | 15:53 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-cli | [tools] | 
            
  | 15:44 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli | [tools] | 
            
  | 15:15 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api | [tools] | 
            
  | 15:03 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component jobs-api | [tools] | 
            
  | 11:38 | <wmbot~dcaro@acme> | END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot_stuck_workers (exit_code=99) for tools-k8s-worker-107, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-3, tools-k8s-worker-nfs-41 | [tools] | 
            
  | 11:38 | <wmbot~dcaro@acme> | START - Cookbook wmcs.toolforge.k8s.reboot_stuck_workers for tools-k8s-worker-107, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-3, tools-k8s-worker-nfs-41 | [tools] | 
            
  | 11:33 | <wmbot~dcaro@acme> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot_stuck_workers (exit_code=0) no stuck workers found | [tools] | 
            
  | 11:33 | <wmbot~dcaro@acme> | START - Cookbook wmcs.toolforge.k8s.reboot_stuck_workers no stuck workers found | [tools] |