| 
      
        2024-02-20
      
      §
     | 
  
    
  | 11:45 | 
  <aborrero@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) | 
  [admin] | 
            
  | 11:45 | 
  <aborrero@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance | 
  [admin] | 
            
  | 11:45 | 
  <aborrero@cloudcumin1001> | 
  END (FAIL) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=99) | 
  [admin] | 
            
  | 11:45 | 
  <aborrero@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance | 
  [admin] | 
            
  | 11:30 | 
  <aborrero@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.post-reimage (exit_code=0) preparing cloudvirt cloudvirt1032.eqiad.wmnet for duty (nova discovery, canary VM) Pending aggregates though. (T319184) | 
  [admin] | 
            
  | 11:30 | 
  <aborrero@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.post-reimage preparing cloudvirt cloudvirt1032.eqiad.wmnet for duty (nova discovery, canary VM) Pending aggregates though. (T319184) | 
  [admin] | 
            
  
    | 
      
        2024-02-19
      
      §
     | 
  
    
  | 12:33 | 
  <aborrero@cloudcumin1001> | 
  END (FAIL) - Cookbook wmcs.openstack.cloudvirt.post-reimage (exit_code=99) preparing cloudvirt cloudvirt1032.eqiad.wmnet for duty (nova discovery, canary VM) Pending aggregates though. (T319184) | 
  [admin] | 
            
  | 12:32 | 
  <aborrero@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.post-reimage preparing cloudvirt cloudvirt1032.eqiad.wmnet for duty (nova discovery, canary VM) Pending aggregates though. (T319184) | 
  [admin] | 
            
  | 12:02 | 
  <aborrero@cloudcumin1001> | 
  END (FAIL) - Cookbook wmcs.openstack.cloudvirt.post-reimage (exit_code=99) preparing cloudvirt cloudvirt1032.eqiad.wmnet for duty (nova discovery, canary VM) Pending aggregates though. (T319184) | 
  [admin] | 
            
  | 12:02 | 
  <aborrero@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.post-reimage preparing cloudvirt cloudvirt1032.eqiad.wmnet for duty (nova discovery, canary VM) Pending aggregates though. (T319184) | 
  [admin] | 
            
  | 12:00 | 
  <aborrero@cloudcumin1001> | 
  END (FAIL) - Cookbook wmcs.openstack.cloudvirt.post-reimage (exit_code=99) preparing cloudvirt cloudvirt1032.eqiad.wmnet for duty (nova discovery, canary VM) Pending aggregates though. (T319184) | 
  [admin] | 
            
  | 12:00 | 
  <aborrero@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.post-reimage preparing cloudvirt cloudvirt1032.eqiad.wmnet for duty (nova discovery, canary VM) Pending aggregates though. (T319184) | 
  [admin] | 
            
  | 10:09 | 
  <aborrero@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.pre-reimage (exit_code=0) prepare cloudvirt1032.eqiad.wmnet for reimage (drain, remove nova agent, etc) (T319184) | 
  [admin] | 
            
  | 09:52 | 
  <aborrero@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.pre-reimage prepare cloudvirt1032.eqiad.wmnet for reimage (drain, remove nova agent, etc) (T319184) | 
  [admin] | 
            
  | 09:49 | 
  <aborrero@cloudcumin1001> | 
  END (FAIL) - Cookbook wmcs.openstack.cloudvirt.pre-reimage (exit_code=99) prepare cloudvirt1032.eqiad.wmnet for reimage (drain, remove nova agent, etc) (T319184) | 
  [admin] | 
            
  | 09:49 | 
  <aborrero@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.pre-reimage prepare cloudvirt1032.eqiad.wmnet for reimage (drain, remove nova agent, etc) (T319184) | 
  [admin] | 
            
  
    | 
      
        2024-02-15
      
      §
     | 
  
    
  | 17:34 | 
  <wmbot~fran@wmf3169> | 
  START - Cookbook wmcs.openstack.roll_reboot_cloudnets (T356975) | 
  [admin] | 
            
  | 15:34 | 
  <taavi> | 
  restart radosgw in eqiad as I am seeing 500 errors | 
  [admin] | 
            
  | 14:22 | 
  <taavi@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) | 
  [admin] | 
            
  | 14:22 | 
  <taavi@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance | 
  [admin] | 
            
  | 14:22 | 
  <taavi@cloudcumin1001> | 
  END (FAIL) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=99) | 
  [admin] | 
            
  | 14:22 | 
  <taavi@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance | 
  [admin] | 
            
  | 12:05 | 
  <aborrero@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) on host 'cloudvirt1031.eqiad.wmnet' (T319184) | 
  [admin] | 
            
  | 11:58 | 
  <dhinus> | 
  restore correct aggregate "localdisk" for cloudvirtlocal1001 and remove "maintenance" | 
  [admin] | 
            
  | 11:52 | 
  <aborrero@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.drain on host 'cloudvirt1031.eqiad.wmnet' (T319184) | 
  [admin] | 
            
  | 05:51 | 
  <andrew@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1044.eqiad.wmnet}' | 
  [admin] | 
            
  | 05:41 | 
  <andrew@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1045.eqiad.wmnet}' | 
  [admin] | 
            
  | 05:19 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1044.eqiad.wmnet}' | 
  [admin] | 
            
  | 05:18 | 
  <andrew@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1047.eqiad.wmnet}' | 
  [admin] | 
            
  | 05:00 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1046.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:52 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1047.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:52 | 
  <andrew@cloudcumin1001> | 
  END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) on hosts matched by 'D{cloudvirt1046.eqiad.wmnet[B}' | 
  [admin] | 
            
  | 04:51 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1046.eqiad.wmnet[B}' | 
  [admin] | 
            
  | 04:50 | 
  <andrew@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1049.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:48 | 
  <andrew@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1048.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:45 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1049.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:44 | 
  <andrew@cloudcumin1001> | 
  END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) on hosts matched by 'D{cloudvirt1049.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:33 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1049.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:33 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1048.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:32 | 
  <andrew@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1051.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:30 | 
  <andrew@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1050.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:10 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1051.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:10 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1050.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:09 | 
  <andrew@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1052.eqiad.wmnet}' | 
  [admin] | 
            
  | 04:03 | 
  <andrew@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1053.eqiad.wmnet}' | 
  [admin] | 
            
  | 03:47 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1052.eqiad.wmnet}' | 
  [admin] | 
            
  | 03:47 | 
  <andrew@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1054.eqiad.wmnet}' | 
  [admin] | 
            
  | 03:43 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1054.eqiad.wmnet}' | 
  [admin] | 
            
  | 03:39 | 
  <andrew@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1053.eqiad.wmnet}' | 
  [admin] | 
            
  | 03:30 | 
  <andrew@cloudcumin1001> | 
  END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) on hosts matched by 'D{cloudvirt1054.eqiad.wmnet}' | 
  [admin] |