| 
      
        2021-03-04
      
      §
     | 
  
    
  | 18:36 | 
  <andrewbogott> | 
  rebooting cloudmetrics1002; the console is hanging | 
  [admin] | 
            
  | 16:59 | 
  <arturo> | 
  rebooting cloudvirt1032 for T275753 | 
  [admin] | 
            
  | 16:34 | 
  <arturo> | 
  draining cloudvirt1032 for T275753 | 
  [admin] | 
            
  | 16:33 | 
  <arturo> | 
  rebooting cloudvirt1031 for T275753 | 
  [admin] | 
            
  | 16:11 | 
  <arturo> | 
  draining cloudvirt1031 for T275753 | 
  [admin] | 
            
  | 16:09 | 
  <arturo> | 
  rebooting cloudvirt1026 for T275753 | 
  [admin] | 
            
  | 15:57 | 
  <arturo> | 
  draining cloudvirt1026 for T275753 | 
  [admin] | 
            
  | 15:55 | 
  <arturo> | 
  rebooting cloudvirt1025 for T275753 | 
  [admin] | 
            
  | 15:41 | 
  <arturo> | 
  draining cloudvirt1025 for T275753 | 
  [admin] | 
            
  | 15:12 | 
  <arturo> | 
  rebooting cloudvirt1024 for T275753 | 
  [admin] | 
            
  | 11:29 | 
  <arturo> | 
  draining cloudvirt1024 for T275753 | 
  [admin] | 
            
  | 11:24 | 
  <dcaro> | 
  rebooted cloudvirt1022, re-adding to ceph and removing from maintenance host aggregate for T275753 | 
  [admin] | 
            
  | 11:01 | 
  <dcaro> | 
  rebooting cloudvirt1022 for T275753 | 
  [admin] | 
            
  | 09:12 | 
  <dcaro> | 
  draining cloudvirt1022 for T275753 | 
  [admin] | 
            
  
    | 
      
        2021-03-03
      
      §
     | 
  
    
  | 17:16 | 
  <andrewbogott> | 
  restarting rabbitmq-server on cloudcontrol1003,1004,1005; trying to explain amqp errors in scheduler logs | 
  [admin] | 
            
  | 16:03 | 
  <dcaro> | 
  draining cloudvirt1022 for T275753 | 
  [admin] | 
            
  | 16:03 | 
  <dcaro> | 
  draining cloudvirt1022 for TT275753 | 
  [admin] | 
            
  | 16:00 | 
  <arturo> | 
  move cloudvirt1013 into the 'toobusy' host aggregate, it has 221% cpu subscription and 82% MEM subscription | 
  [admin] | 
            
  | 15:34 | 
  <arturo> | 
  rebooting cloudvirt1021 for T275753 | 
  [admin] | 
            
  | 14:31 | 
  <arturo> | 
  draining cloudvirt1021 for T275753 | 
  [admin] | 
            
  | 13:59 | 
  <arturo> | 
  rebooting cloudvirt1018 for T275753 | 
  [admin] | 
            
  | 13:28 | 
  <arturo> | 
  draining cloudvirt1018 for T275753 | 
  [admin] | 
            
  | 12:49 | 
  <arturo> | 
  rebooting cloudvirt1017 for T275753 | 
  [admin] | 
            
  | 12:22 | 
  <arturo> | 
  draining cloudvirt1017 for T275753 | 
  [admin] | 
            
  | 12:20 | 
  <arturo> | 
  rebooting cloudvirt1016 for T275753 | 
  [admin] | 
            
  | 12:01 | 
  <arturo> | 
  draining cloudvirt1016 for T275753 | 
  [admin] | 
            
  | 11:59 | 
  <arturo> | 
  cloudvirt1014 now in the ceph host aggregate | 
  [admin] | 
            
  | 11:58 | 
  <arturo> | 
  rebooting cloudvirt1014 for T275753 | 
  [admin] | 
            
  | 11:50 | 
  <arturo> | 
  moved cloudvirt1023 away from the maintenance host aggregate, leave it in the ceph aggregate (was in the 2) | 
  [admin] | 
            
  | 11:47 | 
  <arturo> | 
  moved cloudvirt1014 to the 'maintenance' host aggregate, drain it for T275753 | 
  [admin] | 
            
  | 10:01 | 
  <arturo> | 
  icinga-downtime cloudnet1003 for 14 days bc potential alerting storm due to firmware issues (T271058) | 
  [admin] | 
            
  | 10:00 | 
  <arturo> | 
  rebooting again cloudnet1003 (no network failover) (T271058) | 
  [admin] | 
            
  | 09:58 | 
  <arturo> | 
  update firmware-bnx2x from 20190114-2 to 20200918-1~bpo10+1 on cloudnet1003 (T271058) | 
  [admin] | 
            
  | 09:30 | 
  <arturo> | 
  installing linux kernel 5.10.13-1~bpo10+1 in cloudnet1003 and rebooting it (network failover) (T271058) | 
  [admin] |