| 
      
        2019-04-10
      
      §
     | 
  
    
  | 23:58 | 
  <andrewbogott> | 
  moving tools-clushmaster-02, tools-elastic-03 and tools-paws-worker-1001 to eqiad1-r | 
  [tools] | 
            
  | 18:52 | 
  <bstorm_> | 
  depooled and rebooted tools-sgeexec-0929 because systemd was in a weird state | 
  [tools] | 
            
  | 18:46 | 
  <bstorm_> | 
  depooled and rebooted tools-sgewebgrid-lighttpd-0913 because high load was caused by ancient lsof processes | 
  [tools] | 
            
  | 14:49 | 
  <bstorm_> | 
  cleared E state from 5 queues | 
  [tools] | 
            
  | 13:06 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0906 | 
  [tools] | 
            
  | 12:31 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0926 | 
  [tools] | 
            
  | 12:27 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0925 | 
  [tools] | 
            
  | 12:06 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0901 | 
  [tools] | 
            
  | 11:55 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0924 | 
  [tools] | 
            
  | 11:47 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0921 | 
  [tools] | 
            
  | 11:23 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0940 | 
  [tools] | 
            
  | 11:03 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0928 | 
  [tools] | 
            
  | 10:49 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0923 | 
  [tools] | 
            
  | 10:43 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0915 | 
  [tools] | 
            
  | 10:27 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0935 | 
  [tools] | 
            
  | 10:19 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0914 | 
  [tools] | 
            
  | 10:01 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0907 | 
  [tools] | 
            
  | 09:40 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0918 | 
  [tools] | 
            
  | 09:27 | 
  <arturo> | 
  T218126 hard reboot tools-sgeexec-0932 | 
  [tools] | 
            
  | 09:26 | 
  <arturo> | 
  T218216 hard reboot tools-sgeexec-0932 | 
  [tools] | 
            
  | 09:04 | 
  <arturo> | 
  T218216 add `profile::ldap::client::labs::client_stack: sssd` to prefix puppet for sge-exec nodes | 
  [tools] | 
            
  | 09:03 | 
  <arturo> | 
  T218216 do a controlled rollover of sssd, depooling sgeexec nodes, reboot and repool | 
  [tools] | 
            
  | 08:39 | 
  <arturo> | 
  T218216 disable puppet in all tools-sgeexec-XXXX nodes for controlled sssd rollout | 
  [tools] | 
            
  | 00:32 | 
  <andrewbogott> | 
  migrating tools-worker-1022, 1023, 1025, 1026 to eqiad1-r | 
  [tools] |