2022-08-16
§
|
17:41 |
<andrewbogott> |
removing cloudvirt1025 from the 'ceph' aggregate and adding it to the 'maintenance' aggregate |
[admin] |
17:40 |
<andrewbogott> |
reimaging cloudvirt1025 after I accidentally deleted the hw raid |
[admin] |
17:38 |
<andrewbogott> |
root@cloudcontrol1005:~# cinder-manage volume update_host --currenthost cloudcontrol1003@rbd#RBD --newhost cloudcontrol1005@rbd#RBD |
[admin] |
17:37 |
<andrewbogott> |
root@cloudcontrol1005:~# cinder-manage volume update_host --currenthost cloudcontrol1004@rbd#RBD --newhost cloudcontrol1006@rbd#RBD |
[admin] |
16:26 |
<wm-bot2> |
Ceph cluster at eqiad1 set out of maintenance. - cookbook ran by dcaro@vulcanus |
[admin] |
15:43 |
<wm-bot2> |
Restarting the osd daemons from nodes cloudcephosd1001,cloudcephosd1002,cloudcephosd1003,cloudcephosd1004,cloudcephosd1005,cloudcephosd1006,cloudcephosd1007,cloudcephosd1008,cloudcephosd1009,cloudcephosd1010,cloudcephosd1011,cloudcephosd1012,cloudcephosd1013,cloudcephosd1014,cloudcephosd1015,cloudcephosd1016,cloudcephosd1017,cloudcephosd1018,cloudcephosd1019,cloudcephosd1020,cloudcephosd1021,cloudcephosd1022,cloudcephosd1023,c |
[admin] |
15:42 |
<wm-bot2> |
Finished restarting all the OSD daemons from the nodes ['cloudcephosd2001-dev', 'cloudcephosd2002-dev', 'cloudcephosd2003-dev'] - cookbook ran by dcaro@vulcanus |
[admin] |
15:38 |
<wm-bot2> |
Restarting the osd daemons from nodes cloudcephosd2001-dev,cloudcephosd2002-dev,cloudcephosd2003-dev - cookbook ran by dcaro@vulcanus |
[admin] |
13:08 |
<wm-bot2> |
Restarting the osd daemons from nodes cloudcephosd2001-dev,cloudcephosd2002-dev,cloudcephosd2003-dev - cookbook ran by dcaro@vulcanus |
[admin] |
13:07 |
<wm-bot2> |
Restarting the osd daemons from nodes cloudcephosd2001-dev,cloudcephosd2002-dev,cloudcephosd2003-dev - cookbook ran by dcaro@vulcanus |
[admin] |
13:02 |
<wm-bot2> |
Restarting the osd daemons from nodes cloudcephosd2001-dev,cloudcephosd2002-dev,cloudcephosd2003-dev - cookbook ran by dcaro@vulcanus |
[admin] |
13:01 |
<wm-bot2> |
Restarting the osd daemons from nodes cloudcephosd2001-dev,cloudcephosd2002-dev,cloudcephosd2003-dev - cookbook ran by dcaro@vulcanus |
[admin] |
12:59 |
<wm-bot2> |
Restarting the osd daemons from nodes cloudcephosd2001-dev,cloudcephosd2002-dev,cloudcephosd2003-dev - cookbook ran by dcaro@vulcanus |
[admin] |
2022-08-11
§
|
13:57 |
<andrewbogott> |
decommissioning cloudcontrol1003 + cloudcontrl1004. I backed up $home in case anyone needs their files. |
[admin] |
08:42 |
<wm-bot2> |
The cluster is now rebalanced after adding the new OSDs ['cloudcephosd1025.eqiad.wmnet'] (T314870) - cookbook ran by fran@MacBook-Pro.station |
[admin] |
08:42 |
<wm-bot2> |
Added 1 new OSDs ['cloudcephosd1025.eqiad.wmnet'] (T314870) - cookbook ran by fran@MacBook-Pro.station |
[admin] |
08:42 |
<wm-bot2> |
Added OSD cloudcephosd1025.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@MacBook-Pro.station |
[admin] |
08:40 |
<wm-bot2> |
Finished rebooting node cloudcephosd1025.eqiad.wmnet (T314870) - cookbook ran by fran@MacBook-Pro.station |
[admin] |
08:36 |
<wm-bot2> |
Rebooting node cloudcephosd1025.eqiad.wmnet (T314870) - cookbook ran by fran@MacBook-Pro.station |
[admin] |
08:36 |
<wm-bot2> |
Adding OSD cloudcephosd1025.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@MacBook-Pro.station |
[admin] |
08:36 |
<wm-bot2> |
Adding new OSDs ['cloudcephosd1025.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@MacBook-Pro.station |
[admin] |
2022-07-20
§
|
18:02 |
<dcaro> |
things seem stable, trying to bring up a the last rabbit node, cloudcontrol1007 (T313400) |
[admin] |
17:45 |
<bd808> |
`sudo service striker restart` on labweb1002 |
[admin] |
17:43 |
<bd808> |
`sudo service striker restart` on labweb1001 |
[admin] |
17:10 |
<dcaro> |
things seem stable, trying to bring up a fourth rabbit node, cloudcontrol1006 (T313400) |
[admin] |
16:26 |
<dcaro> |
things seem stable, trying to bring up a third, cloudcontrol1005 (T313400) |
[admin] |
15:51 |
<dcaro> |
things seem stable now with one rabbit node, trying to bring up a second (T313400) |
[admin] |
14:16 |
<dcaro> |
stopping rabbin on cloudcontrol1004, leaving only 1003 alive (T313400) |
[admin] |
13:17 |
<dcaro> |
restarting the whole rabbit cluster (T313400) |
[admin] |