2651-2700 of 4617 results (26ms)
2022-06-30 §
07:45 <wm-bot2> Rebooted cloudnet host cloudnet2005-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
07:40 <wm-bot2> Rebooting cloudnet host cloudnet2005-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
06:52 <wm-bot2> Rebooting cloudnet host cloudnet2005-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
2022-06-29 §
08:45 <wm-bot2> Finished rebooting node cloudcephosd1021.eqiad.wmnet - cookbook ran by dcaro@vulcanus [admin]
08:41 <wm-bot2> Rebooting node cloudcephosd1021.eqiad.wmnet - cookbook ran by dcaro@vulcanus [admin]
2022-06-28 §
13:03 <taavi> grant the tools project access to the g3.cores16.ram64.disk20.10xiops flavor T301949 [admin]
2022-06-22 §
16:48 <taavi> restart designate-*.service on both cloudservices nodes [admin]
16:44 <taavi> restart nova-conductor on all the cloudcontrol nodes [admin]
12:50 <andrewbogott> rebooting each eqiad1 cloudcontrol node in hopes of getting a baseline re: openstack instability [admin]
2022-06-21 §
04:48 <andrewbogott> stopping nova-fullstack agent on cloudcontrol1003; it's going to page us otherwise and we're all AFK tomorrow [admin]
04:02 <andrewbogott> restarting rabbitmq on cloudcontrol100x (one at a time) [admin]
2022-06-17 §
17:53 <andrewbogott> switching to a new python-based health check for galera and haproxy. This may make things more stable, or it may not. T310664 [admin]
06:14 <taavi> restart neutron-linuxbridge-agent on cloudvirt1046 [admin]
2022-06-15 §
11:34 <taavi> restart neutron-linuxbridge-agent on cloudvirt1022 [admin]
2022-06-14 §
16:26 <wm-bot2> OSDs (['cloudcephosd1001', 'cloudcephosd1002', 'cloudcephosd1003', 'cloudcephosd1004', 'cloudcephosd1005', 'cloudcephosd1006', 'cloudcephosd1007', 'cloudcephosd1008', 'cloudcephosd1009', 'cloudcephosd1010', 'cloudcephosd1011', 'cloudcephosd1012', 'cloudcephosd1013', 'cloudcephosd1014', 'cloudcephosd1015', 'cloudcephosd1016', 'cloudcephosd1017', 'cloudcephosd1018', 'cloudcephosd1019', 'cloudcephosd1020', 'cloudcephosd1021', 'cl [admin]
14:38 <wm-bot2> Upgrading OSDs and rebooting the nodes ['cloudcephosd1001', 'cloudcephosd1002', 'cloudcephosd1003', 'cloudcephosd1004', 'cloudcephosd1005', 'cloudcephosd1006', 'cloudcephosd1007', 'cloudcephosd1008', 'cloudcephosd1009', 'cloudcephosd1010', 'cloudcephosd1011', 'cloudcephosd1012', 'cloudcephosd1013', 'cloudcephosd1014', 'cloudcephosd1015', 'cloudcephosd1016', 'cloudcephosd1017', 'cloudcephosd1018', 'cloudcephosd1019', 'cloudceph [admin]
12:55 <wm-bot2> OSDs (['cloudcephosd2001-dev', 'cloudcephosd2002-dev', 'cloudcephosd2003-dev']) upgraded successfully B-) (T309786) - cookbook ran by dcaro@vulcanus [admin]
12:44 <wm-bot2> Upgrading OSDs and rebooting the nodes ['cloudcephosd2001-dev', 'cloudcephosd2002-dev', 'cloudcephosd2003-dev'] (T309786) - cookbook ran by dcaro@vulcanus [admin]
2022-06-13 §
11:14 <wm-bot2> Finished rebooting node cloudcephosd1021.eqiad.wmnet (T309789) - cookbook ran by dcaro@vulcanus [admin]
11:08 <wm-bot2> Rebooting node cloudcephosd1021.eqiad.wmnet (T309789) - cookbook ran by dcaro@vulcanus [admin]
11:07 <wm-bot2> Rebooting node cloudcephosd1021.eqiad.wmnet (T309789) - cookbook ran by dcaro@vulcanus [admin]
11:05 <wm-bot2> Rebooting node cloudcephosd1021.eqiad.wmnet (T309789) - cookbook ran by dcaro@vulcanus [admin]
11:04 <wm-bot2> Rebooting node cloudcephosd1021.eqiad.wmnet (T309789) - cookbook ran by dcaro@vulcanus [admin]
11:03 <wm-bot2> Rebooting node cloudcephosd1021.eqiad.wmnet (T309789) - cookbook ran by dcaro@vulcanus [admin]
09:15 <wm-bot2> Rebooting node cloudcephosd1021.eqiad.wmnet (T309789) - cookbook ran by dcaro@vulcanus [admin]
2022-06-06 §
13:21 <andrewbogott> restarting mysql/galera on cloudcontrol100x in an attempt to stabilize some flapping there [admin]
2022-06-02 §
07:51 <taavi> restart neutron-linuxbridge-agent.service on cloudvirt1034 T309732 [admin]
00:37 <andrewbogott> updated nameservers for codfw1dev instances via 'openstack subnet set --dns-nameserver etc.' [admin]
2022-06-01 §
17:11 <andrewbogott> restarting designate services in cloudservices1xxx hosts [admin]
16:37 <taavi> root@cloudcontrol1005:~# cinder reset-state --state available 491066a4-16a9-4ce4-a9f6-182660616d77 for T309659 [admin]
2022-05-30 §
17:43 <andrewbogott> restarting neutron-rpc and neutron-api services on cloudcontrol1xxx [admin]
2022-05-29 §
14:55 <andrewbogott> restarting nova services on all eqiad1 cloudcontrol nodes to recover from rabbit breakage [admin]
14:15 <andrewbogott> restarting rabbitmq on all eqiad1 cloudcontrol nodes (one at a time) [admin]
2022-05-25 §
20:01 <balloons> clean up cinder backup a bit, restart service due to network outage [admin]
20:01 <balloons> cloudvirt1029 restarted nova due to network outage [admin]
2022-05-19 §
15:21 <andrewbogott> resetting password for the 'troveguest' rabbitmq user. I think I may have broken this during a recent rebuild of the rabbitmq cluster [admin]
2022-05-18 §
15:42 <andrewbogott> updated the 'debian-11.0-bullseye' glance image with a fresh build [admin]
2022-05-14 §
11:33 <taavi> deleted projects 'ores' and 'ores-staging' T308102 [admin]
2022-05-13 §
06:20 <wm-bot2> Safe reboot of 'cloudvirt1045.eqiad.wmnet' finished successfully. - cookbook ran by andrew@buster [admin]
06:20 <wm-bot2> Unset cloudvirt 'cloudvirt1045.eqiad.wmnet' maintenance. - cookbook ran by andrew@buster [admin]
06:16 <wm-bot2> Drained 'cloudvirt1045.eqiad.wmnet'. - cookbook ran by andrew@buster [admin]
06:16 <wm-bot2> Set cloudvirt 'cloudvirt1045.eqiad.wmnet' maintenance. - cookbook ran by andrew@buster [admin]
06:15 <wm-bot2> Draining 'cloudvirt1045.eqiad.wmnet'. - cookbook ran by andrew@buster [admin]
06:15 <wm-bot2> Safe rebooting 'cloudvirt1045.eqiad.wmnet'. - cookbook ran by andrew@buster [admin]
06:11 <wm-bot2> Safe reboot of 'cloudvirt1044.eqiad.wmnet' finished successfully. - cookbook ran by andrew@buster [admin]
06:11 <wm-bot2> Unset cloudvirt 'cloudvirt1044.eqiad.wmnet' maintenance. - cookbook ran by andrew@buster [admin]
06:10 <wm-bot2> Set cloudvirt 'cloudvirt1045.eqiad.wmnet' maintenance. - cookbook ran by andrew@buster [admin]
06:10 <wm-bot2> Draining 'cloudvirt1045.eqiad.wmnet'. - cookbook ran by andrew@buster [admin]
06:09 <wm-bot2> Safe rebooting 'cloudvirt1045.eqiad.wmnet'. - cookbook ran by andrew@buster [admin]
06:07 <wm-bot2> Drained 'cloudvirt1044.eqiad.wmnet'. - cookbook ran by andrew@buster [admin]