2551-2600 of 6182 results (32ms)
2023-02-02 §
12:57 <wm-bot2> Set the ceph cluster for eqiad1 in maintenance, alert silence ids: 7ac2b25a-d1bb-4789-8aa6-b9435b505349 (T316544) - cookbook ran by dcaro@vulcanus [admin]
2023-01-30 §
22:34 <wm-bot2> Upgraded and rebooted host cloudrabbit1002.wikimedia.org - cookbook ran by andrew@bullseye [admin]
21:34 <andrewbogott> merging https://gerrit.wikimedia.org/r/c/operations/puppet/+/884922 and upgrading rabbitmq nodes for T328155 [admin]
2023-01-27 §
20:08 <wm-bot2> Upgraded and rebooted host cloudcontrol2005-dev.wikimedia.org - cookbook ran by andrew@bullseye [admin]
19:22 <wm-bot2> Upgraded and rebooted host cloudcontrol2004-dev.wikimedia.org - cookbook ran by andrew@bullseye [admin]
19:10 <wm-bot2> Upgraded and rebooted host cloudcontrol2001-dev.wikimedia.org - cookbook ran by andrew@bullseye [admin]
15:25 <andrewbogott> restarting openstack services in eqiad1, another attempt to address instability [admin]
2023-01-26 §
20:34 <andrewbogott> shutting down mariadb on cloudbackup2001-dev, testing the waters for T328079 [admin]
2023-01-22 §
03:42 <andrewbogott> reset eqiad1 rabbitmq in an attempt to resolve some mild instability [admin]
2023-01-20 §
15:26 <wm-bot2> Removed cloudweb hosts (cloudweb2002-dev.wikimedia.org) from maintenance mode. - cookbook ran by andrew@bullseye [admin]
15:26 <wm-bot2> Put cloudweb hosts (cloudweb2002-dev.wikimedia.org) into maintenance mode (downtime id: ['f47a3d91-b270-4c90-acc8-d85075a6bf8e'], use this to unset) - cookbook ran by andrew@bullseye [admin]
13:15 <arturo> reinstall python3-neutron (to reset manual patching) on all cloudnet nodes and patch it via puppet, then restart neutron-l3-agent by hand (T327463) [admin]
10:12 <arturo> [codfw1dev] failover neutron-l3-agent between cloudnet2005-dev/cloudnet2006-dev a couple of times T327463 [admin]
02:17 <andrewbogott> stopping neutron-l3-agent on cloudnet1005 because it's logging at a furious rate and about to fill the drive [admin]
2023-01-19 §
18:06 <wm-bot2> Removed cloudweb hosts (cloudweb2002-dev.wikimedia.org) from maintenance mode. - cookbook ran by andrew@bullseye [admin]
18:06 <wm-bot2> Put cloudweb hosts (cloudweb2002-dev.wikimedia.org) into maintenance mode (downtime id: ['36d5af6a-7d8e-4d0c-831e-1bf05c255984'], use this to unset) - cookbook ran by andrew@bullseye [admin]
17:35 <wm-bot2> Removed cloudweb hosts (cloudweb2002-dev.wikimedia.org) from maintenance mode. - cookbook ran by andrew@bullseye [admin]
17:22 <wm-bot2> Put cloudweb hosts (cloudweb2002-dev.wikimedia.org) into maintenance mode (downtime id: ['66ec4f04-d25b-4067-be9e-2fe12cb1d3ff'], use this to unset) - cookbook ran by andrew@bullseye [admin]
2023-01-18 §
22:32 <wm-bot2> Set cloudweb cloudweb2002-dev.wikimedia.org maintenance (downtime id: 347cb75e-215e-4b85-ae14-4ce1934c70c7, use this to unset) - cookbook ran by andrew@bullseye [admin]
22:01 <wm-bot2> Set cloudweb cloudweb2002-dev.wikimedia.org maintenance (downtime id: 9bf6212f-7fdb-4869-8190-b07387b2bc7e, use this to unset) - cookbook ran by andrew@bullseye [admin]
21:40 <wm-bot2> Set cloudweb cloudweb2002-dev.wikimedia.org maintenance (downtime id: 11aec8ea-f443-41ae-b79a-5e1e3aa94546, use this to unset) - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Restarting openstack services on cloudservices2004-dev: ['designate-mdns', 'designate-sink', 'designate-central', 'designate-producer', 'designate-worker', 'designate-agent'] - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Restarting openstack services on cloudservices2005-dev: ['designate-central', 'designate-sink', 'designate-worker', 'designate-producer', 'designate-mdns', 'designate-agent'] - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Restarting openstack services on cloudnet2005-dev: ['neutron-metadata-agent', 'neutron-dhcp-agent', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Restarting openstack services on cloudnet2006-dev: ['neutron-dhcp-agent', 'neutron-metadata-agent', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
20:19 <wm-bot2> Restarting openstack services on cloudbackup1002-dev: ['cinder-backup'] - cookbook ran by andrew@bullseye [admin]
20:19 <wm-bot2> Restarting openstack services on cloudbackup1001-dev: ['cinder-backup'] - cookbook ran by andrew@bullseye [admin]
20:19 <wm-bot2> Restarting openstack services on cloudcontrol2005-dev: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata', 'cinder-volume', 'cinder-scheduler', 'neutron-api', 'neutron-rpc-server', 'trove-api', 'trove-conductor', 'trove-taskmanager', 'keystone', 'keystone-admin', 'glance-api', 'magnum-api', 'magnum-conductor', 'heat-api', 'heat-api-cfn', 'heat-engine'] - cookbook ran by andrew@bullseye [admin]
20:19 <wm-bot2> Restarting openstack services on cloudvirt2003-dev: ['nova-compute', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
20:19 <wm-bot2> Restarting openstack services on cloudvirt2002-dev: ['nova-compute', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
20:18 <wm-bot2> Restarting openstack services on cloudvirt2001-dev: ['nova-compute', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
20:18 <wm-bot2> Restarting openstack services on cloudcontrol2004-dev: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata', 'cinder-scheduler', 'cinder-volume', 'neutron-api', 'neutron-rpc-server', 'trove-api', 'trove-conductor', 'trove-taskmanager', 'keystone', 'keystone-admin', 'glance-api', 'magnum-api', 'magnum-conductor', 'heat-api', 'heat-api-cfn', 'heat-engine'] - cookbook ran by andrew@bullseye [admin]
20:18 <wm-bot2> Restarting openstack services on cloudcontrol2001-dev: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata', 'cinder-scheduler', 'cinder-volume', 'neutron-api', 'neutron-rpc-server', 'trove-api', 'trove-conductor', 'trove-taskmanager', 'keystone', 'keystone-admin', 'glance-api', 'magnum-api', 'magnum-conductor', 'heat-api', 'heat-api-cfn', 'heat-engine'] - cookbook ran by andrew@bullseye [admin]
17:03 <wm-bot2> Restarting openstack services on cloudservices2004-dev: ['designate-mdns', 'designate-sink', 'designate-central', 'designate-producer', 'designate-worker', 'designate-agent'] - cookbook ran by andrew@bullseye [admin]
17:02 <wm-bot2> Restarting openstack services on cloudservices2005-dev: ['designate-central', 'designate-sink', 'designate-worker', 'designate-producer', 'designate-mdns', 'designate-agent'] - cookbook ran by andrew@bullseye [admin]
17:02 <wm-bot2> Restarting openstack services on cloudnet2005-dev: ['neutron-metadata-agent', 'neutron-dhcp-agent', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
17:02 <wm-bot2> Restarting openstack services on cloudnet2006-dev: ['neutron-dhcp-agent', 'neutron-metadata-agent', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
17:02 <wm-bot2> Restarting openstack services on cloudbackup1002-dev: ['cinder-backup'] - cookbook ran by andrew@bullseye [admin]
17:02 <wm-bot2> Restarting openstack services on cloudbackup1001-dev: ['cinder-backup'] - cookbook ran by andrew@bullseye [admin]
17:02 <wm-bot2> Restarting openstack services on cloudcontrol2005-dev: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata', 'cinder-volume', 'cinder-scheduler', 'neutron-api', 'neutron-rpc-server', 'trove-api', 'trove-conductor', 'trove-taskmanager', 'keystone', 'keystone-admin', 'glance-api', 'magnum-api', 'magnum-conductor', 'heat-api', 'heat-api-cfn', 'heat-engine'] - cookbook ran by andrew@bullseye [admin]
17:02 <wm-bot2> Restarting openstack services on cloudvirt2003-dev: ['nova-compute', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
17:01 <wm-bot2> Restarting openstack services on cloudvirt2002-dev: ['nova-compute', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
17:01 <wm-bot2> Restarting openstack services on cloudvirt2001-dev: ['nova-compute', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
17:01 <wm-bot2> Restarting openstack services on cloudcontrol2004-dev: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata', 'cinder-scheduler', 'cinder-volume', 'neutron-api', 'neutron-rpc-server', 'trove-api', 'trove-conductor', 'trove-taskmanager', 'keystone', 'keystone-admin', 'glance-api', 'magnum-api', 'magnum-conductor', 'heat-api', 'heat-api-cfn', 'heat-engine'] - cookbook ran by andrew@bullseye [admin]
17:01 <wm-bot2> Restarting openstack services on cloudcontrol2001-dev: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata', 'cinder-scheduler', 'cinder-volume', 'neutron-api', 'neutron-rpc-server', 'trove-api', 'trove-conductor', 'trove-taskmanager', 'keystone', 'keystone-admin', 'glance-api', 'magnum-api', 'magnum-conductor', 'heat-api', 'heat-api-cfn', 'heat-engine'] - cookbook ran by andrew@bullseye [admin]
14:38 <wm-bot2> Restarting openstack services on cloudvirt2002-dev: ['nova-compute', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
14:38 <wm-bot2> Restarting openstack services on cloudvirt2001-dev: ['nova-compute', 'neutron-linuxbridge-agent'] - cookbook ran by andrew@bullseye [admin]
14:37 <wm-bot2> Restarting openstack services on cloudcontrol2004-dev: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata', 'cinder-scheduler', 'cinder-volume', 'neutron-api', 'neutron-rpc-server', 'trove-api', 'trove-conductor', 'trove-taskmanager', 'keystone', 'keystone-admin', 'glance-api', 'magnum-api', 'magnum-conductor', 'heat-api', 'heat-api-cfn', 'heat-engine'] - cookbook ran by andrew@bullseye [admin]
14:37 <wm-bot2> Restarting openstack services on cloudcontrol2001-dev: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata', 'cinder-scheduler', 'cinder-volume', 'neutron-api', 'neutron-rpc-server', 'trove-api', 'trove-conductor', 'trove-taskmanager', 'keystone', 'keystone-admin', 'glance-api', 'magnum-api', 'magnum-conductor', 'heat-api', 'heat-api-cfn', 'heat-engine'] - cookbook ran by andrew@bullseye [admin]
2023-01-17 §
19:32 <wm-bot2> Upgraded and rebooted host cloudbackup2002.codfw.wmnet - cookbook ran by andrew@bullseye [admin]