| 2012-08-27
      
      § | 
    
  | 23:31 | <Ryan_Lane> | restarting virt6 | [production] | 
            
  | 23:08 | <Ryan_Lane> | upgrading virt6 to precise. This action will cause downtime for a while. | [production] | 
            
  | 22:49 | <mutante> | fixed gmq.planet.wm updates | [production] | 
            
  | 22:34 | <Ryan_Lane> | rebooting virt3 | [production] | 
            
  | 22:24 | <Ryan_Lane> | upgrading virt3 to precise | [production] | 
            
  | 22:23 | <Ryan_Lane> | disabling nova-compute on virt2 | [production] | 
            
  | 21:01 | <dschoon> | ${whoami} | [production] | 
            
  | 21:00 | <Ryan_Lane> | enabling OpenStackManager and LdapAuthentication on labsconsole temporarily for testing | [production] | 
            
  | 21:00 | <Ryan_Lane> | enabling nova-compute on virt2 for testing instance creation | [production] | 
            
  | 20:47 | <ottomata> | starting puppet back up on emery | [production] | 
            
  | 20:32 | <aaron> | synchronized php-1.20wmf9/extensions/MobileFrontend  'deployed f2551fa97ba686101745d5cb97ef4678838cd3bb' | [production] | 
            
  | 20:29 | <aaron> | synchronized php-1.20wmf10/extensions/MobileFrontend  'deployed 75104cf6d3bc6d528d2b85e09761005081fb5cf4' | [production] | 
            
  | 20:21 | <paravoid> | deploying swift origs to all pmtpa backend squids | [production] | 
            
  | 20:09 | <Ryan_Lane> | force running puppet on virt2 | [production] | 
            
  | 20:04 | <Ryan_Lane> | rebooting virt2 | [production] | 
            
  | 19:54 | <paravoid> | deploying swift origs on sq51 (1h ago), sq52, sq53 | [production] | 
            
  | 19:54 | <Ryan_Lane> | upgrading virt2 to precise | [production] | 
            
  | 19:38 | <Ryan_Lane> | upgrading OpenStackManager on labsconsole | [production] | 
            
  | 19:33 | <Ryan_Lane> | adding keystone services and endpoints | [production] | 
            
  | 19:32 | <Ryan_Lane> | upgrading nova, glance and keystone database schemas | [production] | 
            
  | 19:18 | <ottomata> | temporarily stopping puppet on emery to test log2udp AFT clicktracking relay to vanadium | [production] | 
            
  | 19:06 | <aaron> | rebuilt wikiversions.cdb and synchronized wikiversions files: enwiki to 1.20wmf10 | [production] | 
            
  | 18:41 | <Ryan_Lane> | force running puppet on virt0 | [production] | 
            
  | 18:36 | <Ryan_Lane> | restarting virt0 | [production] | 
            
  | 18:09 | <Ryan_Lane> | upgrading virt0 to precise | [production] | 
            
  | 17:54 | <Ryan_Lane> | running a fresh backup of opendj on virt0 | [production] | 
            
  | 17:54 | <Ryan_Lane> | backing up all databases on virt0 | [production] | 
            
  | 17:46 | <Ryan_Lane> | stopping rabbitmq on virt0 | [production] | 
            
  | 17:43 | <Ryan_Lane> | disabling openstack services on the virt infrastructure, that is | [production] | 
            
  | 17:43 | <Ryan_Lane> | disabling all openstack services | [production] | 
            
  | 17:42 | <Ryan_Lane> | disabling OpenStackManager and LdapAuthentication on labsconsole | [production] | 
            
  | 17:37 | <Ryan_Lane> | disabling puppet on all virt nodes | [production] | 
            
  | 17:33 | <paravoid> | rolled back sq51's config; depooled sq51 (10' ago) | [production] | 
            
  | 17:19 | <paravoid> | modified sq51 to serve origs from swift; added sq51 back to the pool | [production] | 
            
  | 16:45 | <paravoid> | depooling sq51 backend | [production] | 
            
  | 15:55 | <paravoid> | upgraded swift on ms-be12 | [production] | 
            
  | 15:54 | <paravoid> | upgraded swift on ms-be5 | [production] | 
            
  | 15:47 | <paravoid> | upgraded swift on ms-be11 | [production] | 
            
  | 14:51 | <paravoid> | upgraded swift on ms-be3 | [production] | 
            
  | 14:40 | <paravoid> | upgraded swift on ms-be4 | [production] | 
            
  | 14:30 | <paravoid> | upgraded swift on ms-be10; (still depooled, no effect) | [production] |