| 2011-06-09
      
      § | 
    
  | 23:54 | <binasher> | upgrading mobile2 to maverick (pre natty) | [production] | 
            
  | 23:04 | <Ryan_Lane> | added preilly account as a restricted user. Added him to mobile1-5 | [production] | 
            
  | 22:30 | <binasher> | upgrading mobile3 to natty ice | [production] | 
            
  | 21:38 | <binasher> | upgrading mobile3 to maverick (then natty) | [production] | 
            
  | 20:02 | <catrope> | synchronizing Wikimedia installation... Revision: 89789: | [production] | 
            
  | 20:02 | <RoanKattouw> | Running scap to deploy ArticleFeedback changes | [production] | 
            
  | 18:59 | <binasher> | mobile5 natty | [production] | 
            
  | 18:42 | <binasher> | mobile5 upgrade to maverick on the way to natty | [production] | 
            
  | 15:28 | <catrope> | synchronized php-1.17/wmf-config/CommonSettings.php  'Raise account creation throttle for Lennart's IP on testwiki' | [production] | 
            
  | 15:10 | <RobH> | dns updated, all nameservers dig correctly. | [production] | 
            
  | 15:10 | <RobH> | pushing dns updates to add mgmt of all eqiad devices | [production] | 
            
  | 01:22 | <Ryan_Lane> | shutting off services on virt1 (including puppet) until I get a chance to firewall them properly | [production] | 
            
  
    | 2011-06-08
      
      § | 
    
  | 22:35 | <Ryan_Lane> | rebooting virt1 | [production] | 
            
  | 21:58 | <hcatlin> | deploying to mobile cluster | [production] | 
            
  | 21:35 | <Ryan_Lane> | changing controller.labs.wikimedia.org to a CNAME for virt1.wikimedia.org | [production] | 
            
  | 21:27 | <mark> | Started ircd on ekrem | [production] | 
            
  | 21:20 | <mark> | Moved routing of vlan 101 from csw1-sdtpa to csw5-pmtpa | [production] | 
            
  | 21:06 | <binasher> | changing pybal healthcheck url for mobile | [production] | 
            
  | 20:47 | <Ryan_Lane> | rebooting controller.labs.wikimedia.org | [production] | 
            
  | 20:44 | <Ryan_Lane> | restarting pdns on ns0, it's locked up | [production] | 
            
  | 20:33 | <Ryan_Lane> | make that deleting virt1.pmtpa.wmnet, and changing virt1.mgmt.pmtpa.wmnet to controller.labs.mgmt.pmtpa.wmnet | [production] | 
            
  | 20:32 | <Ryan_Lane> | changing virt1.pmtpa.wmnet to controller.labs.pmtpa.wmnet | [production] | 
            
  | 20:27 | <Ryan_Lane> | changing ip of controller.labs.wikimedia.org | [production] | 
            
  | 20:17 | <mark> | Rebooting ekrem | [production] | 
            
  | 20:16 | <mark> | Changed ekrem's ip | [production] | 
            
  | 20:10 | <mark> | Added 2nd ip to ekrem (208.80.152.178) | [production] | 
            
  | 19:56 | <binasher> | upgrade mobile4 to natty | [production] | 
            
  | 19:53 | <mark> | Shutdown ekrem's switch port and null routed the ip for a while | [production] | 
            
  | 19:53 | <Ryan_Lane> | deleting vmdev1 and vmdev2, and moving the controller.labs.wikimedia.org entry into the public services subnet | [production] | 
            
  | 19:45 | <binasher> | upgrading mobile4 to maverick on its way to natty | [production] | 
            
  | 19:31 | <Ryan_Lane> | reclaiming sandbox ips for owa.tesla, owa1.tesla, owa2.tesla, lvs1.tesla, and grid.tesla | [production] | 
            
  | 18:10 | <Ryan_Lane> | depooling mobile5 | [production] | 
            
  | 18:05 | <Ryan_Lane> | changing weight for mobile5 | [production] | 
            
  | 17:42 | <Ryan_Lane> | upping the weight of mobile5 to 24 | [production] | 
            
  | 17:27 | <Ryan_Lane> | upping weight of mobile5 to 16 | [production] | 
            
  | 17:18 | <Ryan_Lane> | depooling mobile4 | [production] | 
            
  | 17:16 | <Ryan_Lane> | reducing weight of mobile4/5 to 8 | [production] | 
            
  | 17:13 | <Ryan_Lane> | repooling mobile4/5 with a weight of 16 | [production] | 
            
  | 16:29 | <^demon|away> | irc feeds are down, somebody who has access should investigate. | [production] | 
            
  | 15:39 | <mark> | Rebooting sq71 to remove old temp eqiad tunnel config | [production] | 
            
  | 14:49 | <mark> | Setup PIM-SM on eqiad core routers and joined the pmtpa domain | [production] | 
            
  | 14:49 | <mark> | Changed pmtpa PIM-DM to PIM-SM | [production] | 
            
  | 11:31 | <mark> | Added A records for cr1-eqiad and cr2-eqiad loopback interfaces | [production] | 
            
  | 10:47 | <mark> | Set local-pref value for private peers on cr1-eqiad and cr2-eqiad | [production] | 
            
  | 10:43 | <mark> | Removed cr1-eqiad:ae1.666 configuration to remove the temporary tunneled link | [production] | 
            
  | 10:39 | <mark> | Configured OSPF and OSPFv3 on csw1-sdtpa:e16/1 and cr2-eqiad:xe-5/2/1 | [production] | 
            
  | 10:39 | <mark> | Assigned IP addresses to csw1-sdtpa:e16/1 and cr2-eqiad:xe-5/2/1 to bring up the second 10G wave | [production] | 
            
  | 00:37 | <awjrichards> | synchronized php-1.17/wmf-config/InitialiseSettings.php  'Enabling CustomUserSignup on testwiki' | [production] |