| 2012-07-17
      
      § | 
    
  | 15:06 | <mark> | lvs4 is back up and serving traffic | [production] | 
            
  | 14:52 | <mark> | Reinstalling lvs4 with Ubuntu Precise | [production] | 
            
  | 14:48 | <mark> | Stopped PyBal on lvs4, failing over traffic to lvs3 | [production] | 
            
  | 14:47 | <mark> | Reinstalling lvs1002 with Ubuntu Precise | [production] | 
            
  | 14:46 | <mark> | Stopped PyBal on lvs1002, failing over traffic to lvs1005 | [production] | 
            
  | 14:44 | <mark> | lvs1001 is back up and serving traffic | [production] | 
            
  | 14:11 | <mark> | Stopped PyBal on lvs1001, failing over traffic to lvs1004 | [production] | 
            
  | 13:09 | <maplebed> | changed auth URL for swift to use load balancer rather than round robin DNS | [production] | 
            
  | 13:08 | <mark> | lvs1003 is back up and serving traffic | [production] | 
            
  | 13:07 | <maplebed> | changed order on ganglia swift view graphs to group by metric rather than host | [production] | 
            
  | 13:02 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 12:53 | <mark> | Fixed boot order on lvs1003 | [production] | 
            
  | 12:34 | <mark> | Reinstalling lvs1003 with Ubuntu Precise | [production] | 
            
  | 12:28 | <mark> | Stopped PyBal on lvs1003, failing over traffic to lvs1006 | [production] | 
            
  | 12:10 | <mark> | amslvs1 is back up and serving traffic | [production] | 
            
  | 11:44 | <mark> | Reinstalling amslvs1 with Ubuntu Precise | [production] | 
            
  | 11:40 | <mark> | Stopped PyBal on amslvs1, failing over traffic to amslvs3 | [production] | 
            
  | 11:32 | <mark> | amslvs2 is back up and serving traffic | [production] | 
            
  | 10:47 | <mark> | Reinstalling amslvs2 with Ubuntu Precise | [production] | 
            
  | 10:36 | <mark> | Stopped PyBal on amslvs2, failing over traffic to amslvs4 | [production] | 
            
  | 09:17 | <hashar> | deployment-prep sync upload6 dirs again. root@deployment-nfs-memc:$ rsync -a --progress --inplace /mnt/export/upload6 /data/project/upload6 | [production] | 
            
  | 07:30 | <Tim> | testing envvars/apache2.conf change on srv193 | [production] | 
            
  | 07:09 | <Tim> | restarted apache on srv193 | [production] | 
            
  | 02:58 | <Tim> | graceful restart of all apaches | [production] | 
            
  | 02:52 | <reedy> | synchronized wmf-config/CommonSettings.php | [production] | 
            
  | 02:47 | <LocalisationUpdate> | completed (1.20wmf6) at Tue Jul 17 02:47:06 UTC 2012 | [production] | 
            
  | 02:23 | <LocalisationUpdate> | completed (1.20wmf7) at Tue Jul 17 02:23:41 UTC 2012 | [production] | 
            
  | 00:44 | <Tim> | deploying new redirects.conf | [production] | 
            
  | 00:28 | <Tim> | testing new redirects.conf on mw1 | [production] | 
            
  
    | 2012-07-16
      
      § | 
    
  | 23:03 | <pp-pdf1> | restarted nserve | [production] | 
            
  | 23:02 | <pp-pdf1> | fixed nserve handling of filenames with whitespace | [production] | 
            
  | 22:46 | <pp-pdf1> | restart all services | [production] | 
            
  | 22:45 | <pp-pdf2> | restart all services | [production] | 
            
  | 22:45 | <pp-pdf3> | restart all services | [production] | 
            
  | 22:45 | <pp-pdf1> | update mwlib to 0.13.9 | [production] | 
            
  | 22:45 | <pp-pdf2> | update mwlib to 0.13.9 | [production] | 
            
  | 22:45 | <pp-pdf3> | update mwlib to 0.13.9 | [production] | 
            
  | 22:44 | <pp-pdf2> | update greenlet to 0.4.0 | [production] | 
            
  | 22:44 | <pp-pdf1> | update greenlet to 0.4.0 | [production] | 
            
  | 22:44 | <pp-pdf3> | update greenlet to 0.4.0 | [production] | 
            
  | 22:44 | <pp-pdf2> | upgrade qserve to 0.2.8 | [production] | 
            
  | 22:44 | <pp-pdf3> | upgrade qserve to 0.2.8 | [production] | 
            
  | 22:43 | <pp-pdf1> | upgrade qserve to 0.2.8 | [production] | 
            
  | 22:20 | <awjrichards> | synchronized wmf-config/CommonSettings.php  'Disabling epub generation from Collection extension on all wikis except for simple and test' | [production] | 
            
  | 22:19 | <awjrichards> | synchronized wmf-config/InitialiseSettings.php  'Making epub generation for Collection extension configurable' | [production] | 
            
  | 18:26 | <AaronSchulz> | Running copyFileBackend.php for commons (shards 8-b) | [production] | 
            
  | 18:02 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: enwiki to 1.20wmf7 | [production] | 
            
  | 17:08 | <LeslieCarr> | reactivating XO transit connections on cr1-sdtpa | [production] | 
            
  | 17:03 | <LeslieCarr> | draining cr2-eqiad to cr1-sdtpa link for moving of fiber | [production] | 
            
  | 15:56 | <mark> | lvs3 is back up, and idling | [production] |