| 2012-08-20
      
      § | 
    
  | 15:39 | <reedy> | Finished syncing Wikimedia installation... : testwiki to 1.20wmf10 and rebuild localisation cache for it.. | [production] | 
            
  | 15:01 | <reedy> | Started syncing Wikimedia installation... : testwiki to 1.20wmf10 and rebuild localisation cache for it.. | [production] | 
            
  | 14:46 | <reedy> | synchronized php-1.20wmf10/ | [production] | 
            
  | 14:39 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 14:38 | <reedy> | synchronized php-1.20wmf10/ | [production] | 
            
  | 14:36 | <reedy> | synchronized live-1.5/ | [production] | 
            
  | 14:30 | <Reedy> | Removing php-1.20wmf7 from mediawiki-installation group | [production] | 
            
  | 14:26 | <reedy> | synchronized php-1.20wmf10/  'staging 1.20wmf10 directory' | [production] | 
            
  | 09:49 | <paravoid> | powering off mw8, faulty (#3425), has been flapping a lot | [production] | 
            
  | 09:24 | <paravoid> | restarted gmond in cp3002 | [production] | 
            
  | 02:28 | <LocalisationUpdate> | completed (1.20wmf9) at Mon Aug 20 02:28:20 UTC 2012 | [production] | 
            
  
    | 2012-08-17
      
      § | 
    
  | 23:18 | <Ryan_Lane> | added cn=essextest project to labs LDAP for essex testing. this project isn't compatible with diablo | [production] | 
            
  | 22:16 | <notpeter> | rebooting/reimaging zhen and silver for upgrade to precise | [production] | 
            
  | 22:01 | <mutante> | rebuilding wikitech wiki l18n cache | [production] | 
            
  | 22:00 | <mutante> | rebuilding wikitech-l archives | [production] | 
            
  | 20:15 | <Reedy> | Pointing php to php-1.20wmf9 | [production] | 
            
  | 20:13 | <Reedy> | Deleting php-1.20wmf8/cache/l10n from mediawiki-installation | [production] | 
            
  | 17:48 | <RobH> | once all apaches update, will migrate old logs from nfs to fluorine | [production] | 
            
  | 17:48 | <RobH> | apache logs confirmed hitting fluorine, all will transition over as they puppet update | [production] | 
            
  | 17:24 | <robh> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 17:24 | <robh> | synchronized wmf-config/CommonSettings.php | [production] | 
            
  | 17:23 | <RobH> | moving apache logs back to fluorine | [production] | 
            
  | 17:00 | <cmjohnson1> | shutting down search23 to repair DIMM | [production] | 
            
  | 16:59 | <cmjohnson1> | mc1 shutting down to repair DIMM | [production] | 
            
  | 16:31 | <mark> | Enabled all 3 new 802.3ad aggregated links for asw-d1-sdtpa to asw-d3-sdtpa | [production] | 
            
  | 16:15 | <RobH> | authdns-update to correct typo, nothing really broke badly, i r lucky | [production] | 
            
  | 15:54 | <RobH> | fluorine now coming down for reinstall, all log files backed up to nfs | [production] | 
            
  | 15:47 | <mark> | Created static 802.3ad trunks on csw1-sdtpa for asw-d1-sdtpa to asw-d3-sdtpa uplinks, deployed with only the existing port enabled | [production] | 
            
  | 15:44 | <mark> | Created static 802.3ad trunks on asw-d1-sdtpa and asw-d2-sdtpa for uplinks | [production] | 
            
  | 15:31 | <RobH> | puppet is taking forever on nagios, as soon as it completes, will remove fluorine from decom and rerun | [production] | 
            
  | 15:11 | <RobH> | authdns-update for fluorine ip update | [production] | 
            
  | 15:04 | <RobH> | lvm restructure | [production] | 
            
  | 15:04 | <RobH> | all data copied, all traffic moved, fluorine coming down for reinstallation into internal vlan and lvs restructure | [production] | 
            
  | 15:03 | <RobH> | all log traffic appears to have moved back to nfs1, copying fluorine logs back to nfs for reformat | [production] | 
            
  | 14:53 | <mark> | Setup static 802.3ad trunk on asw-d3-sdtpa with one port up | [production] | 
            
  | 14:27 | <robh> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 14:27 | <robh> | synchronized wmf-config/CommonSettings.php | [production] | 
            
  | 13:48 | <robh> | ran sync-common-all | [production] | 
            
  | 13:34 | <RobH> | moving apache logs from pushing to fluorine back to nfs1 for now | [production] | 
            
  | 05:12 | <Tim> | queued refreshlinks jobs for 1330 commonswiki image description pages with coord templates which weren't refreshed in the previous manual attempt, probably due to failed jobs | [production] | 
            
  | 03:34 | <aaron> | synchronized php-1.20wmf9/includes/media/DjVu.php  'deployed 3ae632c3c9111b16f019574bcafec58fa3ee408d' | [production] | 
            
  | 03:34 | <aaron> | synchronized php-1.20wmf9/includes/media/SVG.php  'deployed 3ae632c3c9111b16f019574bcafec58fa3ee408d ' | [production] | 
            
  | 03:16 | <aaron> | synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php | [production] | 
            
  | 03:14 | <aaron> | synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php | [production] | 
            
  | 03:10 | <aaron> | synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php  'Profiling' | [production] | 
            
  | 03:08 | <aaron> | synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php  'Profiling' | [production] |