| 
      
        2012-08-17
      
      §
     | 
  
    
  | 15:11 | 
  <RobH> | 
  authdns-update for fluorine ip update | 
  [production] | 
            
  | 15:04 | 
  <RobH> | 
  lvm restructure | 
  [production] | 
            
  | 15:04 | 
  <RobH> | 
  all data copied, all traffic moved, fluorine coming down for reinstallation into internal vlan and lvs restructure | 
  [production] | 
            
  | 15:03 | 
  <RobH> | 
  all log traffic appears to have moved back to nfs1, copying fluorine logs back to nfs for reformat | 
  [production] | 
            
  | 14:53 | 
  <mark> | 
  Setup static 802.3ad trunk on asw-d3-sdtpa with one port up | 
  [production] | 
            
  | 14:27 | 
  <robh> | 
  synchronized wmf-config/InitialiseSettings.php  | 
  [production] | 
            
  | 14:27 | 
  <robh> | 
  synchronized wmf-config/CommonSettings.php  | 
  [production] | 
            
  | 13:48 | 
  <robh> | 
  ran sync-common-all  | 
  [production] | 
            
  | 13:34 | 
  <RobH> | 
  moving apache logs from pushing to fluorine back to nfs1 for now | 
  [production] | 
            
  | 05:12 | 
  <Tim> | 
  queued refreshlinks jobs for 1330 commonswiki image description pages with coord templates which weren't refreshed in the previous manual attempt, probably due to failed jobs | 
  [production] | 
            
  | 03:34 | 
  <aaron> | 
  synchronized php-1.20wmf9/includes/media/DjVu.php  'deployed 3ae632c3c9111b16f019574bcafec58fa3ee408d' | 
  [production] | 
            
  | 03:34 | 
  <aaron> | 
  synchronized php-1.20wmf9/includes/media/SVG.php  'deployed 3ae632c3c9111b16f019574bcafec58fa3ee408d ' | 
  [production] | 
            
  | 03:16 | 
  <aaron> | 
  synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php  | 
  [production] | 
            
  | 03:14 | 
  <aaron> | 
  synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php  | 
  [production] | 
            
  | 03:10 | 
  <aaron> | 
  synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php  'Profiling' | 
  [production] | 
            
  | 03:08 | 
  <aaron> | 
  synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php  'Profiling' | 
  [production] | 
            
  | 02:42 | 
  <aaron> | 
  synchronized php-1.20wmf9/includes/filerepo/backend/SwiftFileBackend.php  'Profiling' | 
  [production] | 
            
  | 02:37 | 
  <LocalisationUpdate> | 
  completed (1.20wmf9) at Fri Aug 17 02:37:38 UTC 2012 | 
  [production] | 
            
  | 01:12 | 
  <aaron> | 
  synchronized php-1.20wmf9/includes/filerepo/backend/TempFSFile.php  'deployed 08ea8adc95a4edba0b1f2f13c34f2a57725f1a58' | 
  [production] | 
            
  | 01:12 | 
  <aaron> | 
  synchronized php-1.20wmf9/includes/filerepo/file/File.php  'deployed 08ea8adc95a4edba0b1f2f13c34f2a57725f1a58' | 
  [production] | 
            
  | 00:13 | 
  <Tim> | 
  copying MW log archives from NFS to fluorine via rsync on fenari | 
  [production] | 
            
  
    | 
      
        2012-08-16
      
      §
     | 
  
    
  | 23:39 | 
  <Tim> | 
  on fluorine: chown'd log files and restarted udp2log to get new uid | 
  [production] | 
            
  | 23:39 | 
  <binasher> | 
  reslaved es1002 after conversion to innodb | 
  [production] | 
            
  | 23:31 | 
  <olivneh> | 
  synchronized php-1.20wmf9/extensions/E3Experiments/Experiments.hooks.php  | 
  [production] | 
            
  | 23:26 | 
  <mutante> | 
  fix date/NTP on sq36 | 
  [production] | 
            
  | 23:14 | 
  <kaldari> | 
  Finished syncing Wikimedia installation... :  | 
  [production] | 
            
  | 23:12 | 
  <mutante> | 
  killing nagios-wm. it stopped talking even though stuff gets written to --infile | 
  [production] | 
            
  | 22:51 | 
  <kaldari> | 
  Started syncing Wikimedia installation... :  | 
  [production] | 
            
  | 22:25 | 
  <preilly> | 
  Finished syncing Wikimedia installation... : update mobile frontend | 
  [production] | 
            
  | 22:04 | 
  <preilly> | 
  Started syncing Wikimedia installation... : update mobile frontend | 
  [production] | 
            
  | 22:03 | 
  <notpeter> | 
  restarting puppet on nfs1/2 | 
  [production] | 
            
  | 21:54 | 
  <notpeter> | 
  restarting puppet on locke | 
  [production] | 
            
  | 21:51 | 
  <notpeter> | 
  restarting puppet on emery | 
  [production] | 
            
  | 21:39 | 
  <notpeter> | 
  restarting puppet on oxygen | 
  [production] | 
            
  | 21:37 | 
  <cmjohnson1> | 
  shutting ms-be6 down for hardware testing/replacing | 
  [production] | 
            
  | 21:27 | 
  <notpeter> | 
  stopping puppet on nfs2 and restarting both udp2log instances to switch user. will keep puppet off until I merge new init script into puppet | 
  [production] | 
            
  | 21:20 | 
  <notpeter> | 
  stopping puppet on nfs1 and restarting both udp2log instances to switch user. will keep puppet off until I merge new init script into puppet | 
  [production] | 
            
  | 21:12 | 
  <maplebed> | 
  ms-be6 down for bad disk repair | 
  [production] | 
            
  | 20:58 | 
  <Ryan_Lane> | 
  attempting to upgrade virt1000 to precise | 
  [production] | 
            
  | 20:47 | 
  <cmjohnson1> | 
  ms-be6 shutting down to replace 2 disks | 
  [production] | 
            
  | 20:38 | 
  <notpeter> | 
  stopping puppet on locke and restarting both udp2log instances to switch user. will keep puppet off until I merge new init script into puppet | 
  [production] | 
            
  | 20:12 | 
  <RobH> | 
  ms-be1004 is ready to be installed into service | 
  [production] | 
            
  | 19:57 | 
  <RobH> | 
  ms-be1004 down for ssd install | 
  [production] | 
            
  | 19:44 | 
  <notpeter> | 
  stopping puppet on emery and restarting both udp2log instances to switch user. will keep puppet off until I merge new init script into puppet | 
  [production] | 
            
  | 19:34 | 
  <aaron> | 
  synchronized wmf-config/filebackend.php  'Set $readFromSwiftShared = true;' | 
  [production] | 
            
  | 19:31 | 
  <aaron> | 
  synchronized wmf-config/filebackend.php  | 
  [production] | 
            
  | 19:29 | 
  <notpeter> | 
  stopping puppet on oxygen and restarting both udp2log instances to switch user. will keep puppet off until I merge new init script into puppet | 
  [production] | 
            
  | 19:23 | 
  <aaron> | 
  synchronized wmf-config/filebackend.php  | 
  [production] | 
            
  | 19:17 | 
  <binasher> | 
  running purgeParserCache.php again | 
  [production] | 
            
  | 19:05 | 
  <RobH> | 
  puppet restarted on brewster | 
  [production] |