| 
      
        2012-11-15
      
      §
     | 
  
    
  | 22:41 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf4/extensions/ReaderFeedback/ | 
  [production] | 
            
  | 22:36 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf4/extensions/WikimediaIncubator/ | 
  [production] | 
            
  | 22:29 | 
  <Reedy> | 
  Ran sync-docroot | 
  [production] | 
            
  | 22:08 | 
  <notpeter> | 
  repooling srv25-srv257 | 
  [production] | 
            
  | 22:06 | 
  <notpeter> | 
  repooling srv190-srv192, srv225-srv230 | 
  [production] | 
            
  | 21:33 | 
  <binasher> | 
  starting apache on srv249 | 
  [production] | 
            
  | 21:32 | 
  <binasher> | 
  stopping apache on srv249 | 
  [production] | 
            
  | 21:30 | 
  <binasher> | 
  ran chmod 755 /usr/local/apache/common-local/wmf-config on srv249 | 
  [production] | 
            
  | 20:46 | 
  <Jeff_Green> | 
  adjusted key_buffer_size for fundraisingdbs from 1M to 1G | 
  [production] | 
            
  | 20:42 | 
  <notpeter> | 
  removing srv225-srv230 for upgrade to precise | 
  [production] | 
            
  | 20:34 | 
  <notpeter> | 
  reimaging srv190-srv192 as regular apaches | 
  [production] | 
            
  | 20:09 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf3/ | 
  [production] | 
            
  | 20:05 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf4/ | 
  [production] | 
            
  | 19:37 | 
  <notpeter> | 
  repooling srv200-srv213 | 
  [production] | 
            
  | 19:23 | 
  <notpeter> | 
  removing srv251-srv257 for upgrade to precise | 
  [production] | 
            
  | 19:19 | 
  <logmsgbot_> | 
  reedy synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 18:59 | 
  <paravoid> | 
  upgrading srv{190,219,220,221,222,223,224}.pmtpa.wmnet for libtiff vulnerabilities (USN-1631-1) | 
  [production] | 
            
  | 17:40 | 
  <cmjohnson1> | 
  wtp1  raid1 config is wrong; bringing server down to fix | 
  [production] | 
            
  | 17:12 | 
  <notpeter> | 
  removing srv248, srv249, and srv250 from various pools for upgrade to precise | 
  [production] | 
            
  | 16:47 | 
  <paravoid> | 
  running 'reprepro update' to fetch ceph packages | 
  [production] | 
            
  | 16:47 | 
  <cmjohnson1> | 
  shutting down srv250 to verify drac config | 
  [production] | 
            
  | 15:01 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf4/extensions/SiteMatrix/ | 
  [production] | 
            
  | 14:32 | 
  <notpeter> | 
  removing srv200-srv213 from pybal for upgrades to precise (and this time I mean it) | 
  [production] | 
            
  | 13:48 | 
  <logmsgbot_> | 
  py synchronized wmf-config/db.php  'pulling es2 for conversion to inno' | 
  [production] | 
            
  | 11:26 | 
  <logmsgbot_> | 
  kaldari synchronized php-1.21wmf4/extensions/UploadWizard/resources/mw.UploadWizardUploadInterface.js  'refixing for Safari, part 2' | 
  [production] | 
            
  | 11:26 | 
  <logmsgbot_> | 
  kaldari synchronized php-1.21wmf4/extensions/UploadWizard/resources/mw.UploadWizardDetails.js  'refixing for Safari, part 1' | 
  [production] | 
            
  | 11:15 | 
  <logmsgbot_> | 
  kaldari synchronized php-1.21wmf4/extensions/UploadWizard/resources/mw.UploadWizardUploadInterface.js  'refixing for Safari, part 2' | 
  [production] | 
            
  | 11:14 | 
  <logmsgbot_> | 
  kaldari synchronized php-1.21wmf4/extensions/UploadWizard/resources/mw.UploadWizardDetails.js  'refixing for Safari, part 1' | 
  [production] | 
            
  | 11:04 | 
  <logmsgbot_> | 
  kaldari synchronized php-1.21wmf4/extensions/UploadWizard/resources/mw.UploadWizardDetails.js  'rolling back my fix for Safari, part 1' | 
  [production] | 
            
  | 11:04 | 
  <logmsgbot_> | 
  kaldari synchronized php-1.21wmf4/extensions/UploadWizard/resources/mw.UploadWizardUploadInterface.js  'rolling back my fix for Safari, part 2' | 
  [production] | 
            
  | 10:48 | 
  <logmsgbot_> | 
  kaldari synchronized php-1.21wmf4/extensions/UploadWizard/resources/mw.UploadWizardUploadInterface.js  'syncing UploadWizard fix for Safari, part 2' | 
  [production] | 
            
  | 10:47 | 
  <logmsgbot_> | 
  kaldari synchronized php-1.21wmf4/extensions/UploadWizard/resources/mw.UploadWizardDetails.js  'syncing UploadWizard fix for Safari, part 1' | 
  [production] | 
            
  | 09:14 | 
  <logmsgbot_> | 
  aaron synchronized php-1.21wmf4/maintenance/nextJobDB.php  'deployed 2e0c24df43588bd4ceba5522bde1b3f06fbd05b0' | 
  [production] | 
            
  | 07:25 | 
  <Nemo_bis> | 
  Job queue still suspect since the wmf4 deploy, job runners activity halved; see [[bugzilla:41656|bug 41656]]  | 
  [production] | 
            
  | 04:20 | 
  <K4-713> | 
  updated payments cluster to a24d39a120f61cb | 
  [production] | 
            
  | 02:54 | 
  <logmsgbot_> | 
  LocalisationUpdate completed (1.21wmf3) at Thu Nov 15 02:54:48 UTC 2012 | 
  [production] | 
            
  | 02:34 | 
  <logmsgbot_> | 
  LocalisationUpdate completed (1.21wmf4) at Thu Nov 15 02:34:15 UTC 2012 | 
  [production] | 
            
  | 02:20 | 
  <K4-713> | 
  updated payments cluster to 46bcee539b93f7b | 
  [production] | 
            
  | 01:49 | 
  <logmsgbot_> | 
  catrope synchronized php-1.21wmf4/extensions/VisualEditor  'Updating VE to master' | 
  [production] | 
            
  | 01:49 | 
  <logmsgbot_> | 
  catrope synchronized php-1.21wmf3/extensions/VisualEditor  'Updating VE to master' | 
  [production] | 
            
  | 01:45 | 
  <logmsgbot_> | 
  catrope synchronized php-1.21wmf4/extensions/VisualEditor  'Updating VE to master' | 
  [production] | 
            
  | 01:44 | 
  <logmsgbot_> | 
  catrope synchronized php-1.21wmf3/extensions/VisualEditor  'Updating VE to master' | 
  [production] | 
            
  | 00:46 | 
  <K4-713> | 
  updated payments cluster to ab3d3615bf1ddd0c | 
  [production] | 
            
  | 00:35 | 
  <logmsgbot_> | 
  catrope synchronized php-1.21wmf4/extensions/VisualEditor  'Updating VE to master' | 
  [production] | 
            
  | 00:35 | 
  <logmsgbot_> | 
  catrope synchronized php-1.21wmf3/extensions/VisualEditor  'Updating VE to master' | 
  [production] | 
            
  | 00:23 | 
  <K4-713> | 
  updated payments cluster to 4ccd348df7661 | 
  [production] | 
            
  | 00:11 | 
  <logmsgbot_> | 
  asher synchronized wmf-config/db.php  'pulling db11' | 
  [production] |