| 
      
        2012-11-29
      
      §
     | 
  
    
  | 19:41 | 
  <mutante> | 
  removing mysql server from iron | 
  [production] | 
            
  | 19:29 | 
  <RobH> | 
  authdns-update to add dns for tin | 
  [production] | 
            
  | 19:25 | 
  <demon> | 
  synchronized php-1.21wmf5/extensions/LabeledSectionTransclusion/  'Updating LST to master' | 
  [production] | 
            
  | 19:22 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php  'TemplateSandbox on all wikis' | 
  [production] | 
            
  | 19:21 | 
  <demon> | 
  synchronized wmf-config/CommonSettings.php  'TemplateSandbox on all wikis' | 
  [production] | 
            
  | 18:13 | 
  <aaron> | 
  synchronized wmf-config/PrivateSettings.php  'Set Swift auth URL to the IP address host.' | 
  [production] | 
            
  | 17:30 | 
  <cmjohnson1> | 
  es1004 has raid error...power cycling to enter raid bios | 
  [production] | 
            
  | 15:37 | 
  <cmjohnson1> | 
  provisioning labsdb1002 | 
  [production] | 
            
  | 14:37 | 
  <cmjohnson1> | 
  provisioning labsdb1001 | 
  [production] | 
            
  | 06:09 | 
  <Tim> | 
  fixing searchidx2 by increasing RMI timeout | 
  [production] | 
            
  | 02:45 | 
  <LocalisationUpdate> | 
  completed (1.21wmf4) at Thu Nov 29 02:45:27 UTC 2012 | 
  [production] | 
            
  | 02:24 | 
  <LocalisationUpdate> | 
  completed (1.21wmf5) at Thu Nov 29 02:24:32 UTC 2012 | 
  [production] | 
            
  | 00:48 | 
  <pgehres> | 
  synchronized php-1.21wmf4/extensions/CentralNotice/  'Updating CentralNotice to master to hide banners on mobile devices' | 
  [production] | 
            
  | 00:45 | 
  <pgehres> | 
  synchronized php-1.21wmf5/extensions/CentralNotice/  'Updating CentralNotice to master to hide banners on mobile devices' | 
  [production] | 
            
  | 00:12 | 
  <Tim> | 
  uploaded new lucene-search-2 with fixed init script, will upgrade it on the search servers | 
  [production] | 
            
  
    | 
      
        2012-11-28
      
      §
     | 
  
    
  | 23:30 | 
  <kaldari> | 
  synchronized php-1.21wmf5/skins/vector/screen.css  'updating vector skin' | 
  [production] | 
            
  | 23:30 | 
  <kaldari> | 
  synchronized php-1.21wmf5/skins/modern/main.css  'updating modern skin' | 
  [production] | 
            
  | 23:29 | 
  <kaldari> | 
  synchronized php-1.21wmf5/extensions/Vector/modules/ext.vector.collapsibleNav.css  'adding forward-compat to Vector for [[bugzilla:42452|bug 42452]]' | 
  [production] | 
            
  | 22:59 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: Moving all non-pedias back to wmf5 | 
  [production] | 
            
  | 22:46 | 
  <cmjohnson1> | 
  running authdns update | 
  [production] | 
            
  | 22:45 | 
  <cmjohnson1> | 
  adding DNS entry labsdb1001/1002 .eqiad.wmnet | 
  [production] | 
            
  | 22:41 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: metawiki to wmf5 | 
  [production] | 
            
  | 22:34 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: Rolling wikiversions.dat back to de90c27d, undoing todays deploy | 
  [production] | 
            
  | 22:18 | 
  <aaron> | 
  synchronized php-1.21wmf4/extensions/SwiftCloudFiles | 
  [production] | 
            
  | 22:01 | 
  <Ryan_Lane> | 
  disabled page counters on labsconsole | 
  [production] | 
            
  | 21:27 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: Commonswiki back to 1.21wmf5 | 
  [production] | 
            
  | 21:13 | 
  <kaldari> | 
  synchronized php-1.21wmf5/extensions/UploadWizard/resources/mw.UploadWizardDetails.js  'fixing uplaodwizrd for ie8' | 
  [production] | 
            
  | 20:41 | 
  <aaron> | 
  synchronized php-1.21wmf5/extensions/SwiftCloudFiles  'deployed d50d754971991bb2362455fe47eaadb20aff3a80' | 
  [production] | 
            
  | 20:41 | 
  <RoanKattouw> | 
  Restarted logmsgbot | 
  [production] | 
            
  | 18:48 | 
  <notpeter> | 
  temp stopping puppet on brewster | 
  [production] | 
            
  | 14:45 | 
  <^demon> | 
  fenari:/home/wikipedia/common now reflects Ia319794c - rm'd all traces of submodule | 
  [production] | 
            
  | 05:47 | 
  <Tim> | 
  on searchidx2: killed extra IncrementalUpdater daemon | 
  [production] | 
            
  | 04:12 | 
  <Tim> | 
  on searchidx2: restarted incremental updater | 
  [production] | 
            
  | 00:16 | 
  <MaxSem> | 
  Scap complete | 
  [production] | 
            
  
    | 
      
        2012-11-27
      
      §
     | 
  
    
  | 23:14 | 
  <Ryan_Lane> | 
  restarted gerrit; was getting WARN  org.eclipse.jetty.util.log : Dispatched Failed! errors | 
  [production] | 
            
  | 22:27 | 
  <MaxSem> | 
  scapping as a part of usual mobile deployment | 
  [production] | 
            
  | 22:23 | 
  <mutante> | 
  starting squid backend on knsq18 (cron spam/nagios) | 
  [production] | 
            
  | 21:52 | 
  <Reedy> | 
  reedy synchronized php-1.21wmf5/extensions/LabeledSectionTransclusion | 
  [production] | 
            
  | 21:50 | 
  <robla> | 
  logmsgbot doesn't seem to be working for sync-dir, scap, etc | 
  [production] | 
            
  | 20:47 | 
  <Ryan_Lane> | 
  upgrading salt in production, in batches of 10 | 
  [production] | 
            
  | 20:18 | 
  <Ryan_Lane> | 
  upgrading salt on all labs minions | 
  [production] | 
            
  | 20:07 | 
  <notpeter> | 
  temp stopping puppet on brewster | 
  [production] | 
            
  | 19:31 | 
  <mutante> | 
  upgrade libc* packages on sodium and singer | 
  [production] | 
            
  | 19:29 | 
  <mutante> | 
  installing a bunch of pkg upgrades on zirconium | 
  [production] | 
            
  | 19:28 | 
  <mutante> | 
  installing a bunch of pkg upgrades on | 
  [production] | 
            
  | 16:07 | 
  <paravoid> | 
  ignore previous labs-related entry | 
  [production] | 
            
  | 16:06 | 
  <paravoid> | 
  bastion disabled hazard-sj's cron, as it was accidentally DoSing bastion | 
  [production] | 
            
  | 14:15 | 
  <apergos> | 
  snapshot1002 down for reinstall and testing | 
  [production] | 
            
  | 12:32 | 
  <mark> | 
  Removed ms6 out of the loop as backend server for esams upload squids | 
  [production] | 
            
  | 04:11 | 
  <pgehres> | 
  TimStarling: restarting lucene-search-2 on search1016 due to init script bug | 
  [production] |