| 2012-11-29
      
      § | 
    
  | 19:22 | <demon> | synchronized wmf-config/InitialiseSettings.php  'TemplateSandbox on all wikis' | [production] | 
            
  | 19:21 | <demon> | synchronized wmf-config/CommonSettings.php  'TemplateSandbox on all wikis' | [production] | 
            
  | 18:13 | <aaron> | synchronized wmf-config/PrivateSettings.php  'Set Swift auth URL to the IP address host.' | [production] | 
            
  | 17:30 | <cmjohnson1> | es1004 has raid error...power cycling to enter raid bios | [production] | 
            
  | 15:37 | <cmjohnson1> | provisioning labsdb1002 | [production] | 
            
  | 14:37 | <cmjohnson1> | provisioning labsdb1001 | [production] | 
            
  | 06:09 | <Tim> | fixing searchidx2 by increasing RMI timeout | [production] | 
            
  | 02:45 | <LocalisationUpdate> | completed (1.21wmf4) at Thu Nov 29 02:45:27 UTC 2012 | [production] | 
            
  | 02:24 | <LocalisationUpdate> | completed (1.21wmf5) at Thu Nov 29 02:24:32 UTC 2012 | [production] | 
            
  | 00:48 | <pgehres> | synchronized php-1.21wmf4/extensions/CentralNotice/  'Updating CentralNotice to master to hide banners on mobile devices' | [production] | 
            
  | 00:45 | <pgehres> | synchronized php-1.21wmf5/extensions/CentralNotice/  'Updating CentralNotice to master to hide banners on mobile devices' | [production] | 
            
  | 00:12 | <Tim> | uploaded new lucene-search-2 with fixed init script, will upgrade it on the search servers | [production] | 
            
  
    | 2012-11-28
      
      § | 
    
  | 23:30 | <kaldari> | synchronized php-1.21wmf5/skins/vector/screen.css  'updating vector skin' | [production] | 
            
  | 23:30 | <kaldari> | synchronized php-1.21wmf5/skins/modern/main.css  'updating modern skin' | [production] | 
            
  | 23:29 | <kaldari> | synchronized php-1.21wmf5/extensions/Vector/modules/ext.vector.collapsibleNav.css  'adding forward-compat to Vector for [[bugzilla:42452|bug 42452]]' | [production] | 
            
  | 22:59 | <demon> | rebuilt wikiversions.cdb and synchronized wikiversions files: Moving all non-pedias back to wmf5 | [production] | 
            
  | 22:46 | <cmjohnson1> | running authdns update | [production] | 
            
  | 22:45 | <cmjohnson1> | adding DNS entry labsdb1001/1002 .eqiad.wmnet | [production] | 
            
  | 22:41 | <demon> | rebuilt wikiversions.cdb and synchronized wikiversions files: metawiki to wmf5 | [production] | 
            
  | 22:34 | <demon> | rebuilt wikiversions.cdb and synchronized wikiversions files: Rolling wikiversions.dat back to de90c27d, undoing todays deploy | [production] | 
            
  | 22:18 | <aaron> | synchronized php-1.21wmf4/extensions/SwiftCloudFiles | [production] | 
            
  | 22:01 | <Ryan_Lane> | disabled page counters on labsconsole | [production] | 
            
  | 21:27 | <demon> | rebuilt wikiversions.cdb and synchronized wikiversions files: Commonswiki back to 1.21wmf5 | [production] | 
            
  | 21:13 | <kaldari> | synchronized php-1.21wmf5/extensions/UploadWizard/resources/mw.UploadWizardDetails.js  'fixing uplaodwizrd for ie8' | [production] | 
            
  | 20:41 | <aaron> | synchronized php-1.21wmf5/extensions/SwiftCloudFiles  'deployed d50d754971991bb2362455fe47eaadb20aff3a80' | [production] | 
            
  | 20:41 | <RoanKattouw> | Restarted logmsgbot | [production] | 
            
  | 18:48 | <notpeter> | temp stopping puppet on brewster | [production] | 
            
  | 14:45 | <^demon> | fenari:/home/wikipedia/common now reflects Ia319794c - rm'd all traces of submodule | [production] | 
            
  | 05:47 | <Tim> | on searchidx2: killed extra IncrementalUpdater daemon | [production] | 
            
  | 04:12 | <Tim> | on searchidx2: restarted incremental updater | [production] | 
            
  | 00:16 | <MaxSem> | Scap complete | [production] | 
            
  
    | 2012-11-27
      
      § | 
    
  | 23:14 | <Ryan_Lane> | restarted gerrit; was getting WARN  org.eclipse.jetty.util.log : Dispatched Failed! errors | [production] | 
            
  | 22:27 | <MaxSem> | scapping as a part of usual mobile deployment | [production] | 
            
  | 22:23 | <mutante> | starting squid backend on knsq18 (cron spam/nagios) | [production] | 
            
  | 21:52 | <Reedy> | reedy synchronized php-1.21wmf5/extensions/LabeledSectionTransclusion | [production] | 
            
  | 21:50 | <robla> | logmsgbot doesn't seem to be working for sync-dir, scap, etc | [production] | 
            
  | 20:47 | <Ryan_Lane> | upgrading salt in production, in batches of 10 | [production] | 
            
  | 20:18 | <Ryan_Lane> | upgrading salt on all labs minions | [production] | 
            
  | 20:07 | <notpeter> | temp stopping puppet on brewster | [production] | 
            
  | 19:31 | <mutante> | upgrade libc* packages on sodium and singer | [production] | 
            
  | 19:29 | <mutante> | installing a bunch of pkg upgrades on zirconium | [production] | 
            
  | 19:28 | <mutante> | installing a bunch of pkg upgrades on | [production] | 
            
  | 16:07 | <paravoid> | ignore previous labs-related entry | [production] | 
            
  | 16:06 | <paravoid> | bastion disabled hazard-sj's cron, as it was accidentally DoSing bastion | [production] | 
            
  | 14:15 | <apergos> | snapshot1002 down for reinstall and testing | [production] | 
            
  | 12:32 | <mark> | Removed ms6 out of the loop as backend server for esams upload squids | [production] | 
            
  | 04:11 | <pgehres> | TimStarling: restarting lucene-search-2 on search1016 due to init script bug | [production] | 
            
  | 04:11 | <pgehres> | synchronized wmf-config/reporting-setup.php "Setting S:FundraiserStatistics to use db1013 as the slave" | [production] |