| 
      
        2012-11-27
      
      §
     | 
  
    
  | 19:29 | 
  <mutante> | 
  installing a bunch of pkg upgrades on zirconium | 
  [production] | 
            
  | 19:28 | 
  <mutante> | 
  installing a bunch of pkg upgrades on | 
  [production] | 
            
  | 16:07 | 
  <paravoid> | 
  ignore previous labs-related entry | 
  [production] | 
            
  | 16:06 | 
  <paravoid> | 
  bastion disabled hazard-sj's cron, as it was accidentally DoSing bastion | 
  [production] | 
            
  | 14:15 | 
  <apergos> | 
  snapshot1002 down for reinstall and testing | 
  [production] | 
            
  | 12:32 | 
  <mark> | 
  Removed ms6 out of the loop as backend server for esams upload squids | 
  [production] | 
            
  | 04:11 | 
  <pgehres> | 
  TimStarling: restarting lucene-search-2 on search1016 due to init script bug | 
  [production] | 
            
  | 04:11 | 
  <pgehres> | 
  synchronized wmf-config/reporting-setup.php "Setting S:FundraiserStatistics to use db1013 as the slave" | 
  [production] | 
            
  
    | 
      
        2012-11-26
      
      §
     | 
  
    
  | 19:29 | 
  <logmsgbot_> | 
  reedy rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki and mediawikiwiki to 1.21wmf5 | 
  [production] | 
            
  | 19:27 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf5/extensions/UploadWizard | 
  [production] | 
            
  | 19:24 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf5/cache/l10n/ | 
  [production] | 
            
  | 19:10 | 
  <logmsgbot_> | 
  reedy Started syncing Wikimedia installation... : Rebuild localisation cache for 1.21wmf5 | 
  [production] | 
            
  | 18:58 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf5/cache/l10n/ | 
  [production] | 
            
  | 18:47 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf5/extensions/Wikibase | 
  [production] | 
            
  | 18:45 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf5/extensions/Diff | 
  [production] | 
            
  | 18:37 | 
  <logmsgbot_> | 
  aaron synchronized php-1.21wmf5/includes/filerepo/file/LocalFile.php  'deployed 1fa41352d4e0b3cd2e53e7d8d485ac9098b8bfcc' | 
  [production] | 
            
  | 18:31 | 
  <logmsgbot_> | 
  reedy Started syncing Wikimedia installation... : Build localisation cache for 1.21wmf5 | 
  [production] | 
            
  | 18:16 | 
  <logmsgbot_> | 
  reedy rebuilt wikiversions.cdb and synchronized wikiversions files: test2wiki to 1.21wmf5 for l10n stuffs | 
  [production] | 
            
  | 18:15 | 
  <logmsgbot_> | 
  aaron synchronized php-1.21wmf4/thumb.php  'deployed 81c1d0133f74eec600277ab744a9730c4e974e47' | 
  [production] | 
            
  | 18:14 | 
  <logmsgbot_> | 
  reedy synchronized php-1.21wmf5  'Initial sync of php-1.21wmf5' | 
  [production] | 
            
  | 18:03 | 
  <logmsgbot_> | 
  aaron synchronized php-1.21wmf5/extensions/FlaggedRevs | 
  [production] | 
            
  | 18:02 | 
  <logmsgbot_> | 
  reedy synchronized wmf-config/ExtensionMessages-1.21wmf5.php | 
  [production] | 
            
  | 18:01 | 
  <logmsgbot_> | 
  reedy synchronized live-1.5/static-1.21wmf5/ | 
  [production] | 
            
  | 17:58 | 
  <logmsgbot_> | 
  aaron synchronized php-1.21wmf4/extensions/FlaggedRevs | 
  [production] | 
            
  | 17:55 | 
  <mutante> | 
  authdns-update - adding wikipedia.bg | 
  [production] | 
            
  | 16:29 | 
  <mark> | 
  Dist-upgraded & rebooted eqiad upload Varnish servers | 
  [production] | 
            
  | 15:26 | 
  <Jeff_Green> | 
  dist-upgrades  & reboots for payments100[1-4], pay-lvs100[12] | 
  [production] | 
            
  | 02:24 | 
  <logmsgbot_> | 
  LocalisationUpdate completed (1.21wmf4) at Mon Nov 26 02:23:58 UTC 2012 | 
  [production] | 
            
  
    | 
      
        2012-11-23
      
      §
     | 
  
    
  | 23:15 | 
  <logmsgbot_> | 
  olivneh synchronized wmf-config/InitialiseSettings.php  'Enabling PostEdit on ptwiki & svwiki' | 
  [production] | 
            
  | 18:28 | 
  <logmsgbot_> | 
  maxsem synchronized php-1.21wmf4/includes/Message.php  'Debugging' | 
  [production] | 
            
  | 16:20 | 
  <logmsgbot_> | 
  maxsem synchronized php-1.21wmf4/includes/Message.php  'Debugging' | 
  [production] | 
            
  | 15:09 | 
  <mark> | 
  Dist-upgraded and rebooted all eqiad bits servers (new Varnish package) | 
  [production] | 
            
  | 15:04 | 
  <logmsgbot_> | 
  maxsem synchronized php-1.21wmf4/extensions/CategoryTree/  'https://gerrit.wikimedia.org/r/#/c/34668/' | 
  [production] | 
            
  | 14:57 | 
  <mark> | 
  Dist-upgraded and rebooted all pmtpa bits servers (new Varnish package) | 
  [production] | 
            
  | 14:33 | 
  <mark> | 
  Dist-upgraded and rebooted all esams bits servers (new Varnish package) | 
  [production] | 
            
  | 14:05 | 
  <mark> | 
  Built new varnish 3.0.3plus~rc1-wm6 packages with fixed epoll deadlock, and inserted it into the precise-wikimedia APT repository | 
  [production] | 
            
  | 04:20 | 
  <Tim> | 
  on fenari: updated the "search" dsh node group based on nmap -sP and fixed the remaining search servers | 
  [production] | 
            
  | 04:04 | 
  <Tim> | 
  oh yeah, and I upgraded lucene to my version with the timeouts, deployed to pmtpa only via puppet | 
  [production] | 
            
  | 04:02 | 
  <Tim> | 
  many lucene search servers failed to bind to port 1099 when they were restarted by the upgrade, restarting manually | 
  [production] | 
            
  | 02:24 | 
  <logmsgbot_> | 
  LocalisationUpdate completed (1.21wmf4) at Fri Nov 23 02:24:34 UTC 2012 | 
  [production] |