| 
      
        2012-09-24
      
      §
     | 
  
    
  | 15:51 | 
  <RobH> | 
  mc1001-mc1008 are having memory upgrades installed.  hosts already offline. | 
  [production] | 
            
  | 13:53 | 
  <pp-pdf1> | 
  restart all services | 
  [production] | 
            
  | 13:52 | 
  <pp-pdf1> | 
  updated mwlib.epub to 0.14.2 | 
  [production] | 
            
  | 13:52 | 
  <pp-pdf1> | 
  updated mwlib.rl to 014.1 | 
  [production] | 
            
  | 13:52 | 
  <pp-pdf1> | 
  update mwlib to 0.14.1 | 
  [production] | 
            
  | 13:51 | 
  <pp-pdf1> | 
  updated timelib to 0.2.4 | 
  [production] | 
            
  | 13:50 | 
  <pp-pdf1> | 
  updated pip to 1.2.1 | 
  [production] | 
            
  | 13:50 | 
  <pp-pdf1> | 
  updated distribute to 0.6.28 | 
  [production] | 
            
  | 13:17 | 
  <LeslieCarr> | 
  switching unnumbered GRE tunnel to numbered GRE Tunnel between cr2-knams and cr2-eqiad for routability reasons | 
  [production] | 
            
  | 12:45 | 
  <LeslieCarr> | 
  restarting powerdns on ns2 | 
  [production] | 
            
  | 09:34 | 
  <hashar> | 
  synchronized docroot/bits/static-master/  '([[bugzilla:40419|bug 40419]]) extension assets not available on beta (aka add docroot/bits/static-master' | 
  [production] | 
            
  | 09:23 | 
  <hashar> | 
  synchronized wmf-config/InitialiseSettings.php  '([[bugzilla:40436|bug 40436]]) Namespaces configuration for se.wikipedia' | 
  [production] | 
            
  | 09:19 | 
  <hashar> | 
  synchronized wmf-config/InitialiseSettings.php  '([[bugzilla:40186|bug 40186]]) Favicon for es.wiktionary' | 
  [production] | 
            
  | 02:53 | 
  <LocalisationUpdate> | 
  completed (1.20wmf11) at Mon Sep 24 02:53:05 UTC 2012 | 
  [production] | 
            
  | 02:27 | 
  <LocalisationUpdate> | 
  completed (1.20wmf12) at Mon Sep 24 02:27:55 UTC 2012 | 
  [production] | 
            
  | 01:53 | 
  <Tim> | 
  on hume: running updateCollation.php with --force for [[bugzilla:40462|bug 40462]] | 
  [production] | 
            
  | 01:50 | 
  <Tim> | 
  depooled precise appservers and stopped job runners on precise servers | 
  [production] | 
            
  
    | 
      
        2012-09-21
      
      §
     | 
  
    
  | 20:55 | 
  <notpeter> | 
  removing srv190 from apaches pool to use to test build as an imagescaler | 
  [production] | 
            
  | 20:38 | 
  <asher> | 
  synchronized wmf-config/db.php  'moving ES writes from es1 (cluster23) to es2 (cluster24) and es3 (cluster25)' | 
  [production] | 
            
  | 20:29 | 
  <asher> | 
  synchronized wmf-config/db.php  'db.php with new es shards added, but not being written to' | 
  [production] | 
            
  | 20:29 | 
  <^demon> | 
  restarted gerrit like 10m ago to deploy I03de7c74 (apache timeout increase) | 
  [production] | 
            
  | 20:17 | 
  <Jeff_Green> | 
  apt-get upgrade on manganese, restart apache for adjusted 5->10m gerrit timeout | 
  [production] | 
            
  | 20:15 | 
  <mutante> | 
  analytics1007 seems dead. powercycle but never see any console output on mgmt at all | 
  [production] | 
            
  | 14:49 | 
  <hashar> | 
  labs: cleaned out a 4GB file out of labs-nfs1:/export/home/deployment-prep | 
  [production] | 
            
  | 02:49 | 
  <LocalisationUpdate> | 
  completed (1.20wmf11) at Fri Sep 21 02:49:27 UTC 2012 | 
  [production] | 
            
  | 02:25 | 
  <LocalisationUpdate> | 
  completed (1.20wmf12) at Fri Sep 21 02:25:33 UTC 2012 | 
  [production] | 
            
  | 01:39 | 
  <krinkle> | 
  synchronized php-1.20wmf12/extensions/LiquidThreads  'I5a098a42' | 
  [production] | 
            
  | 01:37 | 
  <krinkle> | 
  synchronized php-1.20wmf12/skins/common/preview.js  'Ia9673a75' | 
  [production] | 
            
  | 00:16 | 
  <binasher> | 
  configured pt-heartbeat on es2 and es3 shards | 
  [production] | 
            
  
    | 
      
        2012-09-20
      
      §
     | 
  
    
  | 23:18 | 
  <kaldari> | 
  synchronized php-1.20wmf12/extensions/PageTriage/modules/ext.pageTriage.defaultTagsOptions/ext.pageTriage.defaultTagsOptions.js  | 
  [production] | 
            
  | 23:17 | 
  <kaldari> | 
  synchronized php-1.20wmf11/extensions/PageTriage/modules/ext.pageTriage.defaultTagsOptions/ext.pageTriage.defaultTagsOptions.js  | 
  [production] | 
            
  | 22:58 | 
  <Ryan_Lane> | 
  restarting gerrit | 
  [production] | 
            
  | 22:36 | 
  <notpeter> | 
  stopping puppet on brewster | 
  [production] | 
            
  | 22:20 | 
  <aaron> | 
  synchronized wmf-config/filebackend.php  | 
  [production] | 
            
  | 22:18 | 
  <aaron> | 
  synchronized wmf-config/filebackend.php  'Enabled short thumbnails for testwiki' | 
  [production] | 
            
  | 22:15 | 
  <olivneh> | 
  synchronized php-1.20wmf11/extensions/E3Experiments/experiments  | 
  [production] | 
            
  | 22:02 | 
  <kaldari> | 
  Finished syncing Wikimedia installation... :  | 
  [production] | 
            
  | 21:32 | 
  <kaldari> | 
  Started syncing Wikimedia installation... :  | 
  [production] | 
            
  | 20:32 | 
  <ottomata> | 
  reinstalling analytics1002-analytics1010 with Ubuntu precise | 
  [production] | 
            
  | 20:25 | 
  <aaron> | 
  synchronized php-1.20wmf12/includes/filebackend/SwiftFileBackend.php  | 
  [production] | 
            
  | 20:25 | 
  <aaron> | 
  synchronized php-1.20wmf11/includes/filebackend/SwiftFileBackend.php  | 
  [production] | 
            
  | 20:13 | 
  <mutante> | 
  bugs.mediawiki.org|bugzilla.wikipedia.org|bugzilla.mediawiki.org all redirect to https:// rather than http now | 
  [production] | 
            
  | 20:12 | 
  <dzahn> | 
  gracefulled all apaches | 
  [production] |