| 
      
        2012-07-05
      
      §
     | 
  
    
  | 23:38 | 
  <Ryan_Lane> | 
  upgrading gerrit on formey | 
  [production] | 
            
  | 23:21 | 
  <Ryan_Lane> | 
  updating database for gerrit | 
  [production] | 
            
  | 23:09 | 
  <Ryan_Lane> | 
  upgrading gerrit on manganese | 
  [production] | 
            
  | 23:08 | 
  <Ryan_Lane> | 
  stopping gerrit on manganese and disabling puppet | 
  [production] | 
            
  | 23:07 | 
  <Ryan_Lane> | 
  stopping gerrit on formey and disabling puppet | 
  [production] | 
            
  | 22:47 | 
  <preilly> | 
  synchronized wmf-config/mobile.php  'add subdomain check' | 
  [production] | 
            
  | 22:34 | 
  <reedy> | 
  synchronized wmf-config/  | 
  [production] | 
            
  | 22:17 | 
  <reedy> | 
  synchronized wmf-config/  'Tidying config for randomrootpage' | 
  [production] | 
            
  | 21:56 | 
  <reedy> | 
  synchronized wmf-config/  'Strike 2' | 
  [production] | 
            
  | 21:50 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'revert' | 
  [production] | 
            
  | 21:48 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Random root page enabled everywhere but wikipedias' | 
  [production] | 
            
  | 21:47 | 
  <kaldari> | 
  Finished syncing Wikimedia installation... :  | 
  [production] | 
            
  | 21:34 | 
  <kaldari> | 
  Started syncing Wikimedia installation... :  | 
  [production] | 
            
  | 21:18 | 
  <kaldari> | 
  synchronized wmf-config/InitialiseSettings.php  'Bumping AFTv5 lottery percentage to 20% of en.wiki' | 
  [production] | 
            
  | 21:15 | 
  <LeslieCarr> | 
  powercycling unresponsive mw1116 | 
  [production] | 
            
  | 21:09 | 
  <maplebed> | 
  added new ms-be pmtpa hosts to DNS | 
  [production] | 
            
  | 20:03 | 
  <preilly> | 
  synchronized php-1.20wmf6/extensions/MobileFrontend  'fix cache issue with token' | 
  [production] | 
            
  | 19:14 | 
  <Jeff_Green> | 
  *somebody* purged binary logs on blondel | 
  [production] | 
            
  | 18:58 | 
  <preilly> | 
  synchronized php-1.20wmf6/extensions/MobileFrontend  'fix cache issue' | 
  [production] | 
            
  | 18:54 | 
  <mark> | 
  Upgraded pybal on all precise LVS servers | 
  [production] | 
            
  | 18:43 | 
  <mark> | 
  Built new pybal_1.03 package and inserted it into the precise-wikimedia APT repository | 
  [production] | 
            
  | 17:41 | 
  <mutante> | 
  powercycling db1048, mw1136, mw1046 | 
  [production] | 
            
  | 17:32 | 
  <midom> | 
  synchronized wmf-config/InitialiseSettings.php  'reenabling db40' | 
  [production] | 
            
  | 17:30 | 
  <mutante> | 
  more powercycling and upgrading: mw1036, mw1134, mw1043 .. | 
  [production] | 
            
  | 17:26 | 
  <mutante> | 
  powercycling db1009,db1010 | 
  [production] | 
            
  | 17:22 | 
  <mutante> | 
  powercycling db1027,db1028 | 
  [production] | 
            
  | 17:14 | 
  <mutante> | 
  powercycling db1013 | 
  [production] | 
            
  | 16:50 | 
  <mutante> | 
  powercycling downed db1015 | 
  [production] | 
            
  | 16:44 | 
  <mutante> | 
  powercycling and upgrading more mw10xx servers, 1017,1023,1025 ... | 
  [production] | 
            
  | 16:27 | 
  <mutante> | 
  mw1002, mw1007,mw1009,mw1011 - crashed,powercycling,dist-upgrading+kernel,reboot | 
  [production] | 
            
  | 16:18 | 
  <mutante> | 
  powercycling mw1002 | 
  [production] | 
            
  | 15:37 | 
  <mutante> | 
  argon back up with new kernel,mysql,grub,.. looks happy afaict | 
  [production] | 
            
  | 15:33 | 
  <mutante> | 
  argon (limesurvey) fscked, dist-upgrading | 
  [production] | 
            
  | 15:28 | 
  <mutante> | 
  powercycling argon | 
  [production] | 
            
  | 15:26 | 
  <hashar> | 
  updated Jenkins configuration on gallium : Updating f407ebe..4b669b9 | 
  [production] | 
            
  | 14:08 | 
  <cmjohnson1> | 
  HUME replacing disk 0 | 
  [production] | 
            
  | 14:08 | 
  <midom> | 
  synchronized wmf-config/InitialiseSettings.php  'disabling parser cache for now' | 
  [production] | 
            
  | 11:54 | 
  <mark> | 
  Inserted new pybal_1.02 package into APT distribution precise-wikimedia | 
  [production] | 
            
  | 11:17 | 
  <mark> | 
  Installed new pybal snapshot build for testing on lvs1005 | 
  [production] | 
            
  | 11:13 | 
  <mutante> | 
  add missing Russian locales on singer, run localegen, run ru.planet update | 
  [production] | 
            
  | 02:26 | 
  <LocalisationUpdate> | 
  completed (1.20wmf6) at Thu Jul  5 02:26:45 UTC 2012 | 
  [production] | 
            
  
    | 
      
        2012-07-04
      
      §
     | 
  
    
  | 16:32 | 
  <mutante> | 
  wikidata.org on now - redirect purged from squids | 
  [production] | 
            
  | 15:43 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'imports' | 
  [production] | 
            
  | 15:40 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Enable LQT on fiwikimedia' | 
  [production] | 
            
  | 15:39 | 
  <Reedy> | 
  Killed php-1.20wmf5 localisation cache from mediawiki-installation group | 
  [production] | 
            
  | 15:37 | 
  <Reedy> | 
  Created LQT tables on fiwikimedia | 
  [production] | 
            
  | 15:25 | 
  <mutante> | 
  wikidata.org works now, besides old redirect may still be cached on cp* boxes (not purged by purgeList.php via multicast?). http://www.wikidata.org/?notcached | 
  [production] | 
            
  | 15:06 | 
  <mutante> | 
  sync-common-file extract2.php, apache-graceful-all | 
  [production] | 
            
  | 15:04 | 
  <dzahn> | 
  synchronized extract2.php  | 
  [production] | 
            
  | 13:23 | 
  <mutante> | 
  apache-graceful-all to add wikidata.org virtual host | 
  [production] |