| 
      
        2011-03-27
      
      §
     | 
  
    
  | 22:28 | 
  <Ryan_Lane> | 
  restarting apache on singer | 
  [production] | 
            
  | 22:19 | 
  <Ryan_Lane> | 
  pushing new star cert to singer | 
  [production] | 
            
  | 21:55 | 
  <RoanKattouw> | 
  SSL certificate for secure.wikimedia.org expired about an hour ago, no ops around. I've just texted Ryan | 
  [production] | 
            
  | 19:52 | 
  <demon> | 
  synchronized php-1.17/extensions/ProofreadPage/ProofreadPage_body.php  'r84871' | 
  [production] | 
            
  | 14:14 | 
  <catrope> | 
  synchronized php-1.17/includes/resourceloader/ResourceLoaderModule.php  'r84845, hopefully fixes catchable fatal error about makeCombinedStyles()' | 
  [production] | 
            
  | 10:55 | 
  <hashar:> | 
  on [[searchidx1]] wmf-config path does not exist triggering an error on code sync | 
  [production] | 
            
  | 10:54 | 
  <hashar:> | 
  auto defers SocialProfile, editor_trends & WikiSnaps | 
  [production] | 
            
  | 10:54 | 
  <hashar> | 
  synchronized php-1.17/wmf-config/codereview.php  | 
  [production] | 
            
  
    | 
      
        2011-03-26
      
      §
     | 
  
    
  | 21:41 | 
  <ariel> | 
  synchronized php-1.17/extensions/Narayam/Narayam.i18n.php  'remove BOM from file, was being written to stdout' | 
  [production] | 
            
  | 19:13 | 
  <apergos> | 
  running rsync for White Cat again to pick up en wiki 20110115 hist files, on dataset2 in screen session as root, please shoot after 3 days (check with user first in channel) | 
  [production] | 
            
  | 15:05 | 
  <mark> | 
  synchronized php-1.17/wmf-config/db.php  'Repooled db38 with lower load (100 instead of 400)' | 
  [production] | 
            
  | 14:57 | 
  <mark> | 
  synchronized php-1.17/wmf-config/db.php  'Depooling db38' | 
  [production] | 
            
  | 10:55 | 
  <midom> | 
  synchronized php-1.17/wmf-config/mc.php  'srv206 to srv243 swap' | 
  [production] | 
            
  | 10:53 | 
  <domas> | 
  I misread before. memcached check was broken for 23 days, not 23 hours. duct-taped by creating symlink php-1.17 -> wmf-deployment on spence, not sure for how long :) | 
  [production] | 
            
  | 10:50 | 
  <domas> | 
  added 'pkill -9 -f smssend' hourly crontab on spence... | 
  [production] | 
            
  
    | 
      
        2011-03-25
      
      §
     | 
  
    
  | 15:36 | 
  <mark> | 
  Upgraded csw1-esams and br1-knams to IronWare 2.7.3a | 
  [production] | 
            
  | 15:31 | 
  <mark> | 
  Reloading br1-knams | 
  [production] | 
            
  | 15:09 | 
  <mark> | 
  Reloading csw1-esams | 
  [production] | 
            
  | 15:09 | 
  <demon> | 
  synchronized php-1.17/includes/HTMLForm.php  'r84748' | 
  [production] | 
            
  | 14:53 | 
  <mark> | 
  Upgraded JUNOS to 10.0S10.0 on csw2-esams | 
  [production] | 
            
  | 13:18 | 
  <mark> | 
  Moving esams traffic to pmtpa for upcoming router firmware upgrade/reload | 
  [production] | 
            
  | 04:49 | 
  <Tim> | 
  reniced apaches to nice level 0 again | 
  [production] | 
            
  | 03:53 | 
  <aaron> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  | 
  [production] | 
            
  | 03:51 | 
  <aaron> | 
  synchronized php-1.17/wmf-config/abusefilter.php  'Moved abusefilter rights here from InitialiseSettings.php' | 
  [production] | 
            
  | 03:36 | 
  <root> | 
  synchronizing Wikimedia installation... Revision: 84728:  | 
  [production] | 
            
  | 03:36 | 
  <Tim> | 
  svn up/scap to r84727 | 
  [production] | 
            
  | 02:45 | 
  <aaron> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'Added missing autopatrolled group to siwiki' | 
  [production] | 
            
  | 02:30 | 
  <aaron> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'Added missing rollbacker group to siwiki' | 
  [production] | 
            
  | 02:07 | 
  <aaron> | 
  synchronized php-1.17/wmf-config/flaggedrevs.php  'DO NOT PUT ABUSEFILTER SETTINGS IN HERE' | 
  [production] | 
            
  | 02:06 | 
  <aaron> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'Copy abusefilter right settings from flaggedrevs.php to here, were they belong' | 
  [production] | 
            
  | 01:55 | 
  <aaron> | 
  synchronized php-1.17/wmf-config/flaggedrevs.php  | 
  [production] | 
            
  | 01:24 | 
  <aaron> | 
  synchronized php-1.17/wmf-config/flaggedrevs.php  'Simple config cleanups' | 
  [production] | 
            
  | 00:48 | 
  <hcatlin> | 
  Started rolling restarts passenger/ruby cluster on mobile2 | 
  [production] | 
            
  
    | 
      
        2011-03-24
      
      §
     | 
  
    
  | 23:50 | 
  <tomaszf> | 
  lowering weight for mobile2 from 45 -> 30 on m. | 
  [production] | 
            
  | 22:51 | 
  <tfinc> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'adding missing redirects for mobile sites' | 
  [production] | 
            
  | 22:33 | 
  <aaron> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  | 
  [production] | 
            
  | 22:27 | 
  <aaron> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'bug 28195 - added two namespaces two sqwikinews' | 
  [production] | 
            
  | 20:23 | 
  <Ryan_Lane> | 
  fixing the authorized_keys file on ragweed | 
  [production] | 
            
  | 17:16 | 
  <rainman-sr> | 
  clearing up some space on searchidx1 | 
  [production] | 
            
  | 17:04 | 
  <RobH> | 
  hooper switch port vlan updated, system ready for installation | 
  [production] | 
            
  | 16:54 | 
  <tomaszf> | 
  rebalancing mobile1/2/3 traffic load to 15/40/60 | 
  [production] | 
            
  | 15:54 | 
  <catrope> | 
  synchronized php-1.17/extensions/Narayam/Narayam.i18n.php  'r84684' | 
  [production] | 
            
  | 15:54 | 
  <catrope> | 
  synchronized php-1.17/extensions/Narayam/Narayam.hooks.php  'r84684' | 
  [production] | 
            
  | 15:54 | 
  <catrope> | 
  synchronized php-1.17/extensions/Narayam/Narayam.php  'r84684' | 
  [production] | 
            
  | 15:40 | 
  <catrope> | 
  synchronized php-1.17/extensions/Gadgets/Gadgets_body.php  'r84683' | 
  [production] | 
            
  | 13:37 | 
  <thedj> | 
  ResourceLoader broken due to bug 28215 | 
  [production] |