| 
      
        2013-06-08
      
      §
     | 
  
    
  | 23:21 | 
  <RoanKattouw> | 
  Restarted Parsoid on a few machines where it was stuck | 
  [production] | 
            
  | 23:01 | 
  <Jamesofur> | 
  synced Board Election translations on cluster | 
  [production] | 
            
  | 16:38 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Actually enable EducationProgram on the swedish wikipedia' | 
  [production] | 
            
  | 10:40 | 
  <Nemo_bis> | 
  git.wikimedia.org 503: Service Temporarily Unavailable | 
  [production] | 
            
  | 07:54 | 
  <Jamesofur> | 
  synced Board Election translations on cluster | 
  [production] | 
            
  | 02:25 | 
  <reedy> | 
  synchronized php-1.22wmf5/extensions/SecurePoll/ | 
  [production] | 
            
  | 02:24 | 
  <reedy> | 
  synchronized php-1.22wmf6/extensions/SecurePoll/ | 
  [production] | 
            
  | 02:19 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Sat Jun  8 02:19:43 UTC 2013 | 
  [production] | 
            
  | 02:08 | 
  <LocalisationUpdate> | 
  completed (1.22wmf6) at Sat Jun  8 02:08:16 UTC 2013 | 
  [production] | 
            
  | 02:06 | 
  <LocalisationUpdate> | 
  completed (1.22wmf5) at Sat Jun  8 02:05:56 UTC 2013 | 
  [production] | 
            
  | 01:37 | 
  <reedy> | 
  synchronized php-1.22wmf6/extensions/SecurePoll/ | 
  [production] | 
            
  | 01:36 | 
  <reedy> | 
  synchronized php-1.22wmf5/extensions/SecurePoll/ | 
  [production] | 
            
  | 00:59 | 
  <RoanKattouw> | 
  Restarting Parsoid on all backends | 
  [production] | 
            
  | 00:53 | 
  <RoanKattouw> | 
  Restarted Apache on gallium AGAIN. Patch for the underlying issue is https://gerrit.wikimedia.org/r/#/c/67551/ | 
  [production] | 
            
  
    | 
      
        2013-06-07
      
      §
     | 
  
    
  | 23:08 | 
  <RoanKattouw> | 
  Restarted Apache on gallium agai | 
  [production] | 
            
  | 22:46 | 
  <gwicke> | 
  update Parsoid to 9ccf7a3365 | 
  [production] | 
            
  | 22:45 | 
  <RoanKattouw> | 
  and on cerium and titanium too | 
  [production] | 
            
  | 22:45 | 
  <RoanKattouw> | 
  Rerunning puppet on nickel for parsoidcache change | 
  [production] | 
            
  | 22:39 | 
  <LeslieCarr> | 
  restart snmptrapd and smptt on neon | 
  [production] | 
            
  | 22:23 | 
  <gwicke> | 
  synchronized wmf-config/CommonSettings.php | 
  [production] | 
            
  | 22:02 | 
  <gwicke> | 
  synchronized wmf-config/CommonSettings.php | 
  [production] | 
            
  | 21:57 | 
  <reedy> | 
  synchronized php-1.22wmf6/extensions/EducationProgram/ | 
  [production] | 
            
  | 21:56 | 
  <reedy> | 
  synchronized php-1.22wmf5/extensions/EducationProgram/ | 
  [production] | 
            
  | 21:48 | 
  <demon> | 
  synchronized wmf-config/CommonSettings.php  'Enable REL1_21 for extdist' | 
  [production] | 
            
  | 21:45 | 
  <cmjohnson1> | 
  auth-dns update | 
  [production] | 
            
  | 21:20 | 
  <mutante> | 
  installing package upgrades on neon now that it has inodes again | 
  [production] | 
            
  | 21:07 | 
  <cmjohnson1> | 
  auth-dns update removing tampa parsoid dns entries | 
  [production] | 
            
  | 21:05 | 
  <mutante> | 
  neon ran out of inodes, culprit /var/spool/snmptt has >300k files, deleting -mtime +2 in batches | 
  [production] | 
            
  | 20:48 | 
  <RoanKattouw> | 
  Actually restarting pybal on lvs4 now | 
  [production] | 
            
  | 20:48 | 
  <RoanKattouw> | 
  Running puppet on lvs4 | 
  [production] | 
            
  | 20:46 | 
  <RoanKattouw> | 
  Restarting pybal on lvs4 | 
  [production] | 
            
  | 20:32 | 
  <RoanKattouw> | 
  Restarting pybal on lvs3 | 
  [production] | 
            
  | 19:49 | 
  <aaron> | 
  synchronized wmf-config/jobqueue-eqiad.php  'Tweaked redis queue timeout and enabled gzip' | 
  [production] |