| 
      
        2010-03-08
      
      §
     | 
  
    
  | 21:41 | 
  <aaron> | 
  synchronized php-1.5/wmf-config/flaggedrevs.php  | 
  [production] | 
            
  | 21:38 | 
  <aaron> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  | 
  [production] | 
            
  | 21:36 | 
  <aaron> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'disable LU on labs sites' | 
  [production] | 
            
  | 21:09 | 
  <catrope> | 
  synchronized php-1.5/includes/LocalisationCache.php  'r63417' | 
  [production] | 
            
  | 20:57 | 
  <aaron> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'revert, no effect on msg issue' | 
  [production] | 
            
  | 20:53 | 
  <aaron> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'settings labs cache dir' | 
  [production] | 
            
  | 20:31 | 
  <aaron> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'Switched labs sites to FlaggedRevs_alpha' | 
  [production] | 
            
  | 20:30 | 
  <aaron> | 
  synchronized php-1.5/wmf-config/flaggedrevs.php  'custom cache dir config for labs sites' | 
  [production] | 
            
  | 19:18 | 
  <tstarling> | 
  synchronized wmf-deployment/cache/trusted-xff.cdb  | 
  [production] | 
            
  | 18:15 | 
  <Fred> | 
  disabled regular http check for mobile1 (leaving the longer timeout one in place) | 
  [production] | 
            
  | 13:58 | 
  <catrope> | 
  synchronized php-1.5/cache/l10n/l10nupdate-et.cache  'Sync file *after* touching it' | 
  [production] | 
            
  | 13:55 | 
  <catrope> | 
  synchronized php-1.5/cache/l10n/l10nupdate-et.cache  'Sync LU cache file' | 
  [production] | 
            
  | 13:52 | 
  <catrope> | 
  synchronized php-1.5/languages/messages/MessagesEt.php  'Try again' | 
  [production] | 
            
  | 13:48 | 
  <catrope> | 
  synchronized php-1.5/languages/messages/MessagesNds_nl.php  'Fix message that didn't get picked up' | 
  [production] | 
            
  | 13:46 | 
  <catrope> | 
  synchronized php-1.5/languages/messages/MessagesEt.php  'See if this works' | 
  [production] | 
            
  | 13:20 | 
  <RoanKattouw> | 
  Rebuilding all l10n caches to prep debugging a LocalisationUpdate issue | 
  [production] | 
            
  | 13:16 | 
  <domas> | 
  something was wrong with sq31/sq32 - cpu/network shot through roof, can't correlate load | 
  [production] | 
            
  | 10:42 | 
  <domas> | 
  *sigh*, squid conf: added highPrecedence to defaults, so it is picked up by configuration builder. also, checked in files into RCS (had to take over locks from mr.Starling, sorry!). unbroke test.wikipedia API | 
  [production] | 
            
  | 09:43 | 
  <domas> | 
  where did test api,php fix go?!  | 
  [production] | 
            
  | 04:38 | 
  <Fred> | 
  removed Bart's ACL in squid cache conf since it was a duplicate ACL.  | 
  [production] | 
            
  
    | 
      
        2010-03-03
      
      §
     | 
  
    
  | 16:16 | 
  <apergos> | 
  reenables gmetric on ms8 a little bit ago, scrub finished with no errors (yeah right(  | 
  [production] | 
            
  | 06:06 | 
  <apergos> | 
  replication off again tonight, zfs nicely borked on ms8, zfs destroy snapshot X gives message "cannot destroy X, dataset already exists" .  time for sun contract and call them | 
  [production] | 
            
  | 05:53 | 
  <apergos> | 
  gmetrics turned off again on ms8, running zpool scrub export  | 
  [production] | 
            
  | 05:40 | 
  <apergos> | 
  reset SYs on ms8 worked (slowly).  gmetrics reenabled.  (and I thought morebots was just being a bad bot) | 
  [production] | 
            
  | 05:39 | 
  <apergos> | 
  ( :-P ) starting catchup for the last day's lost replication ms7 -> ms8  | 
  [production] | 
            
  | 05:22 | 
  <apergos> | 
  reboot failed, bootadm hung probably waiting on zfs. stop SYS claims to work but subsequent show SYS shows power state on. | 
  [production] | 
            
  | 05:14 | 
  <apergos> | 
  rebootiing ms8 to clear zfs deadlock  | 
  [production] | 
            
  
    | 
      
        2010-03-02
      
      §
     | 
  
    
  | 23:32 | 
  <catrope> | 
  synchronized php-1.5/extensions/UsabilityInitiative/OptIn/SpecialOptIn.php  'r63191' | 
  [production] | 
            
  | 23:31 | 
  <catrope> | 
  synchronized php-1.5/extensions/UsabilityInitiative/OptIn/OptIn.php  'r63191' | 
  [production] | 
            
  | 23:31 | 
  <catrope> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'Tweak OptIn settings' | 
  [production] | 
            
  | 22:36 | 
  <RoanKattouw> | 
  Rebuilding localisation cache | 
  [production] | 
            
  | 20:15 | 
  <Tim> | 
  on hume: started recompressTracked.php for all wikis, to fix any newly-tracked pointers from fixBug20757.php and to complete enwiki | 
  [production] | 
            
  | 03:48 | 
  <atglenn> | 
  leaving replication and gmetrics turned off on ms8 for thenight, will go back to looking at scat and mdb output tomorrow.   | 
  [production] | 
            
  | 02:27 | 
  <atglenn> | 
  turning off gmetric on ms8, the dfs are piling up behind a deadlocked zfs list | 
  [production] | 
            
  | 02:13 | 
  <atglenn> | 
  turn of ms7 replication to ms8 temporarily to work out a zfs deadlock issue on ms8 | 
  [production] |