| 2010-03-03
      
      § | 
    
  | 16:16 | <apergos> | reenables gmetric on ms8 a little bit ago, scrub finished with no errors (yeah right( | [production] | 
            
  | 06:06 | <apergos> | replication off again tonight, zfs nicely borked on ms8, zfs destroy snapshot X gives message "cannot destroy X, dataset already exists" .  time for sun contract and call them | [production] | 
            
  | 05:53 | <apergos> | gmetrics turned off again on ms8, running zpool scrub export | [production] | 
            
  | 05:40 | <apergos> | reset SYs on ms8 worked (slowly).  gmetrics reenabled.  (and I thought morebots was just being a bad bot) | [production] | 
            
  | 05:39 | <apergos> | ( :-P ) starting catchup for the last day's lost replication ms7 -> ms8 | [production] | 
            
  | 05:22 | <apergos> | reboot failed, bootadm hung probably waiting on zfs. stop SYS claims to work but subsequent show SYS shows power state on. | [production] | 
            
  | 05:14 | <apergos> | rebootiing ms8 to clear zfs deadlock | [production] | 
            
  
    | 2010-03-02
      
      § | 
    
  | 23:32 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/OptIn/SpecialOptIn.php  'r63191' | [production] | 
            
  | 23:31 | <catrope> | synchronized php-1.5/extensions/UsabilityInitiative/OptIn/OptIn.php  'r63191' | [production] | 
            
  | 23:31 | <catrope> | synchronized php-1.5/wmf-config/CommonSettings.php  'Tweak OptIn settings' | [production] | 
            
  | 22:36 | <RoanKattouw> | Rebuilding localisation cache | [production] | 
            
  | 20:15 | <Tim> | on hume: started recompressTracked.php for all wikis, to fix any newly-tracked pointers from fixBug20757.php and to complete enwiki | [production] | 
            
  | 03:48 | <atglenn> | leaving replication and gmetrics turned off on ms8 for thenight, will go back to looking at scat and mdb output tomorrow. | [production] | 
            
  | 02:27 | <atglenn> | turning off gmetric on ms8, the dfs are piling up behind a deadlocked zfs list | [production] | 
            
  | 02:13 | <atglenn> | turn of ms7 replication to ms8 temporarily to work out a zfs deadlock issue on ms8 | [production] | 
            
  
    | 2010-02-26
      
      § | 
    
  | 23:40 | <Tim> | disabled the temporary squid purge script, I'm assuming it's done its job now | [production] | 
            
  | 19:43 | <catrope> | synchronized php-1.5/includes/api/ApiBase.php  'r63014, r63015' | [production] | 
            
  | 18:24 | <Fred> | bugzilla now residing on Kaulen... Updating server roles. | [production] | 
            
  | 18:07 | <Fred> | switching bugzilla cname to new server. (both servers are pointing to the same DB, so switch should be transparent to most) | [production] | 
            
  | 17:28 | <midom> | synchronized php-1.5/includes/LocalisationCache.php  'profiling' | [production] | 
            
  | 17:26 | <midom> | synchronized php-1.5/includes/diff/DifferenceInterface.php  'removing profiling hook' | [production] | 
            
  | 17:23 | <midom> | synchronized php-1.5/includes/StubObject.php  'tracehooks' | [production] | 
            
  | 17:21 | <midom> | synchronized php-1.5/includes/StubObject.php  'tracehooks' | [production] | 
            
  | 14:38 | <RoanKattouw> | logmsgbot is missing in action AGAIN, starting it as me on fenari | [production] | 
            
  | 14:37 | <RoanKattouw> | Synced r63008 | [production] | 
            
  | 12:17 | <domas> | *sigh*, recovered locke after OOM, added few entries to limits.conf | [production] |