| 
      
        2010-08-01
      
      §
     | 
  
    
  | 17:21 | 
  <mark> | 
  Reinstalled linne.wikimedia.org with Lucid | 
  [production] | 
            
  | 15:41 | 
  <mark> | 
  Fixed the haproxy for puppetmaster on brewster, was broken by the upgrade | 
  [production] | 
            
  | 15:19 | 
  <mark> | 
  Upgraded brewster to Ubuntu 10.04 Lucid | 
  [production] | 
            
  | 15:18 | 
  <mark> | 
  Removed Wikimedia repository default pinning on brewster, as it's doing more harm than good | 
  [production] | 
            
  | 15:07 | 
  <mark> | 
  Explicitly install and deinstall gmond / ganglia-monitor packages in puppet, depending on the ubuntu version | 
  [production] | 
            
  | 14:53 | 
  <mark> | 
  Removed broken amanda backup client from brewster. Broken package install/dpkg state, no logs, no documentation | 
  [production] | 
            
  
    | 
      
        2010-07-28
      
      §
     | 
  
    
  | 15:26 | 
  <Rob> | 
  running updates on sockpuppet | 
  [production] | 
            
  | 13:08 | 
  <andrew> | 
  synchronizing Wikimedia installation... Revision: 70064 | 
  [production] | 
            
  | 12:57 | 
  <Andrew> | 
  about to update LiquidThreads alpha to trunk state | 
  [production] | 
            
  | 07:54 | 
  <tstarling> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'adding 1.16 branch to ExtensionDistributor' | 
  [production] | 
            
  | 05:01 | 
  <apergos> | 
  added bayes to list of clients for exports on dataset1; added /data to fstab on bayes for stats use | 
  [production] | 
            
  | 04:23 | 
  <tstarling> | 
  synchronized php-1.5/includes/api/ApiBase.php  | 
  [production] | 
            
  | 04:22 | 
  <tstarling> | 
  synchronized php-1.5/includes/api/ApiMain.php  | 
  [production] | 
            
  | 04:22 | 
  <Tim> | 
  deploying r70063 and r70064 to fix API fatals | 
  [production] | 
            
  | 02:09 | 
  <root> | 
  synchronizing Wikimedia installation... Revision: 70061 | 
  [production] | 
            
  | 02:04 | 
  <Tim> | 
  doing svn up/scap to r70061, to get the API cache header fix | 
  [production] | 
            
  
    | 
      
        2010-07-27
      
      §
     | 
  
    
  | 23:27 | 
  <robh> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'bug 24466' | 
  [production] | 
            
  | 21:39 | 
  <mark> | 
  Added wikimedia-base to the standard packages list in puppet | 
  [production] | 
            
  | 21:34 | 
  <Rob> | 
  pushed hu language file change on survey.wikimedia.org | 
  [production] | 
            
  | 21:16 | 
  <Rob> | 
  singer is being a pain, ssh isnt running, so forth... working on it | 
  [production] | 
            
  | 19:07 | 
  <Rob> | 
  racktables physical audit of pmtpa done (ignore the bottom of rack a5, will remove them later today when I have a damned mouse) | 
  [production] | 
            
  | 18:50 | 
  <Rob> | 
  physical audit pmtpa row b complete | 
  [production] | 
            
  | 18:40 | 
  <Rob> | 
  physical audit pmtpa row c complete | 
  [production] | 
            
  | 18:18 | 
  <Rob> | 
  uplink successfully moved for asw-b5-sdtpa | 
  [production] | 
            
  | 18:17 | 
  <Rob> | 
  moving the uplink | 
  [production] | 
            
  | 18:17 | 
  <Rob> | 
  all the apaches in sdtpa-b5 are migrated from old to new asw-b5-sdtpa. | 
  [production] | 
            
  | 18:05 | 
  <Rob> | 
  bugzilla admin note: disregard the email bounces for the scireview domain user, their mail server will be fixed shortly | 
  [production] | 
            
  | 17:58 | 
  <Rob> | 
  srv284 fixed, drac online, needs setup | 
  [production] | 
            
  | 17:54 | 
  <Rob> | 
  !log srv281 was shutdown when I came into the DC.  popped case to use as base for rebuilding another system.  need to investigate its initial shutdown | 
  [production] | 
            
  | 17:17 | 
  <mark> | 
  Reinstalled sq68 | 
  [production] | 
            
  | 16:41 | 
  <atglenn> | 
  started up several more workers on snapshot3 doing xml dumps | 
  [production] | 
            
  | 14:43 | 
  <Tim> | 
  running svn cleanup on some ExtensionDistributor working copies | 
  [production] | 
            
  | 14:29 | 
  <Tim> | 
  cleaning up old ext-dist tarballs, removing all that were older than a month | 
  [production] | 
            
  | 11:48 | 
  <mark> | 
  Stracing puppet on spence | 
  [production] | 
            
  | 08:01 | 
  <apergos> | 
  started one thread of xml dumps from screen session on snapshot3 as root, if these look good tomorrow we'll crank up more of 'em | 
  [production] |