| 
      
        2009-08-11
      
      §
     | 
  
    
  | 23:38 | 
  <atglenn> | 
  new backend cache file on sq1 for testing | 
  [production] | 
            
  | 22:52 | 
  <brion> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  'reenable EditWarning now bug is fixed' | 
  [production] | 
            
  | 22:28 | 
  <atglenn> | 
  sq4 back in  | 
  [production] | 
            
  | 21:56 | 
  <atglenn> | 
  take sq4 out of upload pool temporarily for thumb testing | 
  [production] | 
            
  | 20:56 | 
  <AaronSchulz> | 
  Set $wgFlaggedRevTags/$wgFlaggedRevValues on dewikiqoute per bug 19179 | 
  [production] | 
            
  | 20:44 | 
  <aaron> | 
  synchronized php-1.5/wmf-config/flaggedrevs.php  | 
  [production] | 
            
  | 18:53 | 
  <tomaszf> | 
  pulling optin_survey table from enwiki on db28 for Roan | 
  [production] | 
            
  | 18:12 | 
  <domas> | 
  started rsync to storage1 from ms3, in root@ms3's screen | 
  [production] | 
            
  | 18:04 | 
  <mark> | 
  Prepared storage1 for emergency copy of ms3 data | 
  [production] | 
            
  | 18:01 | 
  <RobH_A90> | 
  srv122 shutdown due to broken cooling fan | 
  [production] | 
            
  | 16:30 | 
  <RobH_A90> | 
  db12 back up, restarted mysql | 
  [production] | 
            
  | 16:13 | 
  <mark> | 
  Shutting down MySQL on ms2 | 
  [production] | 
            
  | 16:03 | 
  <domas> | 
  db12 is now down for raid battery replacement (thanks cobi for noticing that) | 
  [production] | 
            
  | 16:01 | 
  <RobH_A90> | 
  shutting down db12 for raid battery replacement | 
  [production] | 
            
  | 15:59 | 
  <RobH_A90> | 
  shutting down mysql on db12 for raid battery replacement | 
  [production] | 
            
  | 15:58 | 
  <robh> | 
  synchronized php-1.5/wmf-config/db-pmtpa.php  'depooling db12 to replace raid controller battery' | 
  [production] | 
            
  | 15:53 | 
  <RobH_A90> | 
  sq49 shutdown and disks 3/4 replaced with ssd for mark | 
  [production] | 
            
  | 12:39 | 
  <domas> | 
  powercycled srv156/srv159, seems like job memory use skyrocketed, and conflicted with apache/memcached/puppet/etc  | 
  [production] | 
            
  | 07:39 | 
  <domas> | 
  db3 and db8 had same server_id, so, replication worked, just with reconnect after every event.. :)  | 
  [production] | 
            
  | 06:59 | 
  <tstarling> | 
  synchronized php-1.5/wmf-config/db-pmtpa.php  'repooling db8, not as broken as I thought' | 
  [production] | 
            
  | 06:41 | 
  <Tim> | 
  restarting the slave thread on db3 since the error log on db13 shows an error flood due to that slave "Start client, asynchronous binlog_dump to slave_server(100236)" | 
  [production] | 
            
  | 06:27 | 
  <Tim> | 
  depooled db8 due to broken replication | 
  [production] | 
            
  | 06:27 | 
  <tstarling> | 
  synchronized php-1.5/wmf-config/db-pmtpa.php  | 
  [production] | 
            
  | 06:20 | 
  <Tim> | 
  investigating replication issues on db8, tried restarting slave thread, no improvement | 
  [production] | 
            
  | 05:48 | 
  <Tim> | 
  disabled EditWarning due to bug 20171 | 
  [production] | 
            
  | 05:47 | 
  <tstarling> | 
  synchronized php-1.5/wmf-config/CommonSettings.php  | 
  [production] | 
            
  
    | 
      
        2009-08-07
      
      §
     | 
  
    
  | 23:20 | 
  <brion> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  'updating sr wikinews logo for 10k event per dungodung' | 
  [production] | 
            
  | 22:09 | 
  <domas> | 
  brought up db18 after some maintenance hybernation | 
  [production] | 
            
  | 22:09 | 
  <domas> | 
  db8 raid toasted | 
  [production] | 
            
  | 22:01 | 
  <brion> | 
  synchronized php-1.5/wmf-config/flaggedrevs.php  'add unreviewedpages, autoreview to coders on MWW' | 
  [production] | 
            
  | 21:54 | 
  <brion> | 
  synchronized php-1.5/wmf-config/flaggedrevs.php  'let coders review and validate on FR pages on mediawikiwiki' | 
  [production] | 
            
  | 21:33 | 
  <brion> | 
  synchronized php-1.5/wmf-config/flaggedrevs.php  | 
  [production] | 
            
  | 21:30 | 
  <brion> | 
  synchronized php-1.5/wmf-config/InitialiseSettings.php  | 
  [production] | 
            
  | 21:27 | 
  <brion> | 
  syncing dblist (sync-dblist now fixed... i hope) | 
  [production] | 
            
  | 21:21 | 
  <brion> | 
  synchronized php-1.5/wmf-config/flaggedrevs.php  'testing flaggedrevs limited config on MW wiki' | 
  [production] | 
            
  | 20:11 | 
  <Rob> | 
  removed UsernameBlacklist from CommonSettings. | 
  [production] |