| 2010-08-15
      
      § | 
    
  | 22:42 | <domas> | resynced db11 and db17 from db27, db33 from ixia, db19 from db1, with accompanying BIOS flashing to 3.0 and OS reinstalls. decommissioned ixia and db1 | [production] | 
            
  | 22:40 | <midom> | synchronized php-1.5/wmf-config/db.php  'drumrolllll' | [production] | 
            
  | 22:04 | <midom> | synchronized php-1.5/wmf-config/db.php | [production] | 
            
  | 21:56 | <midom> | synchronized php-1.5/wmf-config/db.php | [production] | 
            
  | 21:49 | <midom> | synchronized php-1.5/wmf-config/db.php | [production] | 
            
  | 21:39 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24520 - Transwiki import source for ml.wikiquote.org' | [production] | 
            
  | 20:55 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24458 - Enable subpages on frwikisource' | [production] | 
            
  | 20:39 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24623 - Enable 'eliminator' flag on ptwiki' | [production] | 
            
  | 20:36 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24304 - Reconfigure English Wikibooks' | [production] | 
            
  | 20:36 | <jeluf> | synchronized php-1.5/wmf-config/flaggedrevs.php  '24304 - reconfigure enwikibooks' | [production] | 
            
  | 20:35 | <jeluf> | synchronized php-1.5/wmf-config/abusefilter.php  '24304 - reconfigure enwikibooks' | [production] | 
            
  | 20:32 | <midom> | synchronized php-1.5/wmf-config/db.php | [production] | 
            
  | 20:14 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24623 - Enable 'eliminator' flag on ptwiki' | [production] | 
            
  | 20:09 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24321 - ml.wikiquote.org lost its project namespace' | [production] | 
            
  | 20:04 | <midom> | synchronized php-1.5/wmf-config/db.php | [production] | 
            
  | 19:54 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24777 - Request for a patrolling function on the Nynorsk (nn) Wikipedia' | [production] | 
            
  | 19:48 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24535 - Enable "mark as patrolled" feature in hindi wiki' | [production] | 
            
  | 19:44 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24394 - Install AbuseFilter on Hindi Wikipedia' | [production] | 
            
  | 19:41 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24336 - Change Simplewiki's (EN) autoconfirm time/edit rates' | [production] | 
            
  | 19:36 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24790 - Localize Wikipedia sitename in devanagari' | [production] | 
            
  | 19:33 | <JeLuF> | fixed zero byte thumbnail of commons:Shkval_head.jpg | [production] | 
            
  | 19:19 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24789 - Enable AbuseFilter for ja.wikipedia' | [production] | 
            
  | 19:18 | <jeluf> | synchronized php-1.5/wmf-config/abusefilter.php  '24789 -  Enable AbuseFilter for ja.wikipedia' | [production] | 
            
  | 19:14 | <midom> | synchronized php-1.5/wmf-config/db.php | [production] | 
            
  | 18:44 | <midom> | synchronized php-1.5/wmf-config/db.php | [production] | 
            
  | 18:24 | <midom> | synchronized php-1.5/wmf-config/db.php | [production] | 
            
  | 17:44 | <midom> | synchronized php-1.5/wmf-config/db.php | [production] | 
            
  | 16:23 | <midom> | synchronized php-1.5/wmf-config/db.php | [production] | 
            
  
    | 2010-08-14
      
      § | 
    
  | 20:15 | <mark> | Decommissioning srv150 | [production] | 
            
  | 19:56 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24789 - Enable AbuseFilter for ja.wikipedia' | [production] | 
            
  | 19:52 | <jeluf> | synchronized php-1.5/wmf-config/InitialiseSettings.php  '24626 - Add an "autopatrolled" status for frwiktionary' | [production] | 
            
  | 15:37 | <mark> | dobson has failed RAID1 array member /dev/sda. Running long SMART self test on /dev/sda | [production] | 
            
  | 14:18 | <mark> | synchronized php-1.5/wmf-config/db.php  'Add ms2 and ms1 to clusters rc1 an cluster22' | [production] | 
            
  | 14:06 | <mark> | FLUSH TABLES WITH READ LOCK on ms1 for testing | [production] | 
            
  | 13:59 | <mark> | Stopping mysql on ms1 as monitoring test | [production] | 
            
  | 13:59 | <mark> | Granted SELECT on mysql.* to nagios on ms3 | [production] | 
            
  | 10:57 | <mark> | Removed oldest LVM snapshot on ixia | [production] | 
            
  | 09:43 | <mark> | Fixed apparmor profile /etc/apparmor.d/usr.sbin.mysqld on ms1, restarted mysql under apparmor | [production] | 
            
  | 09:39 | <mark> | START SLAVE on ms1, catching up with ms3 | [production] | 
            
  | 09:38 | <mark> | RESET SLAVE on db5 | [production] | 
            
  | 09:37 | <mark> | STOP SLAVE on db5 | [production] | 
            
  | 09:35 | <mark> | Stopped apparmor on ms1 | [production] | 
            
  | 08:41 | <Andrew> | Leaving as-is for now, hoping somebody with appropriate permissions can fix it later. | [production] | 
            
  | 08:40 | <Andrew> | STOP SLAVE on db5 gives me ERROR 1045 (00000): Access denied for user: 'wikiadmin@208.80.152.%' (Using password: NO) | [production] | 
            
  | 08:34 | <Andrew> | Slave is supposedly still running on db5. Assuming Roan didn't stop it when he switched masters a few days ago. Going to text somebody to confirm that stopping is correct course of action. | [production] | 
            
  | 08:24 | <Andrew> | db5 can't be lagged, it's the master ;-). Obviously something wrong with wfWaitForSlaves. | [production] | 
            
  | 08:19 | <Andrew> | db5 lagged 217904 seconds | [production] | 
            
  | 05:09 | <Andrew> | Ran thread_pending_relationship and thread_reaction schema changes on all LiquidThreads wikis | [production] |