| 2013-12-23
      
      § | 
    
  | 22:15 | <awight> | updated crm from ce73eaa0ea6019f76c63d0cca097bf828eb933e2 to 87dea8afb964d0622f3ee92b6e120915846fde56 | [production] | 
            
  | 20:06 | <paravoid> | cr1-eqiad/cr1-sdtpa: new BGP policy to reset MED for XO | [production] | 
            
  | 16:59 | <cmjohnson1> | dns update | [production] | 
            
  | 15:30 | <cmjohnson1> | labnet1001 relocating racks C4 to B3 eqiad | [production] | 
            
  | 13:24 | <springle_> | starting xtrabackup clone db1001 to db1016 | [production] | 
            
  | 10:39 | <akosiaris> | uploaded php5_5.3.10-1ubuntu3.9+wmf1 to apt.wikimedia.org | [production] | 
            
  | 09:54 | <paravoid> | rebooting cp1065 with Linux 3.2 | [production] | 
            
  | 07:43 | <springle> | taking down db1016 for xfs_check, locked up and: XFS (dm-0): xlog_space_left: head behind tail | [production] | 
            
  | 05:57 | <springle> | synchronized wmf-config/db-eqiad.php  'warm up db1033 in s1' | [production] | 
            
  | 02:26 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Mon Dec 23 02:26:35 UTC 2013 | [production] | 
            
  | 02:13 | <LocalisationUpdate> | completed (1.23wmf8) at Mon Dec 23 02:13:09 UTC 2013 | [production] | 
            
  | 02:07 | <LocalisationUpdate> | completed (1.23wmf7) at Mon Dec 23 02:07:23 UTC 2013 | [production] | 
            
  | 00:50 | <springle> | xtrabackup clone db1049 to db1033 | [production] | 
            
  | 00:37 | <springle> | synchronized wmf-config/db-eqiad.php  'rebalance group loads while db1050 out' | [production] | 
            
  
    | 2013-12-22
      
      § | 
    
  | 19:35 | <apergos> | db1050 boot:  (during fsck) hung at "The disk drive for /a is not ready yet or not present. Continue waiting, S to skip.." etc | [production] | 
            
  | 19:34 | <apergos> | db1050 boot:  "[   65.625950] device-mapper: table: 252:3: snapshot: Snapshot cow pairing for exception table handover failed" | [production] | 
            
  | 19:33 | <ori> | synchronized wmf-config/db-eqiad.php  'Depool db1050' | [production] | 
            
  | 19:32 | <ori> | updated /a/common to {{Gerrit|If33894260}}: Depool db1050; unresponsive | [production] | 
            
  | 19:27 | <apergos> | powrcycling db1050, inaccessible | [production] | 
            
  | 02:28 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Sun Dec 22 02:28:45 UTC 2013 | [production] | 
            
  | 02:15 | <LocalisationUpdate> | completed (1.23wmf8) at Sun Dec 22 02:15:27 UTC 2013 | [production] | 
            
  | 02:08 | <LocalisationUpdate> | completed (1.23wmf7) at Sun Dec 22 02:08:39 UTC 2013 | [production] |