| 2014-01-28
      
      § | 
    
  | 02:49 | <LocalisationUpdate> | completed (1.23wmf10) at 2014-01-28 02:25:44+00:00 | [production] | 
            
  | 01:43 | <^d> | gerrit back up, running v2.8.1 stable now | [production] | 
            
  | 01:36 | <^d> | gerrit down for upgrade | [production] | 
            
  | 01:26 | <bsitu> | synchronized php-1.23wmf11/extensions/Flow  'Update Flow with some special contribs cherry-picks' | [production] | 
            
  | 01:19 | <ori> | synchronized php-1.23wmf10/extensions/WikimediaEvents | [production] | 
            
  | 01:18 | <ori> | synchronized php-1.23wmf11/extensions/WikimediaEvents | [production] | 
            
  | 01:05 | <ori> | synchronized php-1.23wmf10/extensions/WikimediaEvents  'Ifc697cbe6: Revert I829790cd5, removing module storage logging' | [production] | 
            
  | 00:57 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1006, warm up' | [production] | 
            
  | 00:56 | <ori> | synchronized php-1.23wmf11/extensions/WikimediaEvents  'Ifc697cbe6: Revert I829790cd5, removing module storage logging' | [production] | 
            
  
    | 2014-01-27
      
      § | 
    
  | 22:02 | <csteipp> | synchronized php-1.23wmf10/includes  'bug 60339' | [production] | 
            
  | 21:58 | <csteipp> | synchronized php-1.23wmf11/includes  'bug 60339' | [production] | 
            
  | 21:08 | <csteipp> | synchronized php-1.23wmf10/extensions/PdfHandler  'bug 60339' | [production] | 
            
  | 21:07 | <csteipp> | synchronized php-1.23wmf11/extensions/PdfHandler  'bug 60339' | [production] | 
            
  | 21:02 | <ori> | dns update | [production] | 
            
  | 20:54 | <ori> | reload-vcl on cp1043: Backend host '"holmium.eikimrfis.org"' could not be resolved to an IP address: Name or service not known. (culprit: https://gerrit.wikimedia.org/r/#/c/109008/) | [production] | 
            
  | 20:35 | <aaron> | synchronized php-1.23wmf11/includes/actions/InfoAction.php  '5f94782c2bf09ce6c00b6ad37a68d08fef802fe5' | [production] | 
            
  | 19:12 | <reedy> | synchronized docroot/bits/favicon/commons.ico | [production] | 
            
  | 19:08 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 19:07 | <ori> | synchronized php-1.23wmf10/extensions/WikimediaShopLink  'I21e34fe0f: Update WikimediaShopLink to master for PHP link insert' | [production] | 
            
  | 19:05 | <ori> | synchronized php-1.23wmf11/extensions/WikimediaShopLink  'I21e34fe0f: Update WikimediaShopLink to master for PHP link insert' | [production] | 
            
  | 18:45 | <ori> | cp1054 appears to have recovered, ops to investigate | [production] | 
            
  | 18:37 | <ori> | dmesg on cp1054 [16167206.414548] XFS: possible memory allocation deadlock in kmem_alloc (mode:0x250) | [production] | 
            
  | 18:35 | <ori> | varnishd saturating cpu on cp1054 | [production] | 
            
  | 17:26 | <csteipp> | synchronized php-1.23wmf10/extensions/TimedMediaHandler  'bug 56699 refix' | [production] | 
            
  | 13:35 | <springle> | xtrabackup clone db1022 to db1006 | [production] | 
            
  | 13:26 | <springle> | synchronized wmf-config/db-eqiad.php  'depool db1006' | [production] | 
            
  | 10:44 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1015, warm up' | [production] | 
            
  | 07:58 | <springle> | xtrabackup clone db1022 to db1015 | [production] | 
            
  | 07:48 | <springle> | synchronized wmf-config/db-eqiad.php  'depool db1015' | [production] | 
            
  | 07:00 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1023' | [production] | 
            
  | 04:10 | <springle> | xtrabackup clone db1022 to db1023 | [production] | 
            
  | 03:16 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1010, depool db1023' | [production] | 
            
  | 03:09 | <andrewbogott> | I'm about to do a bunch of experimental bullshit in the production puppet repo, because I can't test network topology in labs. | [production] | 
            
  | 02:44 | <LocalisationUpdate> | ResourceLoader cache refresh completed at 2014-01-27 02:44:23+00:00 | [production] | 
            
  | 02:22 | <LocalisationUpdate> | completed (1.23wmf11) at 2014-01-27 02:22:27+00:00 | [production] | 
            
  | 02:12 | <LocalisationUpdate> | completed (1.23wmf10) at 2014-01-27 02:12:03+00:00 | [production] |