| 2013-12-18
      
      § | 
    
  | 18:20 | <otto> | synchronized wmf-config  'Syncing wmf-config for Cirrus config updates' | [production] | 
            
  | 18:19 | <otto> | synchronized cirrus.dblist  'Cirrus config update, adding new wikis' | [production] | 
            
  | 17:52 | <otto> | synchronized php-1.23wmf6/extensions/CirrusSearch  'updating php-1.23wmf6/extensions/CirrusSearch to master' | [production] | 
            
  | 17:51 | <otto> | synchronized php-1.23wmf6/extensions/CirrusSearch  'updating php-1.23wmf6/extensions/CirrusSearch to master' | [production] | 
            
  | 17:46 | <hashar> | Zuul upgrade working as expected as far as I monitored it. Mail sent to wikitech-l and eng list. | [production] | 
            
  | 17:27 | <otto> | synchronized php-1.23wmf7/extensions/CirrusSearch  'updating php-1.23wmf7/extensions/CirrusSearch to master' | [production] | 
            
  | 17:23 | <hashar> | Zuul : speedy deploy of a Wikimedia hack  to prevent Zuul from updating build description. Seems to slowing down everything.  Hack is {{gerrit|102465}}.  Deployed using https://www.mediawiki.org/wiki/Continuous_integration/Zuul#upgrading | [production] | 
            
  | 16:46 | <ottomata> | updated varnishkafka in apt to 1.0.0-2 and installed on mobile varnishes (this includes fix for logrotate postrotate command) | [production] | 
            
  | 15:40 | <paravoid> | powercycling cp1065 again, kernel BUG, soft lockup | [production] | 
            
  | 15:29 | <paravoid> | powercycling cp1065, kmem_alloc deadlock | [production] | 
            
  | 15:01 | <hashar> | Gerrit back :-) | [production] | 
            
  | 15:00 | <hashar> | Gerrit dead :( | [production] | 
            
  | 13:51 | <paravoid> | setting up new transit peering in eqiad | [production] | 
            
  | 10:00 | <ori-l> | disabled puppet on tungsten to test mwprof changes; re-enabling now | [production] | 
            
  | 07:37 | <springle> | online schema changes on masters with sql_log_bin=0, indexes only, gerrit 85508 | [production] | 
            
  | 05:32 | <ori-l> | DNS update: scholarships.wikimedia.org CNAME misc-web-lb.eqiad | [production] | 
            
  | 03:09 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Wed Dec 18 03:09:14 UTC 2013 | [production] | 
            
  | 02:47 | <LocalisationUpdate> | completed (1.23wmf6) at Wed Dec 18 02:47:41 UTC 2013 | [production] | 
            
  | 02:25 | <LocalisationUpdate> | completed (1.23wmf7) at Wed Dec 18 02:25:17 UTC 2013 | [production] | 
            
  | 01:52 | <reedy> | synchronized wmf-config/  'touch' | [production] | 
            
  | 01:46 | <reedy> | synchronized wmf-config/  'Ensuring consistency...' | [production] | 
            
  | 00:20 | <mflaschen> | synchronized wmf-config  'Deploy Draft namespace to English Wikipedia.  Remove old Labs overrides that were used for testing this.' | [production] | 
            
  | 00:12 | <springle> | ongoing schema changes on slaves, indexing only, logging gerrit 85508 | [production] | 
            
  | 00:05 | <Krinkle> | Reloading zuul to deploy I6df1f4acccd7a9 | [production] | 
            
  
    | 2013-12-17
      
      § | 
    
  | 23:30 | <LeslieCarr> | starting atftpd on carbon | [production] | 
            
  | 22:37 | <ori> | synchronized php-1.23wmf6/extensions/NavigationTiming  'I933a1e3a2: Add 'mediaWikiLoadComplete' measurement' | [production] | 
            
  | 22:30 | <ori> | synchronized php-1.23wmf7/extensions/NavigationTiming  'I933a1e3a2: Add 'mediaWikiLoadComplete' measurement' | [production] | 
            
  | 22:16 | <ori> | synchronized php-1.23wmf6/resources/startup.js  'I3d5bcf10e: startup.js: log current time as global 'mediaWikiLoadStart'' | [production] | 
            
  | 22:15 | <ori-l> | restarted apache on zirconium for scholarships vhost | [production] | 
            
  | 22:03 | <ori> | synchronized php-1.23wmf7/resources/startup.js  'I3d5bcf10e: startup.js: log current time as global 'mediaWikiLoadStart'' | [production] | 
            
  | 21:48 | <csteipp> | synchronized php-1.23wmf7/extensions/TimedMediaHandler  'bug56699' | [production] | 
            
  | 21:47 | <csteipp> | synchronized php-1.23wmf6/extensions/TimedMediaHandler  'bug56699' | [production] | 
            
  | 21:37 | <csteipp> | synchronized php-1.23wmf7/extensions/CentralAuth  'bug57081' | [production] | 
            
  | 21:36 | <csteipp> | synchronized php-1.23wmf6/extensions/CentralAuth  'bug57081' | [production] | 
            
  | 21:31 | <csteipp> | synchronized php-1.23wmf6/includes  'bug58088' | [production] | 
            
  | 21:30 | <csteipp> | synchronized php-1.23wmf7/includes  'bug58088' | [production] | 
            
  | 21:23 | <reedy> | synchronized wmf-config/  'Enable GWToolset on commonswiki' | [production] | 
            
  | 21:16 | <reedy> | updated /a/common to {{Gerrit|I2d1c666e1}}: Production configuration for GWToolset | [production] | 
            
  | 21:14 | <reedy> | finished scap: Rebuild l10n cache for GWToolset, remove AFT, no wikis moving onto GWToolset at this point (commented out) | [production] | 
            
  | 20:45 | <manybubbles> | finished the rolling restart of the elasticsearch cluster.  it could have been done more quickly but there was no hurry. | [production] | 
            
  | 20:45 | <reedy> | started scap: Rebuild l10n cache for GWToolset, remove AFT, no wikis moving onto GWToolset at this point (commented out) | [production] | 
            
  | 20:05 | <Ryan_Lane> | deploying change 102285 to OpenStackManager on virt0 | [production] | 
            
  | 19:58 | <reedy> | synchronized php-1.23wmf7/extensions/GWToolset  'Staging' | [production] | 
            
  | 19:54 | <reedy> | synchronized php-1.23wmf7/includes/filerepo/LocalRepo.php  'Fix fatal I12513b40453573124e838d54a72a2f9a2d3de338' | [production] | 
            
  | 19:32 | <reedy> | synchronized wmf-config/  'Revert Revert Cross-wiki backlink purging for commons file changes' | [production] | 
            
  | 19:27 | <reedy> | synchronized wmf-config/  'Revert Cross-wiki backlink purging for commons file changes' | [production] | 
            
  | 19:26 | <reedy> | updated /a/common to {{Gerrit|Id4124ef28}}: All non wikipedias to 1.23wmf7 | [production] | 
            
  | 19:13 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: All non wikipedias to 1.23wmf7 | [production] | 
            
  | 19:11 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 18:44 | <reedy> | synchronized wmf-config/ | [production] |