| 
      
        2013-12-19
      
      §
     | 
  
    
  | 19:20 | 
  <reedy> | 
  synchronized docroot and w | 
  [production] | 
            
  | 19:19 | 
  <reedy> | 
  synchronized wmf-config/ | 
  [production] | 
            
  | 19:07 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: phase1 wikis to 1.23wmf8 | 
  [production] | 
            
  | 19:07 | 
  <reedy> | 
  updated /a/common to {{Gerrit|I952672334}}: Rename Persian Wikibooks per community consensus | 
  [production] | 
            
  | 19:05 | 
  <reedy> | 
  synchronized wmf-config/ | 
  [production] | 
            
  | 19:01 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: All wikipedias to 1.23wmf7 | 
  [production] | 
            
  | 19:00 | 
  <reedy> | 
  updated /a/common to {{Gerrit|I22cfefa3c}}: Add an alias to NS_PROJECT on Old English Wiktionary | 
  [production] | 
            
  | 18:33 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki back to 1.23wmf7 till window | 
  [production] | 
            
  | 18:32 | 
  <reedy> | 
  finished scap: Build l10n cache and testwiki to 1.23wmf8 | 
  [production] | 
            
  | 17:56 | 
  <reedy> | 
  started scap: Build l10n cache and testwiki to 1.23wmf8 | 
  [production] | 
            
  | 17:48 | 
  <reedy> | 
  synchronized docroot and w | 
  [production] | 
            
  | 17:47 | 
  <reedy> | 
  updated /a/common to {{Gerrit|Ib6bc15109}}: previously missed deleted wikis, bug 58680 | 
  [production] | 
            
  | 17:46 | 
  <reedy> | 
  synchronized php-1.23wmf8  'Staging' | 
  [production] | 
            
  | 09:50 | 
  <apergos> | 
  powercycled cp1065, unresponsive, no messages on mgmt console | 
  [production] | 
            
  | 03:49 | 
  <Krinkle> | 
  Reloading Zuul to deploy I5b759c388db5e5a | 
  [production] | 
            
  | 03:21 | 
  <springle> | 
  starting schema changes for gerrit 101170 page_links_updated | 
  [production] | 
            
  | 02:58 | 
  <springle> | 
  synchronized database lists files: | 
  [production] | 
            
  | 02:53 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Thu Dec 19 02:53:46 UTC 2013 | 
  [production] | 
            
  | 02:31 | 
  <LocalisationUpdate> | 
  completed (1.23wmf6) at Thu Dec 19 02:31:31 UTC 2013 | 
  [production] | 
            
  | 02:16 | 
  <LocalisationUpdate> | 
  completed (1.23wmf7) at Thu Dec 19 02:16:45 UTC 2013 | 
  [production] | 
            
  | 00:10 | 
  <subbu> | 
  updated parsoid config to add support for tyv and min wikipedias | 
  [production] | 
            
  | 00:09 | 
  <mutante> | 
  doing parsoid restart via salt, with the 10% option | 
  [production] | 
            
  
    | 
      
        2013-12-18
      
      §
     | 
  
    
  | 23:44 | 
  <Krinkle> | 
  Zuul pipeline for gate-and-submit fixed (the revert was someone undone after the reload, deployed the same git commit again) | 
  [production] | 
            
  | 23:36 | 
  <Krinkle> | 
  Reloading Zuul | 
  [production] | 
            
  | 23:34 | 
  <Krinkle> | 
  Zuul is once again not processing any CR+2 events | 
  [production] | 
            
  | 23:03 | 
  <ottomata> | 
  kafka brokers on an21 and an22 are now running with cleaned up topics | 
  [production] | 
            
  | 23:02 | 
  <ori-l> | 
  zirconium: service apache2 reload for wikimania scholarships app | 
  [production] | 
            
  | 22:38 | 
  <Krinkle> | 
  Reloading Zuul to deploy Ic8cf602a0aa72 | 
  [production] | 
            
  | 22:32 | 
  <Krinkle> | 
  Zuul seems to be unresponsive to gate-and-submit events for mediawiki/* projects | 
  [production] | 
            
  | 21:45 | 
  <ottomata> | 
  stopping kafka in order to cleanup  old topic 'webrequest-mobile'.  Usually we won't have to do cleanups like this, but since this topic is new and no one is using it yes, I'd rather clean up now rather than never.' | 
  [production] | 
            
  | 20:39 | 
  <awight> | 
  update crm from 7deaa1e1a6354de545fedb0e90b85f117e7df0bd to ce73eaa0ea6019f76c63d0cca097bf828eb933e2 | 
  [production] | 
            
  | 20:25 | 
  <reedy> | 
  updated /a/common to {{Gerrit|I33c68abb1}}: Have sysops add and remove users from the 'gwtoolset' group | 
  [production] | 
            
  | 20:12 | 
  <reedy> | 
  synchronized wmf-config/ | 
  [production] | 
            
  | 19:06 | 
  <Reedy> | 
  Created EducationProgram tables on fawiki | 
  [production] | 
            
  | 19:05 | 
  <reedy> | 
  synchronized php-1.23wmf7/extensions/GWToolset/ | 
  [production] | 
            
  | 19:04 | 
  <reedy> | 
  synchronized wmf-config/ | 
  [production] | 
            
  | 18:20 | 
  <otto> | 
  synchronized wmf-config  'Syncing wmf-config for Cirrus config updates' | 
  [production] | 
            
  | 18:19 | 
  <otto> | 
  synchronized cirrus.dblist  'Cirrus config update, adding new wikis' | 
  [production] | 
            
  | 17:52 | 
  <otto> | 
  synchronized php-1.23wmf6/extensions/CirrusSearch  'updating php-1.23wmf6/extensions/CirrusSearch to master' | 
  [production] | 
            
  | 17:51 | 
  <otto> | 
  synchronized php-1.23wmf6/extensions/CirrusSearch  'updating php-1.23wmf6/extensions/CirrusSearch to master' | 
  [production] | 
            
  | 17:46 | 
  <hashar> | 
  Zuul upgrade working as expected as far as I monitored it. Mail sent to wikitech-l and eng list. | 
  [production] | 
            
  | 17:27 | 
  <otto> | 
  synchronized php-1.23wmf7/extensions/CirrusSearch  'updating php-1.23wmf7/extensions/CirrusSearch to master' | 
  [production] | 
            
  | 17:23 | 
  <hashar> | 
  Zuul : speedy deploy of a Wikimedia hack  to prevent Zuul from updating build description. Seems to slowing down everything.  Hack is {{gerrit|102465}}.  Deployed using https://www.mediawiki.org/wiki/Continuous_integration/Zuul#upgrading | 
  [production] | 
            
  | 16:46 | 
  <ottomata> | 
  updated varnishkafka in apt to 1.0.0-2 and installed on mobile varnishes (this includes fix for logrotate postrotate command) | 
  [production] | 
            
  | 15:40 | 
  <paravoid> | 
  powercycling cp1065 again, kernel BUG, soft lockup | 
  [production] | 
            
  | 15:29 | 
  <paravoid> | 
  powercycling cp1065, kmem_alloc deadlock | 
  [production] | 
            
  | 15:01 | 
  <hashar> | 
  Gerrit back :-) | 
  [production] | 
            
  | 15:00 | 
  <hashar> | 
  Gerrit dead :( | 
  [production] | 
            
  | 13:51 | 
  <paravoid> | 
  setting up new transit peering in eqiad | 
  [production] | 
            
  | 10:00 | 
  <ori-l> | 
  disabled puppet on tungsten to test mwprof changes; re-enabling now | 
  [production] |