| 2014-11-18
      
      ยง | 
    
  | 18:42 | <ottomata> | starting trusty upgrade of analtyics1030 | [production] | 
            
  | 18:42 | <demon> | Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 04s) | [production] | 
            
  | 18:34 | <ejegg> | update crm from e9e81a828d50e8bddf98eae699c925e09b25927b to 71ec68e8da1de289c4e7adca090c0fdbccbd8b8a | [production] | 
            
  | 18:31 | <MaxSem> | created wikigrok_questions table on test, test2 and enwiki | [production] | 
            
  | 18:25 | <maxsem> | Synchronized php-1.25wmf7/extensions/MobileFrontend/: https://gerrit.wikimedia.org/r/#/c/173316/ (duration: 00m 04s) | [production] | 
            
  | 18:25 | <maxsem> | Synchronized php-1.25wmf8/extensions/MobileFrontend/: https://gerrit.wikimedia.org/r/#/c/173316/ (duration: 00m 08s) | [production] | 
            
  | 18:08 | <ottomata> | starting trusty upgrade of analytics1029 | [production] | 
            
  | 18:04 | <aaron> | Synchronized wmf-config/StartProfiler.php: Added switch-logic for new Profiler config format (duration: 00m 05s) | [production] | 
            
  | 17:40 | <demon> | Synchronized php-1.25wmf8/extensions/Translate/ttmserver/ElasticSearchTTMServer.php: hack (duration: 00m 04s) | [production] | 
            
  | 17:40 | <demon> | Synchronized php-1.25wmf7/extensions/Translate/ttmserver/ElasticSearchTTMServer.php: hack (duration: 00m 04s) | [production] | 
            
  | 17:34 | <_joe_> | restarting icinga | [production] | 
            
  | 17:24 | <demon> | Synchronized php-1.25wmf7/extensions/Translate/scripts/ttmserver-export.php: profiling hack (duration: 00m 04s) | [production] | 
            
  | 17:22 | <ottomata> | starting trusty upgrade of analytics1020 | [production] | 
            
  | 17:21 | <demon> | Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 05s) | [production] | 
            
  | 17:15 | <demon> | Synchronized php-1.25wmf8/extensions/Translate/scripts/ttmserver-export.php: profiling hack (duration: 00m 06s) | [production] | 
            
  | 17:07 | <demon> | Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 04s) | [production] | 
            
  | 17:06 | <demon> | Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 04s) | [production] | 
            
  | 16:35 | <anomie> | Synchronized php-1.25wmf8/extensions/SecurePoll: SWAT: Fix SecurePollContent handling [[gerrit:174125]] (duration: 00m 09s) | [production] | 
            
  | 16:32 | <anomie> | Synchronized php-1.25wmf8/extensions/MultimediaViewer: SWAT: Media Viewer UI bugfixes [[gerrit:174116]] (for real this time) (duration: 00m 09s) | [production] | 
            
  | 16:30 | <godog> | restarting txstatsd on tungsten to drop old metrics | [production] | 
            
  | 16:29 | <anomie> | Synchronized php-1.25wmf8/extensions/MultimediaViewer: SWAT: Media Viewer UI bugfixes [[gerrit:174116]] (duration: 00m 09s) | [production] | 
            
  | 16:28 | <ottomata> | starting upgrade to trusty of analytics1017 | [production] | 
            
  | 16:27 | <anomie> | Synchronized php-1.25wmf8/includes/filebackend: SWAT: Log more details about backend-fail-internal errors [[gerrit:174128]] (duration: 00m 09s) | [production] | 
            
  | 16:18 | <bblack> | rubidium+eeden gdnsd upgraded to 2.1.0 (baham was already there) | [production] | 
            
  | 16:06 | <manybubbles> | replaying 20,000 searches at approximately the same speed that they were issued caused only marginal bounce in load (cluster load average was 13% and two machines went about 20%).  We're ready from a performance standpoint.  yay | [production] | 
            
  | 16:02 | <anomie> | Synchronized wmf-config/InitialiseSettings.php: Touch a random PHP file, supposedly required (duration: 00m 09s) | [production] | 
            
  | 16:02 | <manybubbles> | replaying some searches against cirrus to make *super* *duper* sure it won't fall over tomorrow when we enable enwiki | [production] | 
            
  | 16:01 | <anomie> | Synchronized visualeditor-default.dblist: SWAT: Enable VisualEditor by default on Catalan Wikiquote (cawikiquote) [[gerrit:174036]] (duration: 00m 09s) | [production] | 
            
  | 16:01 | <anomie> | Synchronized visualeditor.dblist: SWAT: Enable VisualEditor by default on Catalan Wikiquote (cawikiquote) [[gerrit:174036]] (duration: 00m 09s) | [production] | 
            
  | 15:43 | <ottomata> | starting trusty upgrade of analytics1016 | [production] | 
            
  | 15:32 | <hashar> | Deleting job https://integration.wikimedia.org/ci/job/mediawiki-vendor-integration/  replaced by mediawiki-phpunit. Clearing out workspaces {{bug|73515}} | [production] | 
            
  | 14:58 | <ottomata> | starting upgrade to Trusty of analytics1015 | [production] | 
            
  | 14:55 | <springle> | fail over m2 to m2-slave (db1046); investigating db1020 | [production] | 
            
  | 14:44 | <hashar> | Gerrit web interface dead with: Cannot open ReviewDb {{bug|73555}} | [production] | 
            
  | 04:16 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Tue Nov 18 04:16:03 UTC 2014 (duration 16m 2s) | [production] | 
            
  | 02:26 | <LocalisationUpdate> | completed (1.25wmf8) at 2014-11-18 02:26:17+00:00 | [production] | 
            
  | 02:22 | <^d> | jenkins locale set from 'en' to 'en_US' since 'en' means Italian somehow. | [production] | 
            
  | 02:14 | <LocalisationUpdate> | completed (1.25wmf7) at 2014-11-18 02:14:18+00:00 | [production] | 
            
  | 01:27 | <Tim> | on osmium: removing ori's custom kernel and rebooting | [production] | 
            
  | 01:23 | <springle> | temporarily reassign db1004 for phab migration tests | [production] | 
            
  | 00:55 | <maxsem> | Synchronized php-1.25wmf8/resources/lib/oojs-ui/: https://gerrit.wikimedia.org/r/#/c/174029/ (duration: 00m 04s) | [production] | 
            
  | 00:53 | <bd808> | Restarted logstash on logstash1002; lots of errors in the log about GELF input >128 chunks | [production] | 
            
  | 00:50 | <bd808> | Restarted hung logstash process on logstash1001 | [production] | 
            
  | 00:47 | <maxsem> | Synchronized wmf-config: https://gerrit.wikimedia.org/r/173878 (duration: 00m 03s) | [production] | 
            
  | 00:44 | <maxsem> | Synchronized visualeditor.dblist: https://gerrit.wikimedia.org/r/172996 (duration: 00m 03s) | [production] | 
            
  | 00:43 | <maxsem> | Synchronized visualeditor-default.dblist: https://gerrit.wikimedia.org/r/172993 (duration: 00m 04s) | [production] | 
            
  | 00:34 | <maxsem> | Synchronized php-1.25wmf7/extensions/VisualEditor/: SWAT (duration: 00m 04s) | [production] | 
            
  | 00:33 | <maxsem> | Synchronized php-1.25wmf7/extensions/MobileFrontend/: SWAT (duration: 00m 04s) | [production] | 
            
  | 00:33 | <maxsem> | Synchronized php-1.25wmf7/extensions/WikiGrok: SWAT (duration: 00m 03s) | [production] | 
            
  | 00:32 | <maxsem> | Synchronized php-1.25wmf8/extensions/VisualEditor/: SWAT (duration: 00m 04s) | [production] |