| 2015-05-31
      
      § | 
    
  | 22:35 | <jgage> | graphite2001 keeps falling off the net due to OOM; swap 100% in use. dist-upgraded & rebooted. dmesg in ~gage/dmesg.2015-05-31 | [production] | 
            
  | 18:37 | <krinkle> | Synchronized php-1.26wmf8/resources/src/mediawiki/mediawiki.js: rl live fix - I717b86573 (duration: 00m 12s) | [production] | 
            
  | 17:36 | <Krinkle> | Confirmed RL problem solved. The jquery|mediawiki&version=bizqqnC request was cached with an old mw.loader implementation somehow. After the touch and sync, the version is now dQAzAsdU and the implementation is up to date. | [production] | 
            
  | 17:33 | <krinkle> | Synchronized php-1.26wmf7/resources: touch mediawiki.js (duration: 00m 13s) | [production] | 
            
  | 17:20 | <Krinkle> | Investigating RL issues (clients are loading mediawiki.notification&version=19700101T000000Z, mw.loader.moduleRegistry contains NaN for versions) | [production] | 
            
  | 17:12 | <gwicke> | performed a rolling restart of RESTBase Cassandra nodes to address elevated request error rates apparently related to schema disagreement | [production] | 
            
  | 05:35 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Sun May 31 05:34:36 UTC 2015 (duration 34m 35s) | [production] | 
            
  | 02:47 | <LocalisationUpdate> | completed (1.26wmf8) at 2015-05-31 02:46:41+00:00 | [production] | 
            
  | 02:43 | <l10nupdate> | Synchronized php-1.26wmf8/cache/l10n: (no message) (duration: 05m 51s) | [production] | 
            
  | 02:26 | <LocalisationUpdate> | completed (1.26wmf7) at 2015-05-31 02:25:44+00:00 | [production] | 
            
  | 02:21 | <l10nupdate> | Synchronized php-1.26wmf7/cache/l10n: (no message) (duration: 06m 41s) | [production] | 
            
  
    | 2015-05-30
      
      § | 
    
  | 21:07 | <bd808> | Upgraded Elasticsearch cluster to 1.3.9 on logstash100[1-6] | [production] | 
            
  | 18:35 | <hoo> | Synchronized php-1.26wmf7/extensions/UploadWizard/: Touch js… (duration: 00m 18s) | [production] | 
            
  | 17:06 | <legoktm> | Synchronized php-1.26wmf8/extensions/WikiEditor/extension.json: Explicitly define module position (duration: 00m 13s) | [production] | 
            
  | 05:32 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Sat May 30 05:31:02 UTC 2015 (duration 31m 1s) | [production] | 
            
  | 02:56 | <LocalisationUpdate> | completed (1.26wmf8) at 2015-05-30 02:55:22+00:00 | [production] | 
            
  | 02:52 | <l10nupdate> | Synchronized php-1.26wmf8/cache/l10n: (no message) (duration: 05m 40s) | [production] | 
            
  | 02:36 | <LocalisationUpdate> | completed (1.26wmf7) at 2015-05-30 02:34:55+00:00 | [production] | 
            
  | 02:30 | <l10nupdate> | Synchronized php-1.26wmf7/cache/l10n: (no message) (duration: 06m 50s) | [production] | 
            
  | 01:15 | <ori> | Deployed rcstream I797bc1244: Handle invalid JSON gracefully | [production] | 
            
  | 00:08 | <krenair> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/212436/ - docs only, no code change (how was this waiting 10 days?) (duration: 00m 14s) | [production] | 
            
  
    | 2015-05-29
      
      § | 
    
  | 23:56 | <ori> | Synchronized w/static/images/project-logos: Ic62747f37: Optimise project logos added since I8c9a6a56 (duration: 00m 13s) | [production] | 
            
  | 21:21 | <ori> | Synchronized wmf-config/throttle.php: Ife45684c5: Add another IP address for Santiago edit-a-thon (duration: 00m 13s) | [production] | 
            
  | 20:43 | <ori> | Synchronized robots.txt: I7b321b62d: allow robots to use RL on domains (duration: 00m 14s) | [production] | 
            
  | 17:18 | <mutante> | fix client_max_body_size syntax error in nginx config of payments1001 | [production] | 
            
  | 15:19 | <anomie> | Synchronized php-1.26wmf8/extensions/ConfirmEdit/: Update ConfirmEdit to fix API breakage [[gerrit:214620]] (duration: 00m 14s) | [production] | 
            
  | 14:52 | <paravoid> | re-redirecting ns0 traffic back to rubidium | [production] | 
            
  | 14:17 | <jynus> | Moving pdns and designate databases from m1 to m5 | [production] | 
            
  | 13:30 | <aude> | Synchronized php-1.26wmf8/extensions/Wikidata: touch js and css files to try to fix issues on test.wikidata (duration: 00m 26s) | [production] | 
            
  | 13:17 | <godog> | roll-restart cassandra on cerium / xenon / praseodymium following java upgrade | [production] | 
            
  | 11:53 | <paravoid> | reimaging rubidium | [production] | 
            
  | 11:45 | <_joe_> | restart nutcracker on mw1150 | [production] | 
            
  | 11:41 | <paravoid> | redirecting ns0 traffic to baham (= ns1) in preparation for rubidium upgrade | [production] | 
            
  | 06:52 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Fri May 29 06:51:45 UTC 2015 (duration 51m 44s) | [production] | 
            
  | 06:13 | <ori> | Synchronized php-1.26wmf7/includes/deferred/SiteStatsUpdate.php: Icc12c07ab: Update context stats in SiteStatsUpdate (duration: 00m 13s) | [production] | 
            
  | 06:12 | <ori> | Synchronized php-1.26wmf8/includes/deferred/SiteStatsUpdate.php: Icc12c07ab: Update context stats in SiteStatsUpdate (duration: 00m 14s) | [production] | 
            
  | 06:03 | <apergos> | salt keys regenerated on all production hosts (minions, not master key) | [production] | 
            
  | 03:09 | <LocalisationUpdate> | completed (1.26wmf8) at 2015-05-29 03:08:15+00:00 | [production] | 
            
  | 03:02 | <l10nupdate> | Synchronized php-1.26wmf8/cache/l10n: (no message) (duration: 10m 08s) | [production] | 
            
  | 02:36 | <LocalisationUpdate> | completed (1.26wmf7) at 2015-05-29 02:35:10+00:00 | [production] | 
            
  | 02:31 | <l10nupdate> | Synchronized php-1.26wmf7/cache/l10n: (no message) (duration: 06m 54s) | [production] | 
            
  | 00:07 | <ori> | Synchronized php-1.26wmf7/includes/diff/UnifiedDiffFormatter.php: d95cac90c7: Make the output of UnifiedDiffFormatter match diff -u (duration: 00m 14s) | [production] | 
            
  | 00:06 | <ori> | Synchronized php-1.26wmf7/extensions/Echo/includes/DiffParser.php: 41d27c4a26: Update Echo for cherry-picks (duration: 00m 13s) | [production] | 
            
  
    | 2015-05-28
      
      § | 
    
  | 23:33 | <jgage> | restarted nutcracker on mw1056 due to errors, per bd808 | [production] | 
            
  | 23:18 | <catrope> | Synchronized php-1.26wmf7/includes/EditPage.php: Fix regression with URL-specified edit tags (duration: 00m 13s) | [production] | 
            
  | 23:18 | <catrope> | Synchronized php-1.26wmf6/includes/EditPage.php: Fix regression with URL-specified edit tags (duration: 00m 13s) | [production] | 
            
  | 23:04 | <catrope> | Synchronized wmf-config/InitialiseSettings.php: Enable A/B test of VE for new accounts on enwiki (duration: 00m 13s) | [production] | 
            
  | 22:48 | <hoo> | Synchronized php-1.26wmf7/: Touching some JS, re-syncing resource definitions to rule out causes for Wikidata JS problem. (duration: 01m 00s) | [production] | 
            
  | 21:52 | <ori> | Synchronized php-1.26wmf7/resources/src/mediawiki/mediawiki.toc.js: Touching file on unconfirmed suspicion of stale cache (duration: 00m 16s) | [production] | 
            
  | 21:52 | <ori> | Synchronized php-1.26wmf8/resources/src/mediawiki/mediawiki.toc.js: Touching file on unconfirmed suspicion of stale cache (duration: 00m 15s) | [production] |