| 
      
        2015-08-05
      
      ยง
     | 
  
    
  | 22:47 | 
  <krinkle> | 
  Synchronized php-1.26wmf17/includes/resourceloader/ResourceLoaderModule.php: T104950 (duration: 00m 12s) | 
  [production] | 
            
  | 22:47 | 
  <krinkle> | 
  Synchronized php-1.26wmf16/includes/resourceloader/ResourceLoaderModule.php: T104950 (duration: 00m 13s) | 
  [production] | 
            
  | 22:29 | 
  <hoo> | 
  Started dumpwikidatajson.sh on snapshot1003 again to create a Wikidata json dump after earlier attempts this week and today failed. | 
  [production] | 
            
  | 22:27 | 
  <hoo> | 
  Synchronized php-1.26wmf17/extensions/Wikidata/: Update Wikibase: Fix use class in CallbackFactory (duration: 00m 21s) | 
  [production] | 
            
  | 22:27 | 
  <hoo> | 
  Synchronized php-1.26wmf16/extensions/Wikidata/: Update Wikibase: Fix use class in CallbackFactory (duration: 00m 20s) | 
  [production] | 
            
  | 22:27 | 
  <ebernhardson> | 
  es1.7.1 upgrade on elastic1015 | 
  [production] | 
            
  | 21:44 | 
  <subbu> | 
  deployed cherry-picked ba49b80bdc3a156604eb3996830af0d5bc45c503 hotfix to the parsoid cluster to deal with crashers from deploy earlier today | 
  [production] | 
            
  | 21:17 | 
  <gwicke> | 
  finished deploy of restbase 9e177f3 (deploy 7006f9f) on restbase cluster | 
  [production] | 
            
  | 21:12 | 
  <hoo> | 
  Started dumpwikidatajson.sh on snapshoot1003 to create a Wikidata json dump after earlier attempts this week failed. | 
  [production] | 
            
  | 21:05 | 
  <ebernhardson> | 
  es1.7.1 upgrade for es1014 | 
  [production] | 
            
  | 20:59 | 
  <gwicke> | 
  restbase 9e177f3 (deploy 7006f9f) canary deploy on restbase1001 | 
  [production] | 
            
  | 20:56 | 
  <hoo> | 
  Synchronized php-1.26wmf17/extensions/Wikidata/: Update Wikibase: Fix the dumpJson and the rebuildItemsPerSite maintenance scripts (duration: 00m 20s) | 
  [production] | 
            
  | 20:55 | 
  <hoo> | 
  Synchronized php-1.26wmf16/extensions/Wikidata/: Update Wikibase: Fix the dumpJson and the rebuildItemsPerSite maintenance scripts (duration: 00m 20s) | 
  [production] | 
            
  | 20:25 | 
  <subbu> | 
  deployed parsoid version d5a5722c | 
  [production] | 
            
  | 20:22 | 
  <krinkle> | 
  Synchronized php-1.26wmf16/includes/resourceloader/ResourceLoaderFileModule.php: T104950 (duration: 00m 12s) | 
  [production] | 
            
  | 20:21 | 
  <krinkle> | 
  Synchronized php-1.26wmf16/includes/resourceloader/ResourceLoader.php: T104950 (duration: 00m 11s) | 
  [production] | 
            
  | 20:13 | 
  <krinkle> | 
  Synchronized php-1.26wmf17/includes/resourceloader/ResourceLoaderFileModule.php: T104950 (duration: 00m 12s) | 
  [production] | 
            
  | 20:12 | 
  <krinkle> | 
  Synchronized php-1.26wmf17/includes/resourceloader/ResourceLoader.php: T104950 (duration: 00m 13s) | 
  [production] | 
            
  | 20:07 | 
  <ori> | 
  Synchronized php-1.26wmf17/extensions/PageTriage: I2089b21fc: Updated mediawiki/core Project: mediawiki/extensions/PageTriage  22eddf4ad5bf6b3fe7c49af5812ce5fcfa5e1911 (duration: 00m 14s) | 
  [production] | 
            
  | 19:55 | 
  <gwicke> | 
  re-enabled puppet on restbase staging cluster in preparation for deploy | 
  [production] | 
            
  | 19:52 | 
  <gwicke> | 
  disabled puppet on restbase hosts in preparation for the deploy | 
  [production] | 
            
  | 19:36 | 
  <dcausse> | 
  es1.7.1: resume writes to indices | 
  [production] | 
            
  | 19:31 | 
  <dcausse> | 
  es1.7.1: restart elastic1013 | 
  [production] | 
            
  | 19:19 | 
  <bblack> | 
  all caches depooled for thermal stuff repooled | 
  [production] | 
            
  | 18:54 | 
  <bblack> | 
  depooled cp1060, cp1064 ( thermal batch 3: https://phabricator.wikimedia.org/T103226 ) | 
  [production] | 
            
  | 18:37 | 
  <dcausse> | 
  es1.7.1: restart elastic1012 | 
  [production] | 
            
  | 18:33 | 
  <twentyafterfour> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: group1 wikis to 1.26wmf17 | 
  [production] | 
            
  | 18:07 | 
  <bblack> | 
  depooled cp1059, cp1062, cp1067 ( thermal batch 2: https://phabricator.wikimedia.org/T103226 ) | 
  [production] | 
            
  | 18:02 | 
  <moritzm> | 
  restarted HHVM on appservers (mw1136-mw1158) for tidy/pcre security updates | 
  [production] | 
            
  | 17:56 | 
  <dcausse> | 
  es1.7.1: restart elastic1011 | 
  [production] | 
            
  | 17:48 | 
  <dcausse> | 
  es1.7.1: freeze indices (take 2) | 
  [production] | 
            
  | 17:36 | 
  <bblack> | 
  Synchronized wmf-config/squid-labs.php: (no message) (duration: 00m 12s) | 
  [production] | 
            
  | 17:15 | 
  <moritzm> | 
  restarted HHVM on appservers (mw1149-mw1151, mw1161-1188, mw1209-1220) for tidy/pcre security updates | 
  [production] | 
            
  | 17:09 | 
  <hoo> | 
  Finished scap: Rebuild l10n cache for wmf17, got forgotten during the train (duration: 26m 02s) | 
  [production] | 
            
  | 17:07 | 
  <bblack> | 
  really depooled cp1046, cp1061, cp1066 ( thermal batch 1: https://phabricator.wikimedia.org/T103226 ) | 
  [production] | 
            
  | 17:02 | 
  <bblack> | 
  depooled cp1046, cp1061, cp1066 ( thermal batch 1: https://phabricator.wikimedia.org/T103226 ) | 
  [production] | 
            
  | 16:43 | 
  <hoo> | 
  Started scap: Rebuild l10n cache for wmf17, got forgotten during the train | 
  [production] | 
            
  | 16:28 | 
  <bblack> | 
  cache puppets disabled for a little while, to make sure do_esi doesn't melt things | 
  [production] | 
            
  | 15:11 | 
  <thcipriani> | 
  Synchronized php-1.26wmf17/extensions/ContentTranslation/modules/tools/ext.cx.tools.mt.js: SWAT: FIX: Not able to set cursor in previous sections [[gerrit:229328]] (duration: 00m 12s) | 
  [production] | 
            
  | 15:02 | 
  <andrewbogott> | 
  rebooting labvirt1009 | 
  [production] | 
            
  | 14:51 | 
  <gwicke> | 
  stopped restbase on restbase1009 | 
  [production] | 
            
  | 14:44 | 
  <moritzm> | 
  restarted HHVM on appservers (mw1026-mw1113) for tidy/pcre security updates | 
  [production] | 
            
  | 14:42 | 
  <jynus> | 
  Synchronized wmf-config/db-eqiad.php: depool db1056 (duration: 00m 12s) | 
  [production] | 
            
  | 14:29 | 
  <jynus> | 
  Synchronized wmf-config/db-eqiad.php: repool db1059 (duration: 00m 13s) | 
  [production] | 
            
  | 13:16 | 
  <hoo> | 
  Removed Wikidata JSON dumps from Monday and Tuesday as they were incomplete/ had the wrong serialization format | 
  [production] | 
            
  | 12:41 | 
  <moritzm> | 
  restarted HHVM on canary appservers for tidy/pcre security updates, remaining app servers following soon | 
  [production] | 
            
  | 12:32 | 
  <paravoid> | 
  upgrading asw-c-codfw and asw-d-codfw to newer junos | 
  [production] | 
            
  | 11:17 | 
  <jynus> | 
  Synchronized wmf-config/db-eqiad.php: repool db1056, depool db1059 (duration: 00m 12s) | 
  [production] | 
            
  | 11:01 | 
  <godog> | 
  depool restbase1009, investigating healthcheck returning 500s | 
  [production] | 
            
  | 10:52 | 
  <godog> | 
  pool restbase100[789] in pybal | 
  [production] |