|
2015-08-05
ยง
|
| 22:27 |
<ebernhardson> |
es1.7.1 upgrade on elastic1015 |
[production] |
| 21:44 |
<subbu> |
deployed cherry-picked ba49b80bdc3a156604eb3996830af0d5bc45c503 hotfix to the parsoid cluster to deal with crashers from deploy earlier today |
[production] |
| 21:17 |
<gwicke> |
finished deploy of restbase 9e177f3 (deploy 7006f9f) on restbase cluster |
[production] |
| 21:12 |
<hoo> |
Started dumpwikidatajson.sh on snapshoot1003 to create a Wikidata json dump after earlier attempts this week failed. |
[production] |
| 21:05 |
<ebernhardson> |
es1.7.1 upgrade for es1014 |
[production] |
| 20:59 |
<gwicke> |
restbase 9e177f3 (deploy 7006f9f) canary deploy on restbase1001 |
[production] |
| 20:56 |
<hoo> |
Synchronized php-1.26wmf17/extensions/Wikidata/: Update Wikibase: Fix the dumpJson and the rebuildItemsPerSite maintenance scripts (duration: 00m 20s) |
[production] |
| 20:55 |
<hoo> |
Synchronized php-1.26wmf16/extensions/Wikidata/: Update Wikibase: Fix the dumpJson and the rebuildItemsPerSite maintenance scripts (duration: 00m 20s) |
[production] |
| 20:25 |
<subbu> |
deployed parsoid version d5a5722c |
[production] |
| 20:22 |
<krinkle> |
Synchronized php-1.26wmf16/includes/resourceloader/ResourceLoaderFileModule.php: T104950 (duration: 00m 12s) |
[production] |
| 20:21 |
<krinkle> |
Synchronized php-1.26wmf16/includes/resourceloader/ResourceLoader.php: T104950 (duration: 00m 11s) |
[production] |
| 20:13 |
<krinkle> |
Synchronized php-1.26wmf17/includes/resourceloader/ResourceLoaderFileModule.php: T104950 (duration: 00m 12s) |
[production] |
| 20:12 |
<krinkle> |
Synchronized php-1.26wmf17/includes/resourceloader/ResourceLoader.php: T104950 (duration: 00m 13s) |
[production] |
| 20:07 |
<ori> |
Synchronized php-1.26wmf17/extensions/PageTriage: I2089b21fc: Updated mediawiki/core Project: mediawiki/extensions/PageTriage 22eddf4ad5bf6b3fe7c49af5812ce5fcfa5e1911 (duration: 00m 14s) |
[production] |
| 19:55 |
<gwicke> |
re-enabled puppet on restbase staging cluster in preparation for deploy |
[production] |
| 19:52 |
<gwicke> |
disabled puppet on restbase hosts in preparation for the deploy |
[production] |
| 19:36 |
<dcausse> |
es1.7.1: resume writes to indices |
[production] |
| 19:31 |
<dcausse> |
es1.7.1: restart elastic1013 |
[production] |
| 19:19 |
<bblack> |
all caches depooled for thermal stuff repooled |
[production] |
| 18:54 |
<bblack> |
depooled cp1060, cp1064 ( thermal batch 3: https://phabricator.wikimedia.org/T103226 ) |
[production] |
| 18:37 |
<dcausse> |
es1.7.1: restart elastic1012 |
[production] |
| 18:33 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group1 wikis to 1.26wmf17 |
[production] |
| 18:07 |
<bblack> |
depooled cp1059, cp1062, cp1067 ( thermal batch 2: https://phabricator.wikimedia.org/T103226 ) |
[production] |
| 18:02 |
<moritzm> |
restarted HHVM on appservers (mw1136-mw1158) for tidy/pcre security updates |
[production] |
| 17:56 |
<dcausse> |
es1.7.1: restart elastic1011 |
[production] |
| 17:48 |
<dcausse> |
es1.7.1: freeze indices (take 2) |
[production] |
| 17:36 |
<bblack> |
Synchronized wmf-config/squid-labs.php: (no message) (duration: 00m 12s) |
[production] |
| 17:15 |
<moritzm> |
restarted HHVM on appservers (mw1149-mw1151, mw1161-1188, mw1209-1220) for tidy/pcre security updates |
[production] |
| 17:09 |
<hoo> |
Finished scap: Rebuild l10n cache for wmf17, got forgotten during the train (duration: 26m 02s) |
[production] |
| 17:07 |
<bblack> |
really depooled cp1046, cp1061, cp1066 ( thermal batch 1: https://phabricator.wikimedia.org/T103226 ) |
[production] |
| 17:02 |
<bblack> |
depooled cp1046, cp1061, cp1066 ( thermal batch 1: https://phabricator.wikimedia.org/T103226 ) |
[production] |
| 16:43 |
<hoo> |
Started scap: Rebuild l10n cache for wmf17, got forgotten during the train |
[production] |
| 16:28 |
<bblack> |
cache puppets disabled for a little while, to make sure do_esi doesn't melt things |
[production] |
| 15:11 |
<thcipriani> |
Synchronized php-1.26wmf17/extensions/ContentTranslation/modules/tools/ext.cx.tools.mt.js: SWAT: FIX: Not able to set cursor in previous sections [[gerrit:229328]] (duration: 00m 12s) |
[production] |
| 15:02 |
<andrewbogott> |
rebooting labvirt1009 |
[production] |
| 14:51 |
<gwicke> |
stopped restbase on restbase1009 |
[production] |
| 14:44 |
<moritzm> |
restarted HHVM on appservers (mw1026-mw1113) for tidy/pcre security updates |
[production] |
| 14:42 |
<jynus> |
Synchronized wmf-config/db-eqiad.php: depool db1056 (duration: 00m 12s) |
[production] |
| 14:29 |
<jynus> |
Synchronized wmf-config/db-eqiad.php: repool db1059 (duration: 00m 13s) |
[production] |
| 13:16 |
<hoo> |
Removed Wikidata JSON dumps from Monday and Tuesday as they were incomplete/ had the wrong serialization format |
[production] |
| 12:41 |
<moritzm> |
restarted HHVM on canary appservers for tidy/pcre security updates, remaining app servers following soon |
[production] |
| 12:32 |
<paravoid> |
upgrading asw-c-codfw and asw-d-codfw to newer junos |
[production] |
| 11:17 |
<jynus> |
Synchronized wmf-config/db-eqiad.php: repool db1056, depool db1059 (duration: 00m 12s) |
[production] |
| 11:01 |
<godog> |
depool restbase1009, investigating healthcheck returning 500s |
[production] |
| 10:52 |
<godog> |
pool restbase100[789] in pybal |
[production] |
| 10:43 |
<paravoid> |
upgrading asw-b-codfw to newer junos |
[production] |
| 10:36 |
<jynus> |
applying schema change for s4 on codfw, some lag expected |
[production] |
| 09:08 |
<dcausse> |
es1.7.1: upgrade elastic1010 |
[production] |
| 07:46 |
<dcausse> |
es1.7.1: upgrade elastic1009 |
[production] |
| 07:12 |
<jynus> |
Synchronized wmf-config/db-eqiad.php: depool db1056 for maintenance, db1064 set to 100% (duration: 00m 12s) |
[production] |