|
2017-09-12
§
|
| 14:39 |
<bblack> |
manually setting fetch_maxchunksize=64MB via varnishadm on cp3034 fe+be instances |
[production] |
| 14:19 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Remove all references to db1059 (duration: 00m 44s) |
[production] |
| 14:18 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Remove all references to db1059 (duration: 00m 45s) |
[production] |
| 14:10 |
<gehel> |
restarting elasticsearch on elastic1020 to validate new plugin deployment - T158560 |
[production] |
| 14:07 |
<gehel> |
switching elasticsearch plugin deployment to .deb instead of scap on cirrus/eqiad - T158560 |
[production] |
| 14:06 |
<volans> |
testing reimage of mw2100 - T166300 |
[production] |
| 14:05 |
<gehel> |
restarting elasticsearch on elastic2001 to validate new plugin deployment - T158560 |
[production] |
| 14:04 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1059, pool db1097 with low load (duration: 00m 45s) |
[production] |
| 14:02 |
<gehel> |
switching elasticsearch plugin deployment to .deb instead of scap on cirrus/codfw - T158560 |
[production] |
| 13:37 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=wdqs1005.eqiad.wmnet |
[production] |
| 13:37 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=wdqs1004.eqiad.wmnet |
[production] |
| 13:37 |
<gehel> |
pooling wdqs100[45] - T171210 |
[production] |
| 13:31 |
<hashar@tin> |
Synchronized php-1.30.0-wmf.17/extensions/CentralAuth/includes/api/ApiSetGlobalAccountStatus.php: API: Unbreak setglobalaccountstatus locked=lock - T175462 (duration: 00m 44s) |
[production] |
| 13:26 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable WikidataPageBanner for Russian Wikimedia chapter wiki - T175356 (duration: 00m 45s) |
[production] |
| 13:25 |
<hashar@tin> |
Synchronized php-1.30.0-wmf.17/includes/changes/ChangesListBooleanFilter.php: WLFilters: Respect default values - T174725 (duration: 00m 45s) |
[production] |
| 13:21 |
<hashar@tin> |
Synchronized wmf-config/CommonSettings.php: Add Extension:Newsletter permissions to CommonSettings (duration: 00m 45s) |
[production] |
| 13:18 |
<hashar@tin> |
Synchronized wmf-config/throttle.php: Lift account creation restrictions for WM Taiwan 10th anniversary - T175534 (duration: 00m 45s) |
[production] |
| 13:15 |
<hashar@tin> |
Synchronized php-1.30.0-wmf.17/extensions/Wikidata/extensions/Wikibase/view/resources/jquery/wikibase/themes/default/jquery.wikibase.entityselector.css: Remove red highlighting on all entity selectors - T175525 (duration: 00m 45s) |
[production] |
| 13:12 |
<hashar@tin> |
Synchronized wmf-config/Wikibase-production.php: Reduce wikiPageUpdaterDbBatchSize to 20 - T173710 (duration: 00m 45s) |
[production] |
| 09:33 |
<moritzm> |
upgrading deployment servers to HHVM-Luasandbox 2.0.14 |
[production] |
| 09:32 |
<moritzm> |
upgrading mw2* to HHVM-Luasandbox 2.0.14 |
[production] |
| 09:25 |
<moritzm> |
upgrading mw1293-mw1295 (image scalers) to HHVM-Luasandbox 2.0.14 (along with HHVM restarts) |
[production] |
| 09:24 |
<moritzm> |
upgrading mw1293-mw1205 (image scalers) to HHVM-Luasandbox 2.0.14 (along with HHVM restarts) |
[production] |
| 08:38 |
<gehel> |
deploying elasticsearch plugins on relforge - T158560 (wrong ticket number before) |
[production] |
| 08:34 |
<gehel> |
deploying elasticsearch plugins on relforge - T175159 |
[production] |
| 08:28 |
<moritzm> |
upgrading mw1189-mw1208 (API servers) to HHVM-Luasandbox 2.0.14 (along with HHVM restarts) |
[production] |
| 07:35 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool es1019 with full weight (duration: 00m 45s) |
[production] |
| 07:15 |
<moritzm> |
upgrading mw1180-mw1188, mw1209-1220 (app servers) to HHVM-Luasandbox 2.0.14 (along with HHVM restarts) |
[production] |
| 02:29 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Sep 12 02:29:20 UTC 2017 (duration 6m 45s) |
[production] |
| 02:22 |
<l10nupdate@tin> |
scap sync-l10n completed (1.30.0-wmf.17) (duration: 07m 31s) |
[production] |
| 02:10 |
<bblack> |
cp1050 - backend restart, mailbox lag |
[production] |
| 02:08 |
<bblack> |
cp1099 - backend restart, mailbox lagh |
[production] |
| 00:48 |
<awight@tin> |
Finished deploy [ores/deploy@42c5663]: Cause ORES service restart (duration: 00m 18s) |
[production] |
| 00:48 |
<awight@tin> |
Started deploy [ores/deploy@42c5663]: Cause ORES service restart |
[production] |
|
2017-09-11
§
|
| 23:06 |
<mutante> |
jenkins-bot says: java.lang.OutofMemoryError: PermGen space |
[production] |
| 22:30 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=cp4026.* |
[production] |
| 22:30 |
<dereckson@tin> |
Synchronized wmf-config/CommonSettings-labs.php: Fix MediaWiki\Auth\AbstractPreAuthenticationProvider case ([[Gerrit:377281]], no-op in prod) (duration: 00m 45s) |
[production] |
| 21:45 |
<awight@tin> |
Finished deploy [ores/deploy@42c5663]: Try ORES filehandle fix on new cluster (duration: 13m 47s) |
[production] |
| 21:41 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=cp4025.* |
[production] |
| 21:31 |
<awight@tin> |
Started deploy [ores/deploy@42c5663]: Try ORES filehandle fix on new cluster |
[production] |
| 21:03 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=cp4023.* |
[production] |
| 20:13 |
<demon@tin> |
Pruned MediaWiki: 1.30.0-wmf.13 (duration: 02m 49s) |
[production] |
| 19:11 |
<chasemp> |
adjust nodepool rate to 10 |
[production] |
| 19:08 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool es1019 with low load (duration: 00m 46s) |
[production] |
| 19:01 |
<mutante> |
restbase-dev1004:/srv# mv /home/eevans/cassandra-1502141526-pid13410.hprof /srv/eevans/ to free disk space |
[production] |
| 18:52 |
<mutante> |
restbase - java[12418]: Error: Could not find or load main class org.wikimedia.cassandra.metrics.service.Service |
[production] |
| 18:40 |
<niharika29@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable responsive reference columns on wiktionaries, wikivoyages and enwiki https://gerrit.wikimedia.org/r/#/c/376573/, https://gerrit.wikimedia.org/r/#/c/371630/ (duration: 00m 46s) |
[production] |
| 18:33 |
<robh> |
firmware update on scs-a1-codfw and scs-c1-codfw has been complete and tested good T174475 |
[production] |
| 18:29 |
<chasemp> |
bump nodepool rate to 15 and max-servers to 18 (no restart as it picks up config periodically) |
[production] |
| 18:29 |
<robh> |
firmware update on scs-a1-codfw and scs-c1-codfw has been started, awaiting completion T174475 |
[production] |