|
2017-07-26
ยง
|
| 22:46 |
<ejegg> |
reactivated remaining fundraising queue consumers |
[production] |
| 22:38 |
<ejegg> |
reactivated antifraud / payment-init queue consumer |
[production] |
| 22:34 |
<ejegg> |
updated CiviCRM from 461900edc1e6f2443894b41c4bfa1c88160f9096 to fb83798f068ba3365a286e7f131eb5eb5b0e7aae |
[production] |
| 21:39 |
<andrewbogott> |
restarting rabbitmq on labcontrol1001 |
[production] |
| 21:10 |
<mobrovac@tin> |
Finished deploy [electron-render/deploy@8dd5f13]: (no justification provided) (duration: 01m 37s) |
[production] |
| 21:08 |
<mobrovac@tin> |
Started deploy [electron-render/deploy@8dd5f13]: (no justification provided) |
[production] |
| 21:03 |
<mobrovac@tin> |
Finished deploy [electron-render/deploy@8dd5f13]: (no justification provided) (duration: 02m 38s) |
[production] |
| 21:00 |
<mobrovac@tin> |
Started deploy [electron-render/deploy@8dd5f13]: (no justification provided) |
[production] |
| 20:54 |
<mforns@tin> |
Finished deploy [analytics/refinery@58176d0]: deploying refinery to use 0.0.49 jars (duration: 03m 12s) |
[production] |
| 20:51 |
<mforns@tin> |
Started deploy [analytics/refinery@58176d0]: deploying refinery to use 0.0.49 jars |
[production] |
| 20:32 |
<mobrovac@tin> |
Finished deploy [mathoid/deploy@44ea6d8]: Switch node_modules to Node v6.11 (duration: 03m 16s) |
[production] |
| 20:31 |
<mobrovac@tin> |
Started deploy [electron-render/deploy@8dd5f13]: Switch node_modules to node v6.11 |
[production] |
| 20:30 |
<mobrovac@tin> |
Finished deploy [recommendation-api/deploy@e7adea0]: Switch node_modules to node v6.11 (duration: 02m 26s) |
[production] |
| 20:29 |
<mobrovac@tin> |
Started deploy [mathoid/deploy@44ea6d8]: Switch node_modules to Node v6.11 |
[production] |
| 20:28 |
<mobrovac@tin> |
Finished deploy [eventstreams/deploy@a2a0f19]: Switch node_modules to Node v6.11 (duration: 02m 57s) |
[production] |
| 20:27 |
<mobrovac@tin> |
Started deploy [recommendation-api/deploy@e7adea0]: Switch node_modules to node v6.11 |
[production] |
| 20:27 |
<mobrovac@tin> |
Finished deploy [trending-edits/deploy@22967f3]: Switch node_modules to node v6.11 (duration: 07m 01s) |
[production] |
| 20:25 |
<mobrovac@tin> |
Started deploy [eventstreams/deploy@a2a0f19]: Switch node_modules to Node v6.11 |
[production] |
| 20:24 |
<mobrovac@tin> |
Finished deploy [changeprop/deploy@444223d]: Switch node_modules to Node v6.11 (duration: 01m 35s) |
[production] |
| 20:22 |
<mobrovac@tin> |
Started deploy [changeprop/deploy@444223d]: Switch node_modules to Node v6.11 |
[production] |
| 20:22 |
<mobrovac@tin> |
Finished deploy [mobileapps/deploy@bb81d91]: Switch node_modules to Node v6.11 (duration: 04m 08s) |
[production] |
| 20:20 |
<mobrovac@tin> |
Started deploy [trending-edits/deploy@22967f3]: Switch node_modules to node v6.11 |
[production] |
| 20:19 |
<mobrovac@tin> |
Finished deploy [graphoid/deploy@1707b3c]: Switch node_modules to node v6.11 (duration: 07m 50s) |
[production] |
| 20:18 |
<mobrovac@tin> |
Started deploy [mobileapps/deploy@bb81d91]: Switch node_modules to Node v6.11 |
[production] |
| 20:12 |
<demon@tin> |
Finished scap: no-op, ideal timing scenario (duration: 03m 35s) |
[production] |
| 20:12 |
<mobrovac@tin> |
Finished deploy [citoid/deploy@43c2776]: Switch node_modules to Node v6.11 (duration: 02m 56s) |
[production] |
| 20:11 |
<mobrovac@tin> |
Started deploy [graphoid/deploy@1707b3c]: Switch node_modules to node v6.11 |
[production] |
| 20:10 |
<mobrovac@tin> |
Finished deploy [cxserver/deploy@f43ef96]: Switch node_modules to node v6.11 (duration: 02m 36s) |
[production] |
| 20:09 |
<demon@tin> |
Started scap: no-op, ideal timing scenario |
[production] |
| 20:09 |
<mobrovac@tin> |
Started deploy [citoid/deploy@43c2776]: Switch node_modules to Node v6.11 |
[production] |
| 20:08 |
<mobrovac@tin> |
Started deploy [cxserver/deploy@f43ef96]: Switch node_modules to node v6.11 |
[production] |
| 20:01 |
<demon@tin> |
Finished scap: group1 to wmf.11 (duration: 13m 22s) |
[production] |
| 19:48 |
<demon@tin> |
Started scap: group1 to wmf.11 |
[production] |
| 19:47 |
<demon@tin> |
scap failed: RuntimeError scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) (duration: 12m 03s) |
[production] |
| 19:47 |
<demon@tin> |
scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) |
[production] |
| 19:35 |
<demon@tin> |
Started scap: group1 to wmf.11 |
[production] |
| 19:30 |
<mutante> |
mx1001 - temp disable puppet to test adjusted sudo privileges for an icinga check |
[production] |
| 19:24 |
<ejegg> |
disabled queue consumers for CiviCRM update |
[production] |
| 19:06 |
<bblack> |
cp1074: run-no-puppet varnish-backend-restart (mailbox lag in icinga) |
[production] |
| 19:01 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=wdqs1001.eqiad.wmnet |
[production] |
| 19:01 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=wdqs1001.wmnet |
[production] |
| 19:01 |
<gehel> |
depooling wdqs1001 for data reload - T166244 |
[production] |
| 18:50 |
<niharika29@tin> |
Synchronized php-1.30.0-wmf.11/resources/src/mediawiki.rcfilters/: RCFilters: Followup I78e23f85c3: Don't disable RCFilters system when fetching results https://gerrit.wikimedia.org/r/#/c/367850/ (duration: 00m 46s) |
[production] |
| 18:35 |
<niharika29@tin> |
Synchronized php-1.30.0-wmf.11/resources/src/mediawiki.rcfilters/: RCFilters: Improve loading animation https://gerrit.wikimedia.org/r/#/c/367833/, RCFilters UI: Unbreak limit and days widgets in non-experimental mode https://gerrit.wikimedia.org/r/#/c/367837/ (duration: 00m 45s) |
[production] |
| 18:20 |
<niharika29@tin> |
Synchronized wmf-config/InitialiseSettings.php: Create 'rollbacker' user group in frwiki https://gerrit.wikimedia.org/r/#/c/365538/ (duration: 00m 47s) |
[production] |
| 17:46 |
<bblack> |
nitrogen: disabled puppet agent, manually hacked puppetdb.service unit file, restarted puppetdb.service... |
[production] |
| 17:12 |
<moritzm> |
restarting gerrit to pick up Java security update |
[production] |
| 17:11 |
<moritzm> |
installing openjdk-8 security updates on cobalt and removing unused openjdk-7 packages |
[production] |
| 16:55 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2070 (duration: 00m 45s) |
[production] |
| 16:27 |
<jynus> |
upgrading and rebooting db2070 |
[production] |