|
2017-07-26
ยง
|
| 20:28 |
<mobrovac@tin> |
Finished deploy [eventstreams/deploy@a2a0f19]: Switch node_modules to Node v6.11 (duration: 02m 57s) |
[production] |
| 20:27 |
<mobrovac@tin> |
Started deploy [recommendation-api/deploy@e7adea0]: Switch node_modules to node v6.11 |
[production] |
| 20:27 |
<mobrovac@tin> |
Finished deploy [trending-edits/deploy@22967f3]: Switch node_modules to node v6.11 (duration: 07m 01s) |
[production] |
| 20:25 |
<mobrovac@tin> |
Started deploy [eventstreams/deploy@a2a0f19]: Switch node_modules to Node v6.11 |
[production] |
| 20:24 |
<mobrovac@tin> |
Finished deploy [changeprop/deploy@444223d]: Switch node_modules to Node v6.11 (duration: 01m 35s) |
[production] |
| 20:22 |
<mobrovac@tin> |
Started deploy [changeprop/deploy@444223d]: Switch node_modules to Node v6.11 |
[production] |
| 20:22 |
<mobrovac@tin> |
Finished deploy [mobileapps/deploy@bb81d91]: Switch node_modules to Node v6.11 (duration: 04m 08s) |
[production] |
| 20:20 |
<mobrovac@tin> |
Started deploy [trending-edits/deploy@22967f3]: Switch node_modules to node v6.11 |
[production] |
| 20:19 |
<mobrovac@tin> |
Finished deploy [graphoid/deploy@1707b3c]: Switch node_modules to node v6.11 (duration: 07m 50s) |
[production] |
| 20:18 |
<mobrovac@tin> |
Started deploy [mobileapps/deploy@bb81d91]: Switch node_modules to Node v6.11 |
[production] |
| 20:12 |
<demon@tin> |
Finished scap: no-op, ideal timing scenario (duration: 03m 35s) |
[production] |
| 20:12 |
<mobrovac@tin> |
Finished deploy [citoid/deploy@43c2776]: Switch node_modules to Node v6.11 (duration: 02m 56s) |
[production] |
| 20:11 |
<mobrovac@tin> |
Started deploy [graphoid/deploy@1707b3c]: Switch node_modules to node v6.11 |
[production] |
| 20:10 |
<mobrovac@tin> |
Finished deploy [cxserver/deploy@f43ef96]: Switch node_modules to node v6.11 (duration: 02m 36s) |
[production] |
| 20:09 |
<demon@tin> |
Started scap: no-op, ideal timing scenario |
[production] |
| 20:09 |
<mobrovac@tin> |
Started deploy [citoid/deploy@43c2776]: Switch node_modules to Node v6.11 |
[production] |
| 20:08 |
<mobrovac@tin> |
Started deploy [cxserver/deploy@f43ef96]: Switch node_modules to node v6.11 |
[production] |
| 20:01 |
<demon@tin> |
Finished scap: group1 to wmf.11 (duration: 13m 22s) |
[production] |
| 19:48 |
<demon@tin> |
Started scap: group1 to wmf.11 |
[production] |
| 19:47 |
<demon@tin> |
scap failed: RuntimeError scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) (duration: 12m 03s) |
[production] |
| 19:47 |
<demon@tin> |
scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) |
[production] |
| 19:35 |
<demon@tin> |
Started scap: group1 to wmf.11 |
[production] |
| 19:30 |
<mutante> |
mx1001 - temp disable puppet to test adjusted sudo privileges for an icinga check |
[production] |
| 19:24 |
<ejegg> |
disabled queue consumers for CiviCRM update |
[production] |
| 19:06 |
<bblack> |
cp1074: run-no-puppet varnish-backend-restart (mailbox lag in icinga) |
[production] |
| 19:01 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=wdqs1001.eqiad.wmnet |
[production] |
| 19:01 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=wdqs1001.wmnet |
[production] |
| 19:01 |
<gehel> |
depooling wdqs1001 for data reload - T166244 |
[production] |
| 18:50 |
<niharika29@tin> |
Synchronized php-1.30.0-wmf.11/resources/src/mediawiki.rcfilters/: RCFilters: Followup I78e23f85c3: Don't disable RCFilters system when fetching results https://gerrit.wikimedia.org/r/#/c/367850/ (duration: 00m 46s) |
[production] |
| 18:35 |
<niharika29@tin> |
Synchronized php-1.30.0-wmf.11/resources/src/mediawiki.rcfilters/: RCFilters: Improve loading animation https://gerrit.wikimedia.org/r/#/c/367833/, RCFilters UI: Unbreak limit and days widgets in non-experimental mode https://gerrit.wikimedia.org/r/#/c/367837/ (duration: 00m 45s) |
[production] |
| 18:20 |
<niharika29@tin> |
Synchronized wmf-config/InitialiseSettings.php: Create 'rollbacker' user group in frwiki https://gerrit.wikimedia.org/r/#/c/365538/ (duration: 00m 47s) |
[production] |
| 17:46 |
<bblack> |
nitrogen: disabled puppet agent, manually hacked puppetdb.service unit file, restarted puppetdb.service... |
[production] |
| 17:12 |
<moritzm> |
restarting gerrit to pick up Java security update |
[production] |
| 17:11 |
<moritzm> |
installing openjdk-8 security updates on cobalt and removing unused openjdk-7 packages |
[production] |
| 16:55 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2070 (duration: 00m 45s) |
[production] |
| 16:27 |
<jynus> |
upgrading and rebooting db2070 |
[production] |
| 16:22 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2069, depool db2070 (duration: 00m 45s) |
[production] |
| 16:14 |
<moritzm> |
upgraded nodejs on restbase* |
[production] |
| 15:48 |
<jynus> |
upgrade and reboot db2069 |
[production] |
| 15:45 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2068, depool db2069, pool db2072 with more weight (duration: 00m 46s) |
[production] |
| 15:39 |
<moritzm> |
rolling upgrade/service restarts of nodejs in eqiad |
[production] |
| 15:32 |
<andrewbogott> |
patching puppetmaster1001, possible puppet hiccups coming up |
[production] |
| 15:29 |
<moritzm> |
upgrade nodejs on remaining scb hosts (along with service restarts) |
[production] |
| 15:20 |
<moritzm> |
upgrade nodejs on scb2001 (currently depooled for testing) |
[production] |
| 15:17 |
<jynus> |
restarting and upgrading db2068 |
[production] |
| 15:15 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Depool db2068 (duration: 00m 46s) |
[production] |
| 14:40 |
<moritzm> |
installing spice security updates |
[production] |
| 14:21 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Pool db2072 (duration: 00m 45s) |
[production] |
| 13:44 |
<gehel> |
restarting cassandra on maps clusters |
[production] |
| 13:36 |
<zeljkof> |
EU SWAT finished |
[production] |