|
2019-08-16
§
|
| 15:42 |
<elukey> |
roll restart of druid broker/historicals to pick up new logging/metrics settings |
[production] |
| 14:39 |
<onimisionipe> |
run `bmc-device --cold-reset; echo $?` in elastic2050 hoping it resets mgmt interface -T230597 |
[production] |
| 14:24 |
<gehel> |
rolling reboot of cloudelastic |
[production] |
| 13:52 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: MachineVision (beta): Request labels targeting Beta Wikidata (duration: 00m 50s) |
[production] |
| 08:18 |
<_joe_> |
stopping php on phab1003, to restart it with systemd |
[production] |
| 06:50 |
<_joe_> |
upgrading envoyproxy across production (http2 CVEs) |
[production] |
| 02:51 |
<vgutierrez> |
repooling cp5002, running compress.so experiment |
[production] |
|
2019-08-15
§
|
| 23:35 |
<smalyshev@deploy1001> |
Finished deploy [wdqs/wdqs@b4da6e4]: Rollback blazegraph due to T230588 (duration: 09m 48s) |
[production] |
| 23:25 |
<smalyshev@deploy1001> |
Started deploy [wdqs/wdqs@b4da6e4]: Rollback blazegraph due to T230588 |
[production] |
| 21:54 |
<smalyshev@deploy1001> |
Finished deploy [wdqs/wdqs@fce8177]: Weekly deploy (duration: 25m 28s) |
[production] |
| 21:28 |
<smalyshev@deploy1001> |
Started deploy [wdqs/wdqs@fce8177]: Weekly deploy |
[production] |
| 21:27 |
<ebernhardson> |
finish restarting cloudelastic-chi-eqiad with -XX:NewRatio=3 |
[production] |
| 21:18 |
<ebernhardson> |
increase cloudelastic indices.recovery.max_bytes_per_sec from 40mbit to 512mbit as these have 10G networking |
[production] |
| 21:07 |
<ebernhardson> |
restart cloudelastic1002 with -XX:NewRatio=3 to match cloudelastic1001 |
[production] |
| 20:22 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 19:37 |
<ema> |
depool cp5002 during the EU night, running compress.so experiment |
[production] |
| 19:28 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.wdqs.reboot-wdqs (exit_code=0) |
[production] |
| 19:19 |
<sbassett> |
Deployed security patch for T230402 (1.34.0-wmf.17) |
[production] |
| 19:18 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 19:18 |
<sbassett> |
Deployed security patch for T229541 (1.34.0-wmf.17) |
[production] |
| 19:17 |
<gehel@cumin1001> |
END (ERROR) - Cookbook sre.wdqs.data-transfer (exit_code=97) |
[production] |
| 19:17 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 19:01 |
<ebernhardson> |
restart elasticsearch on cloudelastic1001 with -XX:NewRatio=3 |
[production] |
| 18:51 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.reboot-wdqs |
[production] |
| 17:58 |
<mbsantos@deploy1001> |
Finished deploy [proton/deploy@fb0b2a5]: Update chromium-renderer to 3f1cc72 (T218220) (duration: 00m 43s) |
[production] |
| 17:58 |
<mbsantos@deploy1001> |
Started deploy [proton/deploy@fb0b2a5]: Update chromium-renderer to 3f1cc72 (T218220) |
[production] |
| 17:47 |
<mbsantos@deploy1001> |
Finished deploy [mobileapps/deploy@1bd2bea]: Update service-mobileapp-node to 5c1da03 (T230067 T229984) (duration: 05m 53s) |
[production] |
| 17:41 |
<mbsantos@deploy1001> |
Started deploy [mobileapps/deploy@1bd2bea]: Update service-mobileapp-node to 5c1da03 (T230067 T229984) |
[production] |
| 17:11 |
<ejegg> |
updated payments-wiki from 44eae2d65f to 00eb090dcc |
[production] |
| 17:02 |
<cstone> |
civicrm revision changed from 3caf54a0d2 to 9c7b2ffbc9 |
[production] |
| 16:53 |
<reedy@deploy1001> |
Synchronized docroot/noc/db.php: Use WmfClusters from seperate file (duration: 00m 47s) |
[production] |
| 16:52 |
<reedy@deploy1001> |
Synchronized src/WmfClusters.php: Move WmfClusters.php (duration: 00m 47s) |
[production] |
| 16:27 |
<XioNoX> |
advertise core v4 range (208.80.152.0/22) from eqord - T167841 |
[production] |
| 16:09 |
<ori> |
Finished messing around with mwdebug1002 |
[production] |
| 16:06 |
<reedy@deploy1001> |
Synchronized docroot/: phpcs fixes (duration: 00m 47s) |
[production] |
| 16:05 |
<reedy@deploy1001> |
Synchronized wmf-config/arclamp.php: phpcs (duration: 00m 47s) |
[production] |
| 16:04 |
<reedy@deploy1001> |
Synchronized tests/: phpunit (duration: 00m 47s) |
[production] |
| 16:03 |
<reedy@deploy1001> |
Synchronized phpcs.xml: more exclusions! (duration: 00m 47s) |
[production] |
| 15:40 |
<ebernhardson> |
unfreeze writes to cloudelastic cluster |
[production] |
| 15:37 |
<ema> |
cp5002: re-pool with compress.so cache:false |
[production] |
| 15:34 |
<herron> |
performing rolling restarts of eqiad kafka-main brokers for security updates |
[production] |
| 15:34 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.wdqs.reboot-wdqs (exit_code=0) |
[production] |
| 15:13 |
<ori> |
Messing around with CommonSettings.php on mwdebug1002 to profile config loading |
[production] |
| 14:58 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.reboot-wdqs |
[production] |
| 14:58 |
<gehel@cumin1001> |
END (ERROR) - Cookbook sre.wdqs.reboot-wdqs (exit_code=97) |
[production] |
| 14:56 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.reboot-wdqs |
[production] |
| 14:52 |
<reedy@deploy1001> |
Synchronized wmf-config/: phpcs cleanup (duration: 00m 47s) |
[production] |
| 14:51 |
<gehel@cumin1001> |
END (ERROR) - Cookbook sre.wdqs.reboot-wdqs (exit_code=97) |
[production] |
| 14:51 |
<reedy@deploy1001> |
Synchronized multiversion/: phpcs cleanup (duration: 00m 47s) |
[production] |
| 14:50 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.reboot-wdqs |
[production] |