|
2021-10-18
§
|
| 09:39 |
<vgutierrez> |
updating acme-chief to version 0.34 on acmechief instances - T292619 |
[production] |
| 09:38 |
<godog> |
sync metrics from graphite1004 to graphite2003 - T247963 |
[production] |
| 09:13 |
<moritzm> |
installing apr security updates on bullseye |
[production] |
| 08:57 |
<godog> |
cleanup graphite metrics not modified for >= ~3yr (1024 days) |
[production] |
| 07:34 |
<ema> |
cp3060 (text), cp3061 (upload): upgrade varnish to 6.0.8 T292290 |
[production] |
| 07:34 |
<elukey> |
depool + restart blazegraph on wdqs1013 |
[production] |
| 07:01 |
<oblivian@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 06:31 |
<oblivian@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 06:09 |
<oblivian@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
|
2021-10-15
§
|
| 23:48 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 23:27 |
<dzahn@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'miscweb' for release 'main' . |
[production] |
| 23:23 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 22:38 |
<mutante> |
apt1001 - removing nginx package, accidentally installed, should just be nginx-light of course, running puppet |
[production] |
| 22:36 |
<mutante> |
apt2001 - removing nginx package, accidentally installed, should just be nginx-light of course, running puppet |
[production] |
| 22:34 |
<mutante> |
apt2001 - upgraded nginx |
[production] |
| 22:18 |
<dzahn@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'miscweb' for release 'main' . |
[production] |
| 22:14 |
<dzahn@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'miscweb' for release 'main' . |
[production] |
| 22:05 |
<dpifke@deploy1002> |
Finished deploy [performance/arc-lamp@40cb764]: Revert problematic arclamp patch to fix daemon crashes (duration: 00m 05s) |
[production] |
| 22:05 |
<dpifke@deploy1002> |
Started deploy [performance/arc-lamp@40cb764]: Revert problematic arclamp patch to fix daemon crashes |
[production] |
| 21:51 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 21:44 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 21:44 |
<dzahn@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'miscweb' for release 'main' . |
[production] |
| 21:36 |
<dzahn@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'miscweb' for release 'main' . |
[production] |
| 20:09 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 18:44 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 17:20 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 17:17 |
<mutante> |
gitlab1001 - disabling puppet for debugging |
[production] |
| 17:05 |
<mutante> |
gitlab2001 - temp stopped puppet - debugging gitlab restore script with Arnold - T283076 |
[production] |
| 17:01 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 16:50 |
<mutante> |
gitlab2001 - temp stopped puppet - debugging gitlab restore script with Arnold |
[production] |
| 16:46 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 16:44 |
<jayme@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'shellbox-constraints' for release 'main' . |
[production] |
| 15:23 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 15:23 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 15:08 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 15:08 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 14:48 |
<oblivian@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 14:31 |
<oblivian@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 14:15 |
<oblivian@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 13:32 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 13:32 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 13:32 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 13:30 |
<elukey> |
start topic rebalancing for kafka main-eqiad (long maintenance, it will last a couple of days) |
[production] |
| 13:24 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 13:21 |
<vgutierrez> |
updating acme-chief to version 0.34 on acmechief-test instances - T292619 |
[production] |
| 13:19 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 13:14 |
<vgutierrez> |
upload acme-chief 0.34 to apt.wikimedia.org (buster) - T292619 |
[production] |