|
2020-05-16
§
|
| 19:10 |
<Krinkle> |
krinkle@mc1028 Pruning the old `echo:seen:` Redis keys that didn't have a ttl yet |
[production] |
| 18:58 |
<Krinkle> |
krinkle@mc1027 Pruning the old `echo:seen:` Redis keys that didn't have a ttl yet, ref T252945 |
[production] |
| 18:54 |
<Krinkle> |
krinkle@mc1026 Pruning the old `echo:seen:` Redis keys that didn't have a ttl yet, ref T252945 |
[production] |
| 18:30 |
<Krinkle> |
krinkle@mc1024 Pruning the old `echo:seen:` Redis keys that didn't have a ttl yet, ref T252945 |
[production] |
| 18:24 |
<Krinkle> |
krinkle@mc1025 Pruning the old `echo:seen:` Redis keys that didn't have a ttl yet, ref T252945 |
[production] |
| 17:56 |
<Krinkle> |
krinkle@mc1023 Pruning old echo:seen: Redis keys that didn't use a ttl yet, ref T252945 |
[production] |
| 17:49 |
<Krinkle> |
krinkle@mwmaint1002: Running cleanupRemovedModules.php to prune old module_deps rows T113916 |
[production] |
| 17:24 |
<Krinkle> |
krinkle@mc1020 Prune old echo:seen: keys that have ttl:-1 from Redis main stash, ref T252945 |
[production] |
| 15:16 |
<Krinkle> |
krinkle@mc1020 Looking at why there are still over 2M echo:seen keys in redis main stash |
[production] |
| 00:55 |
<krinkle@deploy1001> |
Synchronized wmf-config/logging.php: I046868190b472 (duration: 01m 13s) |
[production] |
| 00:24 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 00:21 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:21 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 00:18 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:18 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 00:16 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:16 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 00:13 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:13 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 00:10 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:10 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 00:08 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:06 |
<ryankemper@cumin2001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 00:06 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:05 |
<ryankemper@cumin2001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 00:05 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
|
2020-05-15
§
|
| 23:50 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 23:47 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 23:46 |
<ryankemper@cumin2001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 23:46 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 23:46 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 23:43 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 23:43 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 23:37 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 23:35 |
<ryankemper> |
Pooled wdqs2007 following successful query tests (all data transfers are done now) |
[production] |
| 22:53 |
<krinkle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: I1b1578a57ef5 (duration: 01m 07s) |
[production] |
| 22:51 |
<krinkle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Iaa240eb8cf9 (duration: 01m 06s) |
[production] |
| 21:41 |
<ryankemper> |
depooled wdqs2007 while it catches up on lag |
[production] |
| 21:40 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 20:36 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 20:33 |
<ryankemper> |
pooled wdqs2003 and wdqs1007 following successful query tests |
[production] |
| 19:46 |
<krinkle@deploy1001> |
Synchronized wmf-config/CommonSettings.php: If0fd1b51 (duration: 01m 08s) |
[production] |
| 18:53 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 18:34 |
<ryankemper> |
depooled wdqs2003 while lag catches up |
[production] |
| 18:32 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 17:55 |
<vgutierrez> |
upload acme-chief 0.25 to apt.wm.o (buster) - T252881 |
[production] |
| 17:27 |
<XioNoX> |
renumber cr2-eqord:xe-0/1/1 to xe-0/1/3 - T221259 |
[production] |
| 17:02 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 17:01 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 17:00 |
<ryankemper> |
depooled wqds1007 in preparation for impending wdqs data xfer |
[production] |