|
2020-12-07
§
|
| 12:21 |
<dcausse@deploy1001> |
Synchronized wmf-config/CirrusSearch-common.php: [cirrus] flip activation of MLR rescore window using supported_syntax (duration: 01m 06s) |
[production] |
| 12:15 |
<dcausse@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [cirrus] cleanup mediasearch commons A/B test (duration: 01m 06s) |
[production] |
| 12:04 |
<moritzm> |
installing Linux 4.19.160 updates from Buster point release (initially only package updates, no reboots yet) |
[production] |
| 11:35 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:646642| Bumping portals to master (T128546)]] (duration: 01m 06s) |
[production] |
| 11:34 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:646642| Bumping portals to master (T128546)]] (duration: 01m 40s) |
[production] |
| 09:02 |
<godog> |
bounce apache2 on prometheus1003 |
[production] |
| 08:47 |
<godog> |
add 300G to prometheus global (eqiad) |
[production] |
| 08:04 |
<godog> |
swift eqiad-prod: add weight to ms-be106[0-3] - T268435 |
[production] |
|
2020-12-05
§
|
| 16:25 |
<godog> |
swift disable sdg1 on ms-be1054 |
[production] |
| 06:12 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 06:11 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 06:09 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 06:01 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 05:49 |
<ryankemper> |
restarted pybal on `lvs1015` per the instructions in https://wikitech.wikimedia.org/wiki/PyBal#Services_known_to_PyBal_but_not_to_IPVS |
[production] |
| 05:39 |
<ryankemper> |
restarted pybal on `lvs1016` per the instructions in https://wikitech.wikimedia.org/wiki/PyBal#Services_known_to_PyBal_but_not_to_IPVS |
[production] |
| 04:44 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:44 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:44 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:44 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:28 |
<ryankemper@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 04:26 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 04:25 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 04:23 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:23 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 04:22 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:21 |
<ryankemper@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:21 |
<ryankemper@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:05 |
<ryankemper> |
T269204 reimaging the following instances to debian buster (one each from `[public, internal] x [eqiad, codfw]`): `wdqs1005`, `wdqs2002`, `wdqs1008`, `wdqs2005` |
[production] |
| 03:49 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:49 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:49 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:44 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 03:43 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 03:43 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 02:13 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 02:12 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 02:06 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-reload (exit_code=0) |
[production] |
| 01:58 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 01:09 |
<eileen> |
civicrm revision changed from 5fa107d32a to b0ffb87c5d, config revision is ffe0a99133 |
[production] |
| 00:45 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:45 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:45 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:40 |
<Urbanecm> |
End of mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=$wiki --print-orphaned-records-to=/tmp/urbanecm/$wiki-orphaned.log --progress-markers > $wiki.log in a tmux at mwmaint1002 (wiki=eswiki; T246539) |
[production] |
| 00:35 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-reload |
[production] |
| 00:35 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 00:34 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:32 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 00:30 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 00:28 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 00:27 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |