|
2020-12-07
§
|
| 15:54 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:41 |
<moritzm> |
installing vips security updates |
[production] |
| 14:50 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.36.0-wmf.20 |
[production] |
| 14:46 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.20 |
[production] |
| 14:38 |
<hashar@deploy1001> |
Synchronized php: group1 wikis to 1.36.0-wmf.20 (duration: 01m 06s) |
[production] |
| 14:37 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.20 |
[production] |
| 14:33 |
<hashar@deploy1001> |
Synchronized php-1.36.0-wmf.20/includes: Applying https://gerrit.wikimedia.org/r/c/mediawiki/core/+/645312 T2569396 (duration: 01m 15s) |
[production] |
| 13:38 |
<kart_> |
Deployed apertium service to eqiad and codfw (T255672) |
[production] |
| 13:23 |
<hashar> |
Stopping CI Jenkins for upgrade |
[production] |
| 13:20 |
<kartik@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'apertium' for release 'production' . |
[production] |
| 13:18 |
<kartik@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'apertium' for release 'production' . |
[production] |
| 13:12 |
<hashar> |
Upgrading Jenkins 2.252 > 2.263.1 on contint2001 / contint1001 |
[production] |
| 12:57 |
<Urbanecm> |
EU B&C window done |
[production] |
| 12:53 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 5691a397a9de05deddea94318dc6fa6c59c44833: Revoke urlshortener-create-url from sysops (T229633) (duration: 01m 06s) |
[production] |
| 12:47 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: ee1e40061ac4d52a90f0d44c08f1665aed83a618: Assign urlshortener-create-url permission (T229633) (duration: 01m 06s) |
[production] |
| 12:28 |
<dcausse@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T266027: [cirrus] A/B test perfield build on spaceless languages (duration: 01m 07s) |
[production] |
| 12:27 |
<effie> |
rollour scap 3.16.0-1 to canaries - T268634 |
[production] |
| 12:26 |
<effie> |
rollour scap 3.16.0-1 to canaries |
[production] |
| 12:21 |
<dcausse@deploy1001> |
Synchronized wmf-config/CirrusSearch-common.php: [cirrus] flip activation of MLR rescore window using supported_syntax (duration: 01m 06s) |
[production] |
| 12:15 |
<dcausse@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [cirrus] cleanup mediasearch commons A/B test (duration: 01m 06s) |
[production] |
| 12:04 |
<moritzm> |
installing Linux 4.19.160 updates from Buster point release (initially only package updates, no reboots yet) |
[production] |
| 11:35 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:646642| Bumping portals to master (T128546)]] (duration: 01m 06s) |
[production] |
| 11:34 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:646642| Bumping portals to master (T128546)]] (duration: 01m 40s) |
[production] |
| 09:02 |
<godog> |
bounce apache2 on prometheus1003 |
[production] |
| 08:47 |
<godog> |
add 300G to prometheus global (eqiad) |
[production] |
| 08:04 |
<godog> |
swift eqiad-prod: add weight to ms-be106[0-3] - T268435 |
[production] |
|
2020-12-05
§
|
| 16:25 |
<godog> |
swift disable sdg1 on ms-be1054 |
[production] |
| 06:12 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 06:11 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 06:09 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 06:01 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 05:49 |
<ryankemper> |
restarted pybal on `lvs1015` per the instructions in https://wikitech.wikimedia.org/wiki/PyBal#Services_known_to_PyBal_but_not_to_IPVS |
[production] |
| 05:39 |
<ryankemper> |
restarted pybal on `lvs1016` per the instructions in https://wikitech.wikimedia.org/wiki/PyBal#Services_known_to_PyBal_but_not_to_IPVS |
[production] |
| 04:44 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:44 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:44 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:44 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:28 |
<ryankemper@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 04:26 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 04:25 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 04:23 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:23 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 04:22 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:21 |
<ryankemper@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:21 |
<ryankemper@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:05 |
<ryankemper> |
T269204 reimaging the following instances to debian buster (one each from `[public, internal] x [eqiad, codfw]`): `wdqs1005`, `wdqs2002`, `wdqs1008`, `wdqs2005` |
[production] |
| 03:49 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:49 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:49 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:44 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |