9851-9900 of 10000 results (28ms)
2020-12-07 §
14:38 <hashar@deploy1001> Synchronized php: group1 wikis to 1.36.0-wmf.20 (duration: 01m 06s) [production]
14:37 <hashar@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.20 [production]
14:33 <hashar@deploy1001> Synchronized php-1.36.0-wmf.20/includes: Applying https://gerrit.wikimedia.org/r/c/mediawiki/core/+/645312 T2569396 (duration: 01m 15s) [production]
13:38 <kart_> Deployed apertium service to eqiad and codfw (T255672) [production]
13:23 <hashar> Stopping CI Jenkins for upgrade [production]
13:20 <kartik@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'apertium' for release 'production' . [production]
13:18 <kartik@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'apertium' for release 'production' . [production]
13:12 <hashar> Upgrading Jenkins 2.252 > 2.263.1 on contint2001 / contint1001 [production]
12:57 <Urbanecm> EU B&C window done [production]
12:53 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 5691a397a9de05deddea94318dc6fa6c59c44833: Revoke urlshortener-create-url from sysops (T229633) (duration: 01m 06s) [production]
12:47 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: ee1e40061ac4d52a90f0d44c08f1665aed83a618: Assign urlshortener-create-url permission (T229633) (duration: 01m 06s) [production]
12:28 <dcausse@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T266027: [cirrus] A/B test perfield build on spaceless languages (duration: 01m 07s) [production]
12:27 <effie> rollour scap 3.16.0-1 to canaries - T268634 [production]
12:26 <effie> rollour scap 3.16.0-1 to canaries [production]
12:21 <dcausse@deploy1001> Synchronized wmf-config/CirrusSearch-common.php: [cirrus] flip activation of MLR rescore window using supported_syntax (duration: 01m 06s) [production]
12:15 <dcausse@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [cirrus] cleanup mediasearch commons A/B test (duration: 01m 06s) [production]
12:04 <moritzm> installing Linux 4.19.160 updates from Buster point release (initially only package updates, no reboots yet) [production]
11:35 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:646642| Bumping portals to master (T128546)]] (duration: 01m 06s) [production]
11:34 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:646642| Bumping portals to master (T128546)]] (duration: 01m 40s) [production]
09:02 <godog> bounce apache2 on prometheus1003 [production]
08:47 <godog> add 300G to prometheus global (eqiad) [production]
08:04 <godog> swift eqiad-prod: add weight to ms-be106[0-3] - T268435 [production]
2020-12-05 §
16:25 <godog> swift disable sdg1 on ms-be1054 [production]
06:12 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
06:11 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
06:09 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
06:01 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
05:49 <ryankemper> restarted pybal on `lvs1015` per the instructions in https://wikitech.wikimedia.org/wiki/PyBal#Services_known_to_PyBal_but_not_to_IPVS [production]
05:39 <ryankemper> restarted pybal on `lvs1016` per the instructions in https://wikitech.wikimedia.org/wiki/PyBal#Services_known_to_PyBal_but_not_to_IPVS [production]
04:44 <ryankemper@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
04:44 <ryankemper@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
04:44 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
04:44 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
04:28 <ryankemper@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
04:26 <ryankemper@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
04:25 <ryankemper@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
04:23 <ryankemper@cumin2001> START - Cookbook sre.hosts.downtime [production]
04:23 <ryankemper@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
04:22 <ryankemper@cumin2001> START - Cookbook sre.hosts.downtime [production]
04:21 <ryankemper@cumin1001> START - Cookbook sre.hosts.downtime [production]
04:21 <ryankemper@cumin1001> START - Cookbook sre.hosts.downtime [production]
04:05 <ryankemper> T269204 reimaging the following instances to debian buster (one each from `[public, internal] x [eqiad, codfw]`): `wdqs1005`, `wdqs2002`, `wdqs1008`, `wdqs2005` [production]
03:49 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
03:49 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
03:49 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
03:44 <ryankemper@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
03:43 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
03:43 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
02:13 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
02:12 <ryankemper@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) [production]