9151-9200 of 10000 results (44ms)
2020-05-14 §
02:57 <ryankemper> wdqs1004 was repooled after successful test queries [production]
02:55 <ryankemper> wdqs2006 was repooled after successful test queries [production]
01:32 <ryankemper> depooled wdqs2006 while waiting for lag to recover [production]
00:54 <foks> change password for "Python eggs" [production]
00:37 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
00:31 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
00:08 <twentyafterfour> phabricator update appears to be stable. [production]
00:05 <twentyafterfour> updating phabricator. 1 patch + new translations. Expect only brief downtime. [production]
2020-05-13 §
23:46 <cstone> SmashPig revision changed from cd1a49da5f to 2702b04329 [production]
23:43 <ejegg> updated payments-wiki from dabba1804c to 3c465cb11c [production]
23:36 <ejegg> rolled back payments-wiki to dabba1804c [production]
23:29 <ejegg> updated payment-wiki from dabba1804c to 3c465cb11c [production]
22:40 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
22:39 <ryankemper@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
22:36 <ryankemper> Depooled wdqs1004 for subsequent wdqs data xfer [production]
22:29 <ryankemper> Pooled wdqs2005 given that lag has returned to normal levels and the instance is responding to queries correctly [production]
22:26 <ryankemper> Pooled wdqs1008 given that lag has returned to normal levels and the instance is responding to queries correctly [production]
21:30 <elukey> powercycle analytics1055 [production]
21:05 <eileen> civicrm revision changed from cfb6101e39 to ed4c9522ac, config revision is 2eb75f8dff [production]
20:16 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: T242430 Stop loading the ParsoidBatchAPI extension (duration: 01m 08s) [production]
19:09 <hashar@deploy1001> Synchronized php: group1 wikis to 1.35.0-wmf.32 (duration: 01m 05s) [production]
19:08 <hashar@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.32 [production]
18:54 <twentyafterfour> restarted php-fpm on phab1001 [production]
18:53 <thcipriani> restarting gerrit [production]
18:52 <twentyafterfour> restarting apache on phab1001 for lack of a better idea [production]
18:50 <herron> restarted kafka broker on kafka-main1001 for java security updates [production]
18:22 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 38db3e0: Update production wordmarks (T252143) (duration: 01m 07s) [production]
18:17 <urbanecm@deploy1001> Synchronized static/images/mobile/copyright/: SWAT: 38db3e0: Update production wordmarks (T252143) (duration: 01m 09s) [production]
17:55 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
17:53 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
17:52 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
17:51 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
17:24 <ryankemper> Manually depooled wdqs2005 while lag catches up following the data xfer [production]
17:21 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
17:18 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
17:12 <urandom> restarted cassandra-c, restbase2017 [production]
17:04 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
16:57 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
16:54 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
16:11 <James_F> Running AbuseFilter updateVarDumps on group0 on mwmaint1002 T246539 [production]
16:00 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
15:58 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:41 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
15:38 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:34 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
15:32 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:32 <vgutierrez> upgrade ats to version 8.0.7-1wm7 on cp4032 [production]
15:30 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
15:30 <jayme> imported scap 3.14.0-1 to main for buster-wikimedia [production]
15:30 <jayme> imported scap 3.14.0-1 to main for jessie-wikimedia [production]