|
2020-11-11
§
|
| 15:52 |
<ayounsi@cumin1001> |
END (PASS) - Cookbook sre.network.cf (exit_code=0) |
[production] |
| 15:52 |
<ayounsi@cumin1001> |
START - Cookbook sre.network.cf |
[production] |
| 14:29 |
<akosiaris@cumin1001> |
conftool action : set/pooled=yes; selector: name=cp3054.esams.wmnet |
[production] |
| 13:52 |
<akosiaris@cumin1001> |
conftool action : set/pooled=no; selector: name=cp3054.esams.wmnet |
[production] |
| 12:25 |
<Lucas_WMDE> |
EU backport&config window done |
[production] |
| 12:23 |
<lucaswerkmeister-wmde@deploy1001> |
Synchronized wmf-config/Wikibase.php: Config: [[gerrit:640676|Remove propagateChangeVisibility repo setting]] (duration: 00m 58s) |
[production] |
| 12:16 |
<lucaswerkmeister-wmde@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:636453|Enable propagatePageDeletion on Wikidata]] (duration: 00m 59s) |
[production] |
| 12:11 |
<lucaswerkmeister-wmde@deploy1001> |
Synchronized php-1.36.0-wmf.16/extensions/DiscussionTools/includes/CommentParser.php: Backport: [[gerrit:640497|Fix getHeadlineNodeAndOffset() returning text nodes (T267284)]] (duration: 01m 01s) |
[production] |
| 10:34 |
<XioNoX> |
delete unused interfaces from asw-d-codfw |
[production] |
| 09:53 |
<XioNoX> |
prioritized DE-CIX IXP - T262681 |
[production] |
| 02:18 |
<ryankemper> |
(WDQS deploy completed) |
[production] |
| 00:48 |
<ryankemper> |
Restarting `wdqs-categories` one host at a time across all wdqs production instances: `sudo -E cumin -b 1 'A:wdqs-all and not A:wdqs-test' 'depool && sleep 60 && systemctl restart wdqs-categories && sleep 30 && pool'` |
[production] |
| 00:47 |
<ryankemper> |
Restarted `wdqs-categories` across wdqs test hosts: `sudo -E cumin 'A:wdqs-test' 'systemctl restart wdqs-categories'` |
[production] |
| 00:47 |
<ryankemper> |
Restarted `wdqs-updater` simultaneously across all wdqs hosts: `sudo -E cumin -b 4 'A:wdqs-all' 'systemctl restart wdqs-updater'` |
[production] |
| 00:47 |
<ryankemper> |
[wdqs deploy] following deploy, example query succeeds on `query.wikidata.org`, proceeding to post deploy steps |
[production] |
| 00:46 |
<ryankemper@deploy1001> |
Finished deploy [wdqs/wdqs@03219df]: 0.3.55 (duration: 11m 24s) |
[production] |
| 00:46 |
<ryankemper> |
T222669 [Elasticsearch reindex] Began long-running reindex of cirrus elasticsearch for `codfw`, `eqiad`, and `cloudelastic`. 3 tmux sessions on `ryankemper@mwmaint1002`: `reindex_eqiad`, `reindex_codfw`, `reindex_cloudelastic` |
[production] |
| 00:38 |
<ryankemper> |
Following deploy to canary `wdqs1003`, automated tests are passing as is a manual test of an example query. Proceeding... |
[production] |
| 00:34 |
<ryankemper@deploy1001> |
Started deploy [wdqs/wdqs@03219df]: 0.3.55 |
[production] |
| 00:32 |
<ryankemper> |
About to begin wdqs deploy; before-deploy tests on canary `wdqs1003` are passing |
[production] |
| 00:09 |
<eileen> |
civicrm revision changed from d0cd7f6dbb to e5d12cc46c, config revision is e2d133eff4 |
[production] |
|
2020-11-10
§
|
| 22:14 |
<mholloway-shell@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'nontls' . |
[production] |
| 22:14 |
<mholloway-shell@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 22:08 |
<mholloway-shell@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'nontls' . |
[production] |
| 22:08 |
<mholloway-shell@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 22:05 |
<mholloway-shell@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . |
[production] |
| 21:59 |
<mholloway-shell@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 21:58 |
<jgleeson> |
update civicrm revision changed from c36a5cc1b1 to d0cd7f6dbb |
[production] |
| 21:57 |
<mholloway-shell@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 21:55 |
<mholloway-shell@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . |
[production] |
| 21:47 |
<ebernhardson> |
unban elastic1050 from eqiad search psi cluster |
[production] |
| 21:28 |
<cstone> |
civicrm revision changed from b1342c4129 to c36a5cc1b1 |
[production] |
| 21:24 |
<brennen@deploy1001> |
sync-file aborted: Testing: README.md sync-file with ssh -n for T223287 (duration: 00m 37s) |
[production] |
| 21:23 |
<brennen> |
testing some scap operations, modified to use ssh -n for debugging T223287 |
[production] |
| 21:11 |
<ebernhardson> |
ban elastic1050 from eqiad psi cluster due to excessive load |
[production] |
| 21:02 |
<brennen@deploy1001> |
Finished scap: Backport: [[gerrit:640487|language: Honor $wgTranslateNumerals, even if PHP does digit translation(T267614)]] and [[gerrit:640488|Downgrade the severity of the non-numeric argument to formatNum warnings (T267370, T267587)]] (duration: 34m 46s) |
[production] |
| 20:27 |
<brennen@deploy1001> |
Started scap: Backport: [[gerrit:640487|language: Honor $wgTranslateNumerals, even if PHP does digit translation(T267614)]] and [[gerrit:640488|Downgrade the severity of the non-numeric argument to formatNum warnings (T267370, T267587)]] |
[production] |
| 20:10 |
<brennen@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:640254|Turn on formatnum logging (T267587, T267370)]] (duration: 01m 02s) |
[production] |
| 19:06 |
<hknust> |
holger mwmaint1002 Stop T219279 |
[production] |
| 18:31 |
<hknust> |
holger mwmaint1002 Start T219279 |
[production] |
| 17:57 |
<effie> |
pool mw1263 mw1264 |
[production] |
| 17:31 |
<effie> |
briefly depool mw1263 and mw1264 |
[production] |
| 17:30 |
<jynus> |
about to shutdown db1139 for hw maintenance T261405 |
[production] |
| 17:13 |
<dwisehaupt> |
upping thank you mail flow through frmx's to 30% of the total runs |
[production] |
| 16:32 |
<XioNoX> |
add cloud-storage1-b-codfw to, well, codfw switches - T267378 |
[production] |
| 16:20 |
<effie> |
pool mw1263 |
[production] |
| 16:17 |
<hashar> |
Restarting Gerrit on gerrit1001 |
[production] |
| 16:12 |
<hashar> |
Restarted Gerrit on gerrit2001 for config change |
[production] |
| 15:53 |
<zpapierski@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@1ab89ed]: Deploying venv workaround for Debian 9 (duration: 01m 06s) |
[production] |
| 15:52 |
<zpapierski@deploy1001> |
Started deploy [wikimedia/discovery/analytics@1ab89ed]: Deploying venv workaround for Debian 9 |
[production] |