|
2020-04-08
ยง
|
| 19:51 |
<gehel> |
restart wdqs-updater after deployment |
[production] |
| 19:49 |
<mstyles@deploy1001> |
Finished deploy [wdqs/wdqs@c2995eb]: WDQS version 0.3.21 (duration: 14m 37s) |
[production] |
| 19:44 |
<dpifke@deploy1001> |
Finished deploy [performance/navtiming@4acb04d]: Deploy new navtiming with First Input Delay metric https://phabricator.wikimedia.org/T238091 (duration: 00m 05s) |
[production] |
| 19:44 |
<dpifke@deploy1001> |
Started deploy [performance/navtiming@4acb04d]: Deploy new navtiming with First Input Delay metric https://phabricator.wikimedia.org/T238091 |
[production] |
| 19:35 |
<mstyles@deploy1001> |
Started deploy [wdqs/wdqs@c2995eb]: WDQS version 0.3.21 |
[production] |
| 19:08 |
<jhuneidi@deploy1001> |
Synchronized php: group1 wikis to 1.35.0-wmf.27 refs T247774 (duration: 01m 06s) |
[production] |
| 19:07 |
<jhuneidi@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.27 refs T247774 |
[production] |
| 19:02 |
<longma> |
deploying 1.35.0-wmf.27 to group1 |
[production] |
| 18:37 |
<jforrester@deploy1001> |
Synchronized php-1.35.0-wmf.27/skins/Vector: T248761: Revert moving indicators in DOM (duration: 01m 07s) |
[production] |
| 18:17 |
<reedy@deploy1001> |
Synchronized php-1.35.0-wmf.27/extensions/TemplateData/includes/TemplateDataHooks.php: T236809 (duration: 01m 06s) |
[production] |
| 18:16 |
<reedy@deploy1001> |
Synchronized php-1.35.0-wmf.26/extensions/TemplateData/includes/TemplateDataHooks.php: T236809 (duration: 01m 10s) |
[production] |
| 17:31 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . |
[production] |
| 17:23 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . |
[production] |
| 17:13 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 17:13 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:16 |
<ema> |
cache_upload: rolling varnish-fe restarts to bump transient storage limit T185968 |
[production] |
| 15:21 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 15:19 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:11 |
<ema> |
cp3051: param.set shortlived=0 to try ease pressure on transient memory |
[production] |
| 14:23 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1112 after schema change', diff saved to https://phabricator.wikimedia.org/P10947 and previous config saved to /var/cache/conftool/dbconfig/20200408-142341-marostegui.json |
[production] |
| 14:14 |
<jeh@deploy1001> |
Finished deploy [horizon/deploy@0d18f67]: update horizon submodule to enable server groups (duration: 03m 30s) |
[production] |
| 14:10 |
<jeh@deploy1001> |
Started deploy [horizon/deploy@0d18f67]: update horizon submodule to enable server groups |
[production] |
| 13:40 |
<mutante> |
stopped and masked zuul-merger service on contint2001 via puppet (T224591) |
[production] |
| 13:30 |
<ema> |
cp3050: stop vhtcpd, start purged T249583 |
[production] |
| 13:22 |
<vgutierrez> |
enable inbound TLSv1.3 in text@ulsfo - T170567 |
[production] |
| 13:05 |
<ema> |
purged 0.1 uploaded to buster-wikimedia T249583 |
[production] |
| 12:31 |
<tgr@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: re-sync (duration: 01m 07s) |
[production] |
| 12:29 |
<tgr@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:585219|Enable GrowthExperiments suggested edits on uk, hu, hy, eu wikipedias (T247308)]] (duration: 01m 08s) |
[production] |
| 12:17 |
<tgr@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:584135|Enable GrowthExperiments welcome survey on Ukrainian, Hungarian, Armenian Wikipedias (T238295) (duration: 01m 08s) |
[production] |
| 12:09 |
<tgr@deploy1001> |
Synchronized wmf-config/: SWAT: [[gerrit:584183|Enable GrowthExperiments on French Wiktionary (T235964)]] (duration: 01m 06s) |
[production] |
| 11:56 |
<tgr@deploy1001> |
Synchronized dblists/: SWAT: [[gerrit:584183|Enable GrowthExperiments on French Wiktionary (T235964)]] (duration: 01m 03s) |
[production] |
| 11:48 |
<mutante> |
logstash1009 - restarted logstash |
[production] |
| 11:43 |
<tgr@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:585766|Enable WikibaseQualityConstraints on test commons (T248117)]] (duration: 01m 05s) |
[production] |
| 11:43 |
<marostegui> |
Deploy schema change on db1112, this will generate lag on labs s3 |
[production] |
| 11:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1112 for schema change', diff saved to https://phabricator.wikimedia.org/P10942 and previous config saved to /var/cache/conftool/dbconfig/20200408-114315-marostegui.json |
[production] |
| 11:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1078 after schema change', diff saved to https://phabricator.wikimedia.org/P10941 and previous config saved to /var/cache/conftool/dbconfig/20200408-113901-marostegui.json |
[production] |
| 11:29 |
<tgr@deploy1001> |
Synchronized wmf-config/: SWAT: [[gerrit:584133|Deploy GrowthExperiments on Serbian Wikipedia (T241181)]] (duration: 01m 06s) |
[production] |
| 11:28 |
<tgr@deploy1001> |
Synchronized dblists/: SWAT: [[gerrit:584133|Deploy GrowthExperiments on Serbian Wikipedia (T241181)]] (duration: 01m 17s) |
[production] |
| 11:05 |
<XioNoX> |
push urpf log only to codfw - T244147 |
[production] |
| 10:39 |
<jbond42> |
restarting idp.wikimedia.org |
[production] |
| 10:14 |
<marostegui> |
Deploy schema change on db1078 |
[production] |
| 10:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1078 for schema change', diff saved to https://phabricator.wikimedia.org/P10940 and previous config saved to /var/cache/conftool/dbconfig/20200408-101431-marostegui.json |
[production] |
| 09:30 |
<jynus> |
stopping and removing db1095:s8 instance |
[production] |
| 09:20 |
<godog> |
upgrade grafana on cloudmetrics hosts - T244208 |
[production] |
| 09:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1075 after schema change', diff saved to https://phabricator.wikimedia.org/P10939 and previous config saved to /var/cache/conftool/dbconfig/20200408-091728-marostegui.json |
[production] |
| 09:11 |
<gehel> |
setting weight=10 for all pooled wdqs servers in codfw - T246343 |
[production] |
| 09:10 |
<marostegui> |
Reload proxies on dbproxy1018 and dbproxy1019 to depool labsdb1011 - T249188 T248592 |
[production] |
| 09:07 |
<gehel> |
pooling wdqs200[78] - new servers ready to go! - T246343 |
[production] |
| 08:46 |
<marostegui> |
Rename wb_terms and recreate views on labsdb1009-labsdb1011 - T248592 T248086 |
[production] |
| 08:39 |
<godog> |
upgrade grafana on grafana1002 - T244208 |
[production] |