|
2019-12-17
ยง
|
| 14:40 |
<cdanis> |
tearing down tmux sessions generating nic_saturation kludge on some memcached hosts |
[production] |
| 14:36 |
<gehel> |
restarting elastic1054 for config change (new master) |
[production] |
| 14:15 |
<ema> |
cp: rolling ats-tls-restart to clear issues caused by T240950 |
[production] |
| 14:10 |
<marostegui> |
Upgrade db2091 |
[production] |
| 14:05 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@b95d700]: Deploying a fix to the puppet prefix tab (duration: 03m 26s) |
[production] |
| 14:02 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@b95d700]: Deploying a fix to the puppet prefix tab |
[production] |
| 13:53 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 13:51 |
<gehel@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 13:49 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 13:48 |
<gehel@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 13:39 |
<jforrester@deploy1001> |
rebuilt and synchronized wikiversions files: group0 to 1.35.0-wmf.11 T233859 |
[production] |
| 13:34 |
<jforrester@deploy1001> |
Finished scap: testwiki to php-1.35.0-wmf.11 and rebuild l10n cache T233859 (duration: 36m 46s) |
[production] |
| 13:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1091', diff saved to https://phabricator.wikimedia.org/P9915 and previous config saved to /var/cache/conftool/dbconfig/20191217-132554-marostegui.json |
[production] |
| 13:02 |
<moritzm> |
installing intel-microcode updates to 3.20191115.1 on buster/stretch (with one Xeon type rolled back to a regression causing failing reboots, see DSA 4562-2) |
[production] |
| 13:01 |
<ayounsi@deploy1001> |
Finished deploy [homer/deploy@359de04]: Homer release v0.1.1 - T228388 (duration: 00m 30s) |
[production] |
| 13:01 |
<ayounsi@deploy1001> |
Started deploy [homer/deploy@359de04]: Homer release v0.1.1 - T228388 |
[production] |
| 12:58 |
<jforrester@deploy1001> |
Started scap: testwiki to php-1.35.0-wmf.11 and rebuild l10n cache T233859 |
[production] |
| 12:50 |
<jforrester@deploy1001> |
Pruned MediaWiki: 1.35.0-wmf.8 (duration: 09m 25s) |
[production] |
| 12:41 |
<James_F> |
Running scap clean for wmf.8 T233859 |
[production] |
| 11:37 |
<akosiaris@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . |
[production] |
| 11:37 |
<akosiaris@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . |
[production] |
| 11:37 |
<akosiaris@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . |
[production] |
| 11:34 |
<addshore> |
finished my last maint script run early after https://phabricator.wikimedia.org/P9914 |
[production] |
| 11:27 |
<addshore> |
addshore@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildPropertyTerms.php --wiki=wikidatawiki --batch-size=1 # T237984 |
[production] |
| 11:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1091', diff saved to https://phabricator.wikimedia.org/P9913 and previous config saved to /var/cache/conftool/dbconfig/20191217-112134-marostegui.json |
[production] |
| 11:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1091', diff saved to https://phabricator.wikimedia.org/P9912 and previous config saved to /var/cache/conftool/dbconfig/20191217-111400-marostegui.json |
[production] |
| 11:06 |
<dcausse@deploy1001> |
Finished deploy [wdqs/wdqs@665d9d3]: (no justification provided) (duration: 00m 25s) |
[production] |
| 11:06 |
<dcausse@deploy1001> |
Started deploy [wdqs/wdqs@665d9d3]: (no justification provided) |
[production] |
| 11:05 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T234318 ContentTranslation: Make available by default for new Wikipedias (duration: 00m 58s) |
[production] |
| 11:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1134', diff saved to https://phabricator.wikimedia.org/P9911 and previous config saved to /var/cache/conftool/dbconfig/20191217-110322-marostegui.json |
[production] |
| 10:54 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1134', diff saved to https://phabricator.wikimedia.org/P9910 and previous config saved to /var/cache/conftool/dbconfig/20191217-105425-marostegui.json |
[production] |
| 10:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1134', diff saved to https://phabricator.wikimedia.org/P9908 and previous config saved to /var/cache/conftool/dbconfig/20191217-104530-marostegui.json |
[production] |
| 10:38 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1134', diff saved to https://phabricator.wikimedia.org/P9907 and previous config saved to /var/cache/conftool/dbconfig/20191217-103810-marostegui.json |
[production] |
| 10:29 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1134 - s1 candidate master for upgrade', diff saved to https://phabricator.wikimedia.org/P9905 and previous config saved to /var/cache/conftool/dbconfig/20191217-102907-marostegui.json |
[production] |
| 10:13 |
<volans@deploy1001> |
Finished deploy [homer/deploy@996f7be]: Homer release v0.1.0 - T228388 (duration: 00m 32s) |
[production] |
| 10:13 |
<volans@deploy1001> |
Started deploy [homer/deploy@996f7be]: Homer release v0.1.0 - T228388 |
[production] |
| 10:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1075', diff saved to https://phabricator.wikimedia.org/P9902 and previous config saved to /var/cache/conftool/dbconfig/20191217-101125-marostegui.json |
[production] |
| 10:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1075', diff saved to https://phabricator.wikimedia.org/P9901 and previous config saved to /var/cache/conftool/dbconfig/20191217-100250-marostegui.json |
[production] |
| 09:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1075', diff saved to https://phabricator.wikimedia.org/P9900 and previous config saved to /var/cache/conftool/dbconfig/20191217-095152-marostegui.json |
[production] |
| 09:44 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1075', diff saved to https://phabricator.wikimedia.org/P9899 and previous config saved to /var/cache/conftool/dbconfig/20191217-094418-marostegui.json |
[production] |
| 09:30 |
<oblivian@deploy1001> |
helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . |
[production] |
| 09:25 |
<oblivian@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'cxserver' for release 'production' . |
[production] |
| 09:15 |
<akosiaris> |
delete all namespaces in kubernetes staging cluster for initialization with etcd3 backing datastore. T239835 |
[production] |
| 09:13 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1075 s3 candidate master for upgrade', diff saved to https://phabricator.wikimedia.org/P9896 and previous config saved to /var/cache/conftool/dbconfig/20191217-091316-marostegui.json |
[production] |
| 08:56 |
<oblivian@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 08:45 |
<oblivian@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 08:45 |
<jiji@cumin1001> |
conftool action : set/weight=20; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1252.eqiad.wmnet |
[production] |
| 08:45 |
<jiji@cumin1001> |
conftool action : set/weight=20; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1251.eqiad.wmnet |
[production] |
| 08:44 |
<jiji@cumin1001> |
conftool action : set/weight=20; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1250.eqiad.wmnet |
[production] |
| 08:44 |
<jiji@cumin1001> |
conftool action : set/weight=20; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1249.eqiad.wmnet |
[production] |