|
2020-01-15
ยง
|
| 19:57 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable partial blocks on last wiki, Commons T242570 (duration: 01m 03s) |
[production] |
| 19:54 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable banner for wikis that recently opted in to partial blocks T240300 T242570 T242569 (duration: 01m 05s) |
[production] |
| 18:10 |
<anomie@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Set OAuth 2 access token expiry to "infinity" (duration: 01m 04s) |
[production] |
| 17:50 |
<anomie@deploy1001> |
Synchronized private/PrivateSettings.php: Setting RSA keys for OAuth 2.0 (T242872) (duration: 01m 05s) |
[production] |
| 16:27 |
<elukey> |
import key 0xDBBF9D42B7B4BD70 (Apache BigTop) manually on install1002's gpg |
[production] |
| 15:54 |
<ladsgroup@deploy1001> |
Synchronized php-1.35.0-wmf.15/extensions/WikibaseQualityConstraints/extension.json: [[gerrit:565012|Fix service injection for special page (T242846)]] (duration: 01m 08s) |
[production] |
| 15:40 |
<ladsgroup@deploy1001> |
Synchronized php-1.35.0-wmf.15/extensions/Wikibase/client/includes/Api/PageTerms.php: [[gerrit:565034|Fix invalid iteration over false in PageTerms (T242856)]] (duration: 01m 06s) |
[production] |
| 15:37 |
<vgutierrez> |
rolling restart of ats-tls instances - T196558 T242778 |
[production] |
| 15:28 |
<ema> |
cp3064: ats-tls-restart to apply https://gerrit.wikimedia.org/r/559711 T196558 |
[production] |
| 15:20 |
<moritzm> |
installing OpenSSL security updates on db* hosts |
[production] |
| 15:02 |
<moritzm> |
installing OpenSSL security updates on mw* |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1252.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1251.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1250.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1249.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1248.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1247.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1246.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1245.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1244.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1243.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1242.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1241.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1240.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1239.eqiad.wmnet |
[production] |
| 14:54 |
<jiji@cumin1001> |
conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1238.eqiad.wmnet |
[production] |
| 14:54 |
<effie> |
lower weights on slower servers mw1238-mw1252 |
[production] |
| 14:53 |
<effie> |
pool mw1238, mw1240, mw1246 |
[production] |
| 14:43 |
<XioNoX> |
reject RPKI invalids in dfw - T220669 |
[production] |
| 14:30 |
<moritzm> |
rolling restart of FPM on mw1261-mw1265 to pick up OpenSSL security update |
[production] |
| 14:25 |
<XioNoX> |
reject RPKI invalids in ams - T220669 |
[production] |
| 14:18 |
<godog> |
reenable puppet on cp hosts, after https://gerrit.wikimedia.org/r/c/operations/puppet/+/563430 deployment |
[production] |
| 14:08 |
<effie> |
depool mw1238, mw1240, mw1246 |
[production] |
| 14:06 |
<liw@deploy1001> |
Synchronized php: group1 wikis to 1.35.0-wmf.15 (duration: 01m 07s) |
[production] |
| 14:05 |
<liw@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.15 |
[production] |
| 13:58 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:56 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:54 |
<akosiaris@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . |
[production] |
| 13:54 |
<akosiaris@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . |
[production] |
| 13:53 |
<akosiaris> |
update calico policy on eqiad/codfw/staging. Add new urldownloaders. T224551 |
[production] |
| 13:52 |
<akosiaris@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . |
[production] |
| 13:02 |
<_joe_> |
restarting gerrit |
[production] |
| 12:50 |
<XioNoX> |
reject RPKI invalids in eqsin - T220669 |
[production] |
| 12:38 |
<vgutierrez> |
Pooling ulsfo for ncredir service - T242321 |
[production] |
| 12:27 |
<awight> |
EU SWAT done |
[production] |
| 12:24 |
<awight@deploy1001> |
Synchronized php-1.35.0-wmf.14/extensions/Cite: SWAT: [[gerrit:564002|Don't fail with a LogicException during section preview (T242434)]] (duration: 01m 10s) |
[production] |
| 12:22 |
<vgutierrez> |
upgrading ats on cp4026, cp4032, cp5006 and cp5012 - T242778 T242620 |
[production] |
| 12:06 |
<XioNoX> |
reject RPKI invalids in ulsfo - T220669 |
[production] |
| 11:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1112', diff saved to https://phabricator.wikimedia.org/P10161 and previous config saved to /var/cache/conftool/dbconfig/20200115-115826-marostegui.json |
[production] |
| 11:36 |
<elukey> |
restart all varnishkafka daemons on cp4031 |
[production] |