|
2019-08-05
ยง
|
| 18:44 |
<Urbanecm> |
Morning SWAT done |
[production] |
| 18:39 |
<urbanecm@deploy1001> |
Synchronized php-1.34.0-wmf.16/extensions/AbuseFilter/: SWAT: d358f17: Revert "Better handling of DNONE" (T214674, T228677) (duration: 00m 47s) |
[production] |
| 18:32 |
<urbanecm@deploy1001> |
Synchronized php-1.34.0-wmf.16/extensions/AbuseFilter/: SWAT: 936a462: Better handling of DNONE (T214674, T228677) (duration: 00m 47s) |
[production] |
| 18:29 |
<urbanecm@deploy1001> |
Synchronized php-1.34.0-wmf.16/extensions/WikimediaEvents/: SWAT: 3ee0e84: Temporarily log search to two schemas (duration: 00m 47s) |
[production] |
| 18:25 |
<Urbanecm> |
Deployed patch for T207094 |
[production] |
| 18:21 |
<urbanecm@deploy1001> |
Synchronized dblists/: SWAT: a9e4ed8: Remove related-articles-footer-blacklisted-skins.dblist (T229644, 3/3) (duration: 00m 46s) |
[production] |
| 18:20 |
<urbanecm@deploy1001> |
Synchronized wmf-config/CommonSettings.php: SWAT: a9e4ed8: Remove related-articles-footer-blacklisted-skins.dblist (T229644, 2/3) (duration: 00m 47s) |
[production] |
| 18:19 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: a9e4ed8: Remove related-articles-footer-blacklisted-skins.dblist (T229644, 1/3) (duration: 00m 49s) |
[production] |
| 18:09 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 254ecc1: Switch testwiki to use kask (only) for sessions (T222099) (duration: 00m 48s) |
[production] |
| 18:07 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: e44a6e6: Enable editor gender surveys (T227793) (duration: 00m 48s) |
[production] |
| 18:06 |
<onimisionipe> |
reinit postgres on maps1001 - T229788 |
[production] |
| 17:33 |
<jijiki> |
Pool restbase2009 - T227408 |
[production] |
| 17:28 |
<fsero@puppetmaster1001> |
conftool action : set/pooled=true; selector: dnsdisc=sessionstore|citoid|cxserver|eventgate-analytics|eventgate-main|termbox|blubberoid|mathoid|zotero,name=codfw |
[production] |
| 16:53 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . |
[production] |
| 16:53 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . |
[production] |
| 16:52 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'blubberoid' for release 'production' . |
[production] |
| 16:37 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'sessionstore' for release 'production' . |
[production] |
| 16:32 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'mathoid' for release 'production' . |
[production] |
| 16:22 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-main' for release 'main' . |
[production] |
| 16:22 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-main' for release 'main' . |
[production] |
| 16:18 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . |
[production] |
| 16:16 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'citoid' for release 'production' . |
[production] |
| 16:10 |
<fsero> |
recreating citoid eventgate-analytics eventgate-main mathoid sessionstore namespaces and redeploying from helmfile T228837 |
[production] |
| 16:06 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'zotero' for release 'production' . |
[production] |
| 16:04 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'termbox' for release 'production' . |
[production] |
| 16:02 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:58 |
<Urbanecm> |
Deploy patch for T200104 |
[production] |
| 15:41 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=ncredir2002.codfw.wmnet |
[production] |
| 15:36 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'zotero' for release 'production' . |
[production] |
| 15:32 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:27 |
<fsero> |
recreating zotero and termbox namespaces and services from helmfile codfw - T228837 |
[production] |
| 15:26 |
<fsero> |
recreating zotero and termbox from helmfile codfw - T228837 |
[production] |
| 15:21 |
<marostegui> |
Add db2127 to tendril and zarcillo (s3) - T228969 |
[production] |
| 15:18 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=ncredir2002.codfw.wmnet |
[production] |
| 14:31 |
<marostegui> |
Reload haproxy on dbproxy1011 to depool labsdb1010 T222978 |
[production] |
| 14:24 |
<papaul> |
shut down rstbase2009 for battery replacement |
[production] |
| 14:12 |
<fsero@puppetmaster1001> |
conftool action : set/pooled=false; selector: dnsdisc=sessionstore|citoid|cxserver|eventgate-analytics|eventgate-main|termbox|blubberoid|mathoid|zotero,name=codfw |
[production] |
| 14:08 |
<jiji@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 14:08 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:07 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:07 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:06 |
<jijiki> |
Depool and restart restbase2009 for maint - T227408 |
[production] |
| 14:05 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . |
[production] |
| 14:04 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . |
[production] |
| 14:00 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . |
[production] |
| 13:57 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . |
[production] |
| 13:56 |
<fsero> |
deploying calico controller in codfw via helmfile - T228837 |
[production] |
| 13:42 |
<fsero> |
deploying tiller in kube-system for helmfile changes - T228837 |
[production] |
| 13:37 |
<volans> |
run cumin 'A:cumin' 'rm -v /usr/local/sbin/{wmf-upgrade-varnish,wmf-upgrade-and-reboot,wmf-downtime-host,wmf-decommission-host}' T205886 |
[production] |
| 13:28 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.kafka.roll-restart-mirror-maker (exit_code=0) |
[production] |