|
2019-08-05
ยง
|
| 16:53 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . |
[production] |
| 16:53 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . |
[production] |
| 16:52 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'blubberoid' for release 'production' . |
[production] |
| 16:37 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'sessionstore' for release 'production' . |
[production] |
| 16:32 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'mathoid' for release 'production' . |
[production] |
| 16:22 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-main' for release 'main' . |
[production] |
| 16:22 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-main' for release 'main' . |
[production] |
| 16:18 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . |
[production] |
| 16:16 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'citoid' for release 'production' . |
[production] |
| 16:10 |
<fsero> |
recreating citoid eventgate-analytics eventgate-main mathoid sessionstore namespaces and redeploying from helmfile T228837 |
[production] |
| 16:06 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'zotero' for release 'production' . |
[production] |
| 16:04 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'termbox' for release 'production' . |
[production] |
| 16:02 |
<crusnov@> |
helmfile [CODFW] Ran 'apply' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:58 |
<Urbanecm> |
Deploy patch for T200104 |
[production] |
| 15:41 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=ncredir2002.codfw.wmnet |
[production] |
| 15:36 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'zotero' for release 'production' . |
[production] |
| 15:32 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'termbox' for release 'production' . |
[production] |
| 15:27 |
<fsero> |
recreating zotero and termbox namespaces and services from helmfile codfw - T228837 |
[production] |
| 15:26 |
<fsero> |
recreating zotero and termbox from helmfile codfw - T228837 |
[production] |
| 15:21 |
<marostegui> |
Add db2127 to tendril and zarcillo (s3) - T228969 |
[production] |
| 15:18 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=ncredir2002.codfw.wmnet |
[production] |
| 14:31 |
<marostegui> |
Reload haproxy on dbproxy1011 to depool labsdb1010 T222978 |
[production] |
| 14:24 |
<papaul> |
shut down rstbase2009 for battery replacement |
[production] |
| 14:12 |
<fsero@puppetmaster1001> |
conftool action : set/pooled=false; selector: dnsdisc=sessionstore|citoid|cxserver|eventgate-analytics|eventgate-main|termbox|blubberoid|mathoid|zotero,name=codfw |
[production] |
| 14:08 |
<jiji@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 14:08 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:07 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:07 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:06 |
<jijiki> |
Depool and restart restbase2009 for maint - T227408 |
[production] |
| 14:05 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'coredns' . |
[production] |
| 14:04 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . |
[production] |
| 14:00 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . |
[production] |
| 13:57 |
<fsero@> |
helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . |
[production] |
| 13:56 |
<fsero> |
deploying calico controller in codfw via helmfile - T228837 |
[production] |
| 13:42 |
<fsero> |
deploying tiller in kube-system for helmfile changes - T228837 |
[production] |
| 13:37 |
<volans> |
run cumin 'A:cumin' 'rm -v /usr/local/sbin/{wmf-upgrade-varnish,wmf-upgrade-and-reboot,wmf-downtime-host,wmf-decommission-host}' T205886 |
[production] |
| 13:28 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.kafka.roll-restart-mirror-maker (exit_code=0) |
[production] |
| 13:16 |
<elukey@cumin1001> |
START - Cookbook sre.kafka.roll-restart-mirror-maker |
[production] |
| 13:01 |
<jbond42> |
rolling update of openjdk-8 on restbase |
[production] |
| 12:44 |
<moritzm> |
restarting cassandra on restbase-dev1004 |
[production] |
| 12:44 |
<moritzm> |
restarting cassandra on restbase-dev1040 |
[production] |
| 12:33 |
<moritzm> |
uploaded openjdk-8 u222 for jessie-wikimedia |
[production] |
| 12:26 |
<Krinkle> |
mwscript deleteEqualMessages.php --wiki fywiktionary (requested at [[m:Steward_requests/Miscellaneous]]) |
[production] |
| 12:13 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:526657|Switch property terms migration to WRITE_NEW on production wikidata (T225053)]] (duration: 00m 48s) |
[production] |
| 12:01 |
<Urbanecm> |
EU SWAT done |
[production] |
| 11:46 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 0032b0a: Enable Page Previews as default on hewikivoyage (T222017) (duration: 00m 47s) |
[production] |
| 11:43 |
<jbond@cumin1001> |
END (FAIL) - Cookbook sre.elasticsearch.rolling-restart (exit_code=99) |
[production] |
| 11:43 |
<jbond@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-restart |
[production] |
| 11:42 |
<jbond@cumin1001> |
END (ERROR) - Cookbook sre.elasticsearch.rolling-restart (exit_code=97) |
[production] |
| 11:42 |
<jbond@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-restart |
[production] |