|
2019-03-14
ยง
|
| 17:45 |
<mforns@deploy1001> |
Started deploy [analytics/aqs/deploy@13203f1]: Deploying AQS for node10 upgrade |
[production] |
| 17:43 |
<mforns> |
Deploying AQS using scap (node10 upgrade) |
[production] |
| 17:32 |
<arlolra> |
Updated Parsoid to f3e2209 (T213950) |
[production] |
| 17:24 |
<arlolra@deploy1001> |
Finished deploy [parsoid/deploy@8cf4107]: Updating Parsoid to f3e2209 (duration: 07m 09s) |
[production] |
| 17:17 |
<arlolra@deploy1001> |
Started deploy [parsoid/deploy@8cf4107]: Updating Parsoid to f3e2209 |
[production] |
| 17:15 |
<jijiki> |
Pool mw1280 back - T218006 |
[production] |
| 17:12 |
<jijiki> |
Depool mw2206 - T215415 |
[production] |
| 16:51 |
<otto@deploy1001> |
scap-helm eventgate-analytics finished |
[production] |
| 16:51 |
<otto@deploy1001> |
scap-helm eventgate-analytics cluster staging completed |
[production] |
| 16:51 |
<otto@deploy1001> |
scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] |
[production] |
| 16:50 |
<crusnov@deploy1001> |
Finished deploy [netbox/deploy@59430dd]: Deploy Ganeti Sync and Upgrade to upstream v2.5.8 (netmon1002) - T215229 (duration: 00m 50s) |
[production] |
| 16:49 |
<crusnov@deploy1001> |
Started deploy [netbox/deploy@59430dd]: Deploy Ganeti Sync and Upgrade to upstream v2.5.8 (netmon1002) - T215229 |
[production] |
| 16:46 |
<crusnov@deploy1001> |
Finished deploy [netbox/deploy@59430dd]: Deploy Ganeti Sync and Upgrade to upstream v2.5.8 - T215229 (duration: 00m 30s) |
[production] |
| 16:45 |
<crusnov@deploy1001> |
Started deploy [netbox/deploy@59430dd]: Deploy Ganeti Sync and Upgrade to upstream v2.5.8 - T215229 |
[production] |
| 16:32 |
<XioNoX> |
add default deny to mr1-* junos-host policies - T218234 |
[production] |
| 16:30 |
<addshore@deploy1001> |
Synchronized php-1.33.0-wmf.21/extensions/Wikibase/lib/includes/Store/Sql/TermSqlIndex.php: [[gerrit:496481]] TermSqlIndex, track calls to getTermsOfEntities (duration: 00m 50s) |
[production] |
| 16:22 |
<otto@deploy1001> |
scap-helm eventgate-analytics finished |
[production] |
| 16:22 |
<otto@deploy1001> |
scap-helm eventgate-analytics cluster staging completed |
[production] |
| 16:22 |
<otto@deploy1001> |
scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] |
[production] |
| 16:08 |
<arturo> |
reimaging cloudvirt1015 again |
[production] |
| 16:04 |
<akosiaris> |
reboot one final time all sessionstore[12]00[123] servers |
[production] |
| 16:02 |
<arturo> |
T216497 drop python-dogpile.cache from jessie-wikimedia/openstack-mitaka-jessie |
[production] |
| 14:57 |
<marostegui> |
Start replication on db2070 after testing url_notes |
[production] |
| 14:53 |
<mutante> |
analytics-tool1003 - stopping idle screen session |
[production] |
| 14:43 |
<marostegui> |
Stop replication on db2070 to test the url_notes (will alert only on IRC) |
[production] |
| 14:21 |
<otto@deploy1001> |
scap-helm eventgate-analytics finished |
[production] |
| 14:21 |
<otto@deploy1001> |
scap-helm eventgate-analytics cluster eqiad completed |
[production] |
| 14:21 |
<otto@deploy1001> |
scap-helm eventgate-analytics upgrade production -f eventgate-analytics-eqiad-values.yaml --set main_app.version=v1.0.3-wmf0 stable/eventgate-analytics [namespace: eventgate-analytics, clusters: eqiad] |
[production] |
| 14:09 |
<otto@deploy1001> |
scap-helm eventgate-analytics finished |
[production] |
| 14:09 |
<otto@deploy1001> |
scap-helm eventgate-analytics cluster staging completed |
[production] |
| 14:09 |
<otto@deploy1001> |
scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] |
[production] |
| 13:54 |
<godog> |
take a snapshot of data on prometheus2004 |
[production] |
| 13:50 |
<arturo> |
reimaging cloudvirt1015 |
[production] |
| 13:36 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1081 into API (duration: 00m 48s) |
[production] |
| 13:15 |
<arturo> |
T216497 drop libpulse0 from jessie-wikimedia/openstack-mtiaka-jessie |
[production] |
| 13:15 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1081 into API (duration: 00m 49s) |
[production] |
| 13:10 |
<arturo> |
T216497 drop python-mysqldb from jessie-wikimedia/openstack-mtiaka-jessie |
[production] |
| 13:10 |
<zfilipin@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.21 |
[production] |
| 12:50 |
<jiji@cumin1001> |
conftool action : set/pooled=yes; selector: dc=codfw,service=cxserver,cluster=scb,name=kubernetes.* |
[production] |
| 12:49 |
<jiji@cumin1001> |
conftool action : set/weight=1; selector: dc=codfw,service=cxserver,cluster=scb,name=kubernetes.* |
[production] |
| 12:42 |
<jijiki> |
Rump up k8s cxserver traffic to 8% - T213195 |
[production] |
| 12:22 |
<jiji@cumin1001> |
conftool action : set/pooled=yes; selector: dc=eqiad,service=cxserver,cluster=scb,name=kubernetes.* |
[production] |
| 12:21 |
<jiji@cumin1001> |
conftool action : set/weight=1; selector: dc=eqiad,service=cxserver,cluster=scb,name=kubernetes.* |
[production] |
| 12:17 |
<jijiki> |
Send ~4% of cxserver traffic to eqiad k8s - T213195 |
[production] |
| 12:14 |
<zeljkof> |
EU SWAT finished |
[production] |
| 12:13 |
<kartik@deploy1001> |
Synchronized wmf-config: SWAT: [[gerrit:496418]] Revert "Correct the enable context detection configuration" (duration: 00m 56s) |
[production] |
| 12:12 |
<arturo> |
T216497 drop some packages from jessie-wikimedia/openstack-mtiaka-jessie: qemu-XXX |
[production] |
| 12:06 |
<arturo> |
T216497 drop some packages from jessie-wikimedia/openstack-mtiaka-jessie: libvirt*, librados2, librbd1, because they induce the resolver to conflict with those included in stretch |
[production] |
| 12:02 |
<kartik@deploy1001> |
Synchronized wmf-config: SWAT: Revert [[gerrit:496412]] Fix content detection config (duration: 00m 56s) |
[production] |
| 11:58 |
<kartik@deploy1001> |
scap failed: average error rate on 3/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) |
[production] |