|
2017-03-14
ยง
|
| 20:29 |
<bd808> |
Deleted tools-exec-12* nodes (T160457) |
[tools] |
| 20:27 |
<bd808> |
Disassociated floating IPs from tools-exec-12* nodes (T160457) |
[tools] |
| 20:17 |
<twentyafterfour> |
scap was unable to connect to mw2256.codfw.wmnet |
[production] |
| 20:14 |
<twentyafterfour@tin> |
Finished scap: full scap of new branch, move test wikis to 1.29.0-wmf.16 refs T158997 (duration: 56m 05s) |
[production] |
| 19:22 |
<thcipriani> |
removed alien nodepool instance via: openstack server delete ci-jessie-wikimedia-566503 |
[releng] |
| 19:18 |
<twentyafterfour@tin> |
Started scap: full scap of new branch, move test wikis to 1.29.0-wmf.16 refs T158997 |
[production] |
| 19:14 |
<ema> |
restarting pybal on lvs1010-11 T160405 |
[production] |
| 19:13 |
<Reedy> |
Delete 2FA for User:Conny per request on IRC. Identy verified via Lydia_WMDE |
[production] |
| 18:43 |
<nuria> |
rolling back prior eventlogging deployment, userAgent column is restricted to 191 chars, needs to be bigger or UAs are truncated |
[analytics] |
| 18:42 |
<nuria@tin> |
Finished deploy [eventlogging/analytics@417c40f]: (no justification provided) (duration: 00m 02s) |
[production] |
| 18:42 |
<nuria@tin> |
Started deploy [eventlogging/analytics@417c40f]: (no justification provided) |
[production] |
| 18:39 |
<gehel> |
removing swap from elasticsearch servers - T158884 |
[production] |
| 18:37 |
<ottomata> |
upgrading librdkafka to 0.9.4 and restarting varnishkafka on cache text hosts |
[production] |
| 18:19 |
<ottomata> |
upgrading librdkafka to 0.9.4 and restarting varnishkafka on cache upload hosts |
[production] |
| 18:13 |
<ottomata> |
upgrading librdkafka to 0.9.4 and restarting varnishkafka on cache misc hosts |
[production] |
| 18:11 |
<nuria@tin> |
Finished deploy [eventlogging/analytics@c3ccb4a]: (no justification provided) (duration: 00m 03s) |
[production] |
| 18:11 |
<nuria@tin> |
Started deploy [eventlogging/analytics@c3ccb4a]: (no justification provided) |
[production] |
| 17:41 |
<madhuvishy> |
Hand fix tools-puppetmaster by removing the old mariadb submodule directory |
[tools] |
| 17:23 |
<madhuvishy> |
Remove role::toollabs::precise_reminder from tools-bastion-03 |
[tools] |
| 17:07 |
<ottomata> |
upgrading librdkafka to 0.9.4 on cache misc and restarting varnishkafka |
[production] |
| 16:29 |
<jynus> |
no reponse from db1057 after powercycle- trying to hard reset it |
[production] |
| 16:10 |
<urandom> |
T111113: Restart Cassandra in RESTBase Staging to enable optional client encryption |
[production] |
| 15:40 |
<bd808> |
Installing toollabs-webservice 0.36 across cluster using clush |
[tools] |
| 15:39 |
<godog> |
shut ms-be2002 for idrac / bios troubleshooting T155689 |
[production] |
| 15:36 |
<bd808> |
Upgraded toollabs-webservice to 0.36 on tools-bastion-02.tools |
[tools] |
| 15:25 |
<bd808> |
Installing jobutils 1.21 across cluster using clush |
[tools] |
| 15:24 |
<chasemp> |
silence toolschecker precise job start check in anticipation of removal |
[production] |
| 15:23 |
<bd808> |
Installed jobutils 1.21 on tools-bastion-02 |
[tools] |
| 15:18 |
<twentyafterfour> |
preparing to branch 1.29.0-wmf.16 refs T158997 |
[production] |
| 15:03 |
<bd808> |
Shutting down webservices running on Precise job grid nodes |
[tools] |
| 14:36 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1080 - T132416 (duration: 00m 40s) |
[production] |
| 14:36 |
<marostegui> |
Enabled parallel replication (5 threads) on db2033 (x1) - T160407 |
[production] |
| 14:20 |
<chasemp> |
labsdb100[9|10|11] 'maintain-views --all-databases --table page --replace-all --debug' |
[production] |
| 14:18 |
<chasemp> |
labsdb1003 time maintain-views --all-databases --table page --replace-all --debug |
[production] |
| 14:14 |
<elukey> |
analytics1043 back in service |
[analytics] |
| 14:01 |
<Dereckson> |
Purged portals URL |
[production] |
| 13:56 |
<dereckson@tin> |
Synchronized portals: Resync portals/ directory after touch (duration: 00m 42s) |
[production] |
| 13:56 |
<chasemp> |
labsdb1001 maintain-views --all-databases --table page --replace-all --debug |
[production] |
| 13:46 |
<dereckson@tin> |
Synchronized portals: Bump to e576c18522ff (duration: 00m 41s) |
[production] |
| 13:45 |
<dereckson@tin> |
Synchronized portals/prod/wikipedia.org/assets: Bump to e576c18522ff (duration: 00m 41s) |
[production] |
| 13:18 |
<elukey> |
started redis-cli --bigkeys -i 0.1 on rdb1008 (eqiad jobqueue slave) |
[production] |
| 13:15 |
<dereckson@tin> |
Synchronized portals: (no justification provided) (duration: 00m 41s) |
[production] |
| 13:14 |
<dereckson@tin> |
Synchronized portals/prod/wikipedia.org/assets: (no justification provided) (duration: 00m 41s) |
[production] |
| 13:00 |
<gehel> |
restarting elasticsearch on relforge1001 to test gelf appender |
[production] |
| 12:53 |
<elukey> |
restarted webrequest-load-wf-upload-2017-3-14-11 |
[analytics] |
| 12:53 |
<elukey> |
restarted webrequest-load-wf-text-2017-3-14-11 |
[analytics] |
| 12:41 |
<elukey> |
reimage analytics1043 to Debian Jessie |
[production] |
| 12:32 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1054 with full weight after warmup (duration: 00m 40s) |
[production] |
| 12:28 |
<jynus> |
stopping mariadb on db1057, preparing to backup and reimage |
[production] |
| 12:24 |
<addshore@tin> |
Synchronized dblists/: T150183 wmgUseInterwikiSorting true for all wikidata clients [[gerrit:341034|#1]] [[gerrit:342611|#2]] PT 4/4 (duration: 00m 41s) |
[production] |