Elassandra
2 likes•793 views
This document discusses Elassandra, which combines Elasticsearch and Cassandra to provide search capabilities within Cassandra. It maps concepts between Elasticsearch and Cassandra, describes chaos testing by taking nodes down and performance testing. Performance tests showed some degradation using Cassandra but acceptable, and Elasticsearch performance was 50% lower than standalone Elasticsearch but workarounds exist. Throughput improved up to 50% from version 2.1.1 to 2.4.2.
Elassandra
- 2. @diego_pacheco ❏ Cat's Father ❏ Principal Software Architect ❏ Agile Coach ❏ SOA Expert ❏ DevOps Practitioner ❏ Speaker ❏ Author diegopacheco http://diego-pacheco.blogspot.com.br/ https://goo.gl/eEqvzl About me...
- 3. Cassandra: Great! But there is no real search capability.
- 4. ES: Loose DATA! Can't be used as Source of Truth.
- 5. Elassandra +
- 6. Elassandra = ES Search in Cassandra
- 7. Elassandra Concepts mapping Elasticsearch Cassandra Mapping Schema Cluster Virtual DC Index Keyspace Type Table Document Row Field Column Object Nested Field User Defined Type
- 8. Elassandra Chaos Elassandra Cluster Node 1 Node 2 NDbench
- 9. Elassandra Chaos - Strategy ❏ Via Cassandra API (Reads/Writes) ❏ Test 1. Tear Down node 2 -> Keep node 1 up ❏ Test 2. Tear Down node 1 -> Them put it up after 10s ❏ Via ES API (Reads/Writes) ❏ Test 1. Tear Down node 2 -> Keep node 1 up ❏ Test 2. Tear Down node 1 -> Them put it up after 10s
- 10. Elassandra Chaos - Results - Reads
- 11. Elassandra Chaos - Results - Reads
- 12. Elassandra Chaos - Results - Reads
- 13. Elassandra Chaos - Results - Writes
- 14. Elassandra Performance - versions 2.2.8 2.1.1 2.1.1-18 on CASS 2.2.8 NDBench 2.2.2 Master (github) ORACLE JDK 8_u45 2.4.2 on CASS 2.2.8
- 15. Elassandra Performance - ELA and ES CLUSTER CLUSTER
- 16. Elassandra Performance - ELA and CASS CLUSTER CLUSTER NDBench NDBench
- 17. Elassandra Performance - Results - the surprise! Hardware Memoria CPU m4.4xlarge 64 GB 16 vCPU 2.2.8 2.1.1 2.1.1-18 on CASS 2.2.8 Reads 28k Writes 121k Reads 20k Writes 2k Reads 24k Writes 97k Reads 3k Writes 5k
- 18. Elassandra Performance - Results - Details! 2.1.1 2.1.1-18 on CASS 2.2.8 Read ALL 21k Read Get ID 59k Read Search ID 15k Read ALL 3k Read Get ID 17k Read Search ID 3k
- 19. Elassandra Performance - Results - tuning! 2.1.1-18 on CASS 2.2.8 Read ALL OK 3k Read ALL KO 4k Threadpool.index.queue_size = 10k Threadpool.index.queue_size = 1k (default) Read ALL OK 5k Read ALL KO 6k
- 20. Elassandra Performance - Results - 2.4.2! 2.4.2 on CASS 2.2.8 Read ALL OK 12k Read ALL KO 13k
- 21. Elassandra ES Reads - 12k OK Hardware Memoria CPU m4.4xlarge 64 GB 16 vCPU
- 22. ELA ES READ 12k Search CASS JAVASTATS
- 23. ELA ES READ 12k Search CASS Nodes
- 24. ELA ES READ 12k Search ES
- 25. ELA ES READ 12k Search ES NODE
- 26. ELA ES READ 12k Search GATLING OS
- 27. Conclusions ❏ Performance has some minor degradation with CASS but is acceptable. ❏ ES performance still 50% less than original elasticsearch but you can manage to workaround. ❏ Tests was run on 4th JAN 2017 from 12:00 to 17:00 BRT ❏ CPU CASS Improvement from 1500% to 500% ❏ Up to 50% Throughput improvement from 2.1.1 to 2.4.2 (5k to 12k) ❏ OS Used was Amazon Linux (CentOS Based) ❏ All Boxes was Optimized in sense on Open Files (65k) and Gatling TCP Settings https://gist.github.com/diegopacheco/31091f9c507b9c78128e73b48f2ce08a