Scylla Summit 2016: ScyllaDB, Present and Future
4 likes•3,234 views
The document outlines ScyllaDB's commitments focusing on high throughput, low latency, compatibility with Apache Cassandra, and workload conditioning. It details the roadmap for short and long-term features, including improvements in performance, support for various workloads, and innovations like multi-tenancy and a new storage format. The emphasis is on reducing operational costs and enhancing compatibility while simplifying management through modern approaches like REST APIs.
Scylla Summit 2016: ScyllaDB, Present and Future
- 1. ScyllaDB, Present and Future Avi Kivity (@AviKivity) CTO @ScyllaDB
- 2. Agenda • The Three ScyllaDB Commitments • ScyllaDB Road Map
- 3. Agenda ▸The Three ScyllaDB Commitments • ScyllaDB Road Map
- 4. The Three ScyllaDB Commitments • High Throughput and Low Latency •Compatibility with the Apache Cassandra Ecosystem • Workload Conditioning
- 5. Agenda • The Three ScyllaDB Commitments ▸Throughput and latency • ScyllaDB Road Map
- 8. Throughput and Latency • Reduces Capital and/or Cloud Expenses • Reduces the need to manage large clusters • Reduces support costs • Reduces failure rate
- 9. Not Just Equipment Cost! • Fewer nodes = fewer emergencies • Reduce risk of double failure
- 10. Not Just Equipment Cost! • Lose fewer customers due to page-load time • Win more real-time bids
- 11. Agenda • The Three ScyllaDB Commitments ▸Cassandra Compatibility • ScyllaDB Road Map
- 13. Cassandra Ecosystem Compatibility • Reuse existing investments and knowledge • Leverage existing software • Reduce dev effort, time to market
- 14. Agenda • The Three ScyllaDB Commitments ▸Workload Conditioning • ScyllaDB Road Map
- 15. Workload Conditioning • Internal feedback loops to balance competing loads Memtable Seastar Scheduler Compaction Query Repair Commitlog SSD Compaction Backlog Monitor Memory Monitor Adjust priority Adjust priority WAN CPU
- 16. Workload Conditioning Examples • Prevent compaction from falling behind • Ensure repair makes forward progress • Prevent memtable memory from filling up • Isolate read loads from write loads • Ramp up load to a newly started node until its cache is warm
- 17. ScyllaDB Commitments Recap SELECT * FROM ScyllaDB.Commitments; Commitment | Value ------------------------+------------------------------------- Performance & Latency | Reduced CapEx/CloudEx Ecosystem Compatibility | Reduced time-to-market and dev cost Workload Conditioning | Simplified operations
- 18. Road Map
- 19. Near Term (4Q16) • Materialized Views / Secondary Indexes • Counters • Lightweight Transactions • Management stack phase 1 • Formal support for REST API • Container Orchestration Integration ▪ For less critical throughput/latency
- 20. Why REST API? • JMX slow, somewhat clumsy ▪ Hard to operate from non-Java applications • Need standard, documented, simple approach to automating ScyllaDB cluster operations
- 21. Medium-long Term (1 / 2) • New storage format • Multitenancy • Analytics • Search • Additional protocol support
- 22. Medium-Long Term (2 / 2) • Filesystem bypass • NVDIMM / 3DXpoint
- 23. New Filesystem Format • C* 2.x format metadata intensive • C* 3.x format improves, but large partition support remains slower • Scylla will provide first-class large partition performance
- 24. Multitenancy • Many orgs run multiple small-ish clusters ▪ Wish to isolate performance considerations • Problems ▪ Underutilized hardware ▪ Duplication of ops effort
- 25. Multitenancy • Run several “virtual ScyllaDB clusters” on top of one physical ScyllaDB cluster ▪ Share resources, ops efforts ▪ Workload Conditioning isolates distinct workloads ▪ Each virtual cluster receives an SLA
- 26. Roadmap Recap • Bridging the gap with Cassandra • ScyllaDB features • ScyllaDB meta-features
- 27. Thank You! Contact: [email protected], @AviKivity