Scylla Summit 2018: Scylla 3.0 and Beyond
Download as PPTX, PDF•3 likes•1,671 views
Scylla 3.0 will include several new features and performance improvements including incremental compaction to reduce storage requirements, columnar storage to boost analytics performance, and multi-tenancy to fully isolate user workloads. It will also add lightweight transactions and improve analytics queries, large partition support, and observability tools. Underlying infrastructure changes involve optimizing Linux and Seastar for Scylla's needs.
Scylla Summit 2018: Scylla 3.0 and Beyond
- 1. Scylla 3.0 and Beyond Avi Kivity CTO, ScyllaDB
- 2. Presenter bio Avi Kivity has extensive background in distributed storage (Exanet) and virtualization (the Linux KVM project). He is now on this stage as ScyllaDB’s CTO and co-founder.
- 3. Agenda Storage Multi-user, multi-workload Analytics Features and Performance Observability Manageability The Infrastructure’s Infrastructure
- 4. Storage
- 5. ▪ Reduced storage requirements ▪ Better large partition performance ▪ Cassandra 3 / 4 compatibility ▪ Talk: Vladimir “mc” Format SSTables
- 6. In-memory Storage Option For the most demanding workloads… Talk: Gleb
- 7. Incremental compaction ▪ Traditional size-tiered compaction requires 50% temporary space Old sstable Old sstable New sstable
- 8. Incremental compaction ▪ Traditional size-tiered compaction requires 50% temporary space ▪ Incremental compaction removes this requirement Old sstable Old sstable New sstable
- 9. Columnar storage ▪ Reduce storage requirements for time-series ▪ Increase analytics performance
- 10. Tiered storage ▪ Use a combination of fast+expensive and slow+cheap storage o NVMe + SSD o i3 NVME + EBS o SSD + HDD? o NVDIMM + NVMe ▪ Place larger compaction tiers on slow storage ▪ Move hot data to fast disks, cold data to slow disks
- 12. Managing write rate ▪ Limiting write speed now to prevent timeouts later ▪ Balancing between synchronous and asynchronous updates Talk: Nadav
- 13. Workload Isolation ▪ Dedicate resources to different workloads on the same data ▪ Multiple scenarios o OLTP vs. Ingestion o OLTP vs. Analytics o OLTP vs. Ingestion vs. Analytics Talk: Glauber
- 14. Multi-tenancy ▪ Build on workload isolation to fully isolate users o Full accounting of all node resources ▪ Share capacity o Split queues ▪ Issues we’re addressing o Metadata replication for small tables o Dynamic tenant growth o Security
- 15. Analytics
- 16. Full scan performance improvements ▪ Making queries stateful without impacting fault tolerance ▪ Driver improvements to reduce network hops Talk: Botond
- 17. ALLOW FILTERING ▪ Server-side row filtering
- 18. Future analytics driver improvements ▪ Token-aware drivers for range scans
- 20. Lightweight Transactions ▪ Last major missing feature ▪ Using the Raft algorithm ▪ Underlying infrastructure will be utilized for schema and cluster management ▪ Simplify Materialized Views implementation Talk: Duarte
- 21. MV + SI = GA! ▪ Materialized views propagate changes from one table to another ▪ Naive synchronous model impacts availability ▪ Asynchronous model requires backpressure Talk: Piotr
- 22. Improving streaming/repair performance ▪ CPU, storage and network bandwidth reductions ▪ Reduce false sharing • Cache/memtable bypass ▪ Isolation improvements Talk: Asias
- 23. Improved large partition support + Incremental sstable index reads + Incremental large partition cache eviction + Incremental large partition memtable merge
- 24. Shard-aware drivers ▪ Remove node-internal CPU-to- CPU hop ▪ Prevent overloaded cores ▪ Improve both throughput and latency ▪ Fully compatible Talk: Piotr
- 25. Observability
- 26. Monitoring stack 2.0 ▪ Improved dashboards ▪ Prometheus 2.x ▪ Improved deployment Talk: Amnon
- 27. Performance drilldown Using Scylla dashboards to diagnose performance problems Talk: Vlad Avi
- 28. Improving your applications ▪ Using “badness” metrics to find and eliminate inefficient code Talk: Shlomi
- 29. Manageability
- 30. Role-based access control ▪ Baseline for • Security • Audit • Workload isolation ▪ Increasing integration with external directories Talk: Jesse
- 31. Cluster-wide management ▪ No need to deal with individual nodes ▪ Stateful operation Talk: Michał
- 32. Scylla and Kubernetes ▪ Integrating with new datacenter operation technologies Talk: Moreno
- 33. Scylla Cloud ▪ Off-load all database management ▪ Significant cost reductions compared to other managed services ▪ Better performance in non-ideal scenarios
- 35. Improving Linux for Seastar/Scylla ▪ XFS performance improvements ▪ New AIO poll API
- 36. Seastar improvements ▪ CPU Scheduler ▪ I/O Scheduler v2 ▪ Native streaming
- 37. Seastar + Ceph ▪ Red Hat’s Ceph being rebuilt on top of Seastar Talk: Kefu
- 38. Thank You Any Questions ? Please stay in touch [email protected] @AviKivity