ScyllaDB’s Monstrous Engineering Advances by Avi Kivity
0 likes•130 views
Join ScyllaDB CTO Avi as he highlights the engineering milestones achieved over the past year, diving into new features, performance improvements, and innovations. Plus, get an exclusive sneak peek into what's coming next for ScyllaDB and the future of high-performance databases.
ScyllaDB’s Monstrous Engineering Advances by Avi Kivity
- 1. A ScyllaDB Community ScyllaDB Status Report Avi Kivity CTO
- 2. Avi Kivity ■ Linux KVM ■ First line written ■ Seastar ■ First line written ■ ScyllaDB ■ First line written
- 3. ■ 2024 Advances ■ 2025 Plans Agenda
- 4. ■ Maintenance & Operations ■ Security ■ Performance & Footprint ■ Reliability & Scaling ■ Features Agenda
- 6. Maintenance & Operations ■ Native nodetool ■ Maintenance socket ■ Task Management task_id type kind scope state sequence_number keyspace table entity shard start_time end_time a9b96262-3544-4189-8712-ef28abe69fb5 upgrade sstables compaction node keyspace running 1303 keyspace1 0 2025-02-24T21:32:08Z 657148a9-46f1-4093-8a90-2f239ff5faf8 upgrade sstables compaction node keyspace done 1301 system_distributed_everywhere 0 2025-02-24T21:32:08Z 2025-02-24T21:32:08Z aebdcb14-75dd-4360-a5f6-deb5e5f3d5fe upgrade sstables compaction node keyspace done 1300 system_auth 0 2025-02-24T21:32:08Z 2025-02-24T21:32:08Z 44bd1318-1c91-4f30-8596-c77a46efd30e upgrade sstables compaction node keyspace done 1275 system_schema 0 2025-02-24T21:31:43Z 2025-02-24T21:31:44Z 2be36997-5de6-4fcf-8044-0fc1d52bde4f upgrade sstables compaction node keyspace done 1270 system_distributed 0 2025-02-24T21:31:43Z 2025-02-24T21:31:43Z
- 7. Maintenance & Operations ■ Native backup/restore ■ Utilize full bandwidth via Seastar scheduler integration ■ Exercise S3 driver ■ Reduce Recovery Point Objective ■ Tablet-aware ■ Rack-aware ■ Also in load-and-stream 6TB, 6xi4i.8xlarge, with stress test, 3TB backup size Native ScyllaDB rclone Backup/ Minute 00:30:32 04:02:09 P99 Latency Read 9.05 3.41 P99 Latency Write 6.04 3.17
- 8. Repair improvements ■ Small-table repair performance ■ Automatic repair scheduling ■ Avoid whack-a-mole ■ Incremental repair ■ Repair more frequently
- 9. Security
- 10. Security - Enterprise Features ■ Encryption at Rest ■ LDAP Authentication and Authorization ■ Audit
- 11. Security - New ■ Transitional Authentication ■ Alternator Role-Based Access Control (RBAC)
- 13. LTO/PGO ■ Link-Time Optimization ■ Optimize the program as a whole rather than a collection of modules ■ Profile-Guided Optimization ■ Optimize the program based on actual behavior in a training run ■ 30% Improvement (combined) ■ Fewer/small nodes ■ More work!
- 14. Compression ■ Dictionary compression for RPC ■ Reduce cross-AZ cost ■ Dictionary compression for sstables ■ Reduce storage costs
- 15. 90% Storage Utilization with Tablets ■ Tablets can migrate at line speed ■ Up to 10 GB/s on large instances ■ Schema-independent ■ Launch new instances just-in-time ■ Decommission them when unneeded Time to finish Stream bandwidth Bytes on wire per tablet CPU Load Mutation based streaming 3003 seconds 100 MB/s 20090 MB 12% File based streaming 116 seconds 1000 MB/s 7280 MB 4% Difference 25X 10X 2.75X 3X
- 16. Storage Tiering ■ Place cold data on S3 ■ Cheap but slow ■ Data migration is transparent ■ Targeting time-series workloads ■ Using Time-Window Compaction Strategy
- 18. More system tables in Raft ■ Avoid repair and replication management chores ■ Avoid hot node problems ■ Avoid authentication problems during reduced availability periods ■ Moved: auth, service levels, CDC metadata
- 19. Coordinator-only (“zero-token”) nodes ■ Node that do not own any data ■ Symmetric -> two-tier architecture ■ Arbiter for two datacenters ✔ ✔ ✔ ✔ ✔ ✔ ✔
- 20. Non-Voting Nodes ■ Reduce Raft latency in large clusters ■ Maintain Quorum in asymmetric clusters ✔ ✔ ✔ ✔ ✔ ✔ ✔
- 21. Simplified Schema Healing ■ Replace merge-by-content with linear history ■ Eliminates stalls with large schemas
- 22. Materialized View Write Path Improvements ■ Improve throttling of base-table writes when view updates lag ■ Improve many-row DELETE performance
- 23. ■ Capture more details ■ Roles and grants ■ Hashed passwords ■ CDC DESCRIBE SCHEMA round-tripping cqlsh> DESCRIBE SCHEMA CREATE KEYSPACE ks1 WITH replication = {'class': 'org.apache.cassandra.locator.NetworkTopologyStrategy', 'datacenter1': '1'} AND durable_writes = true AND tablets = {'enabled': true}; CREATE TABLE ks1.t1 ( id int, c int, s1 set<int>, s2 set<int>, s3 frozen<set<int>>, PRIMARY KEY (id) ) WITH bloom_filter_fp_chance = 0.01 AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'} AND comment = '' AND compaction = {'class': 'IncrementalCompactionStrategy'} AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'} AND crc_check_chance = 1 AND default_time_to_live = 0 AND gc_grace_seconds = 864000 AND max_index_interval = 2048 AND memtable_flush_period_in_ms = 0 AND min_index_interval = 128 AND speculative_retry = '99.0PERCENTILE' AND tombstone_gc = {'mode': 'timeout', 'propagation_delay_in_seconds': '3600'}; CREATE TABLE ks1.t2 ( pk int, ck int, c int, PRIMARY KEY (pk, ck) ) WITH CLUSTERING ORDER BY (ck ASC) AND bloom_filter_fp_chance = 0.01 AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
- 24. ■ Internal addressing of nodes using UUID ■ Important when IP addresses are ephemeral ■ Kubernetes Reduced Reliance on IP Addresses
- 25. ■ Dynamically adjust Bloom filter memory ■ Drop filters during high memory demand ■ Reload filters when memory is available ■ Rebuild filter for optimal size Bloom Filter Dynamic Memory
- 26. ■ Reduces chance of bitrot ■ Mostly on non-compressed sstables ■ Compressed sstables already protected More SSTable Checksum Validation
- 27. CQL
- 28. CQL Features ■ WHERE … NOT IN ■ SELECT … set_intersection(set1, set2) ■ SELECT … USING SERVICE LEVEL = ‘...’
- 29. vector<> ALTER TABLE tab ADD embedding vector<float, 768> ■ Indexing = work in progress ■ Offload to dedicated search engine ■ Use GPU when available ■ Avoid scaling problems
- 30. Tablet plans
- 31. Tablets, Tablets, & Tablets ■ Complete feature integration with tablets ■ Materialized views (simple case) ■ Materialized views (complex cases) ■ Change Data Capture (CDC) ■ Lightweight Transactions (LWT) ■ Counter columns ■ Leverage fast scaling to reduce TCO ■ 90% storage utilization ■ Improve load-balancing algorithms ■ In-place migration from vnodes
- 33. Strong consistency model ■ Leverage previous Raft work for data plane consistency
- 34. Stay in Touch Avi Kivity [email protected] @AviKivity @avikivity