Cassandra Virtual Node talk
16 likes10,351 views
The document discusses the introduction of virtual nodes in Cassandra 1.2. It explains that virtual nodes allow a single server to handle multiple token ranges, improving hardware utilization and simplifying operations. The transition involves changing configuration settings to enable multiple tokens per node and initiating a shuffling process to redistribute data. Virtual nodes provide benefits like faster rebuilds and adding new nodes without complex token management.
![Transitioning to vnodes
After all Cassandra instances have been reset
Initialize a shuffle operation
[patrick@cassandra0 ~]$ cassandra-shuffle create
Enable shuffling
[patrick@cassandra0 ~]$ cassandra-shuffle enable
List pending relocations*
[patrick@cassandra0 ~]$ cassandra-shuffle ls
Let’s walk through it...
*This is a slow op. Be patient.
©2012 DataStax
16
Friday, February 15, 13](https://image.slidesharecdn.com/visforvnodes-130215230441-phpapp02/85/Cassandra-Virtual-Node-talk-23-320.jpg)
Cassandra Virtual Node talk
- 1. V is for vnodes Patrick McFadin, Sr Solution Architect DataStax ©2012 DataStax 1 Friday, February 15, 13
- 2. Agenda for today • What is a node? • How vnodes work • Converting your cluster • Benefits ©2012 DataStax 2 Friday, February 15, 13
- 3. Since the beginning... Cassandra has had... Clusters, which have... Keyspaces, which have... Column Families, which have... ©2012 DataStax 3 Friday, February 15, 13
- 4. Row Keys Unique in a column family Can be up to 64k in size Can be sorted in the cluster Byte Ordered Partitioner OR... Can be randomly placed in cluster Random Partitioner ©2012 DataStax 4 Friday, February 15, 13
- 5. Row Keys How do you... • Create a random number? • Make sure the number is big enough? • Make it reproducible? MD5 does the job Input a Row Key MD5 Get a 128 bit number ©2012 DataStax 5 Friday, February 15, 13
- 6. Row Keys Input Get @PatrickMcFadin MD5 0xcfc2d0610aaa712a8c36711d08a2550a Input Get 8675309 MD5 0x6cc0d36686e6a433aa76f96773852d35 The number produced is a range between: 0 and 2128-1... but Cassandra uses 2127-1 2128 = 340,282,366,920,938,463,463,374,607,431,768,211,456 ...otherwise known as a HUGE number. ©2012 DataStax 6 Friday, February 15, 13
- 7. ©2012 DataStax 7 Friday, February 15, 13
- 8. Token Assignment • Each Cassandra node is assigned a token • Each token is a number inside the huge range • Tokens mark the ownership range of Row Keys From: Token = 0 To: Token = 56713727820156410577229101238628035242 From: To: Token = 113427455640312821154458202477256070484 ©2012 DataStax 8 Friday, February 15, 13
- 9. Row Key to Token Input Get @PatrickMcFadin MD5 276161727147663567581939045564154008842 Token = 0 I’ll Token = 56713727820156410577229101238628035242 take it! Token = 113427455640312821154458202477256070484 ©2012 DataStax 9 Friday, February 15, 13
- 10. Row Key to Token Input Get @PatrickMcFadin MD5 276161727147663567581939045564154008842 Token = 0 I’ll Token = 56713727820156410577229101238628035242 take it! Token = 113427455640312821154458202477256070484 ©2012 DataStax 9 Friday, February 15, 13
- 11. Row Key to Token Input Get @PatrickMcFadin MD5 276161727147663567581939045564154008842 Token = 0 I’ll Token = 56713727820156410577229101238628035242 take it! Token = 113427455640312821154458202477256070484 ©2012 DataStax 9 Friday, February 15, 13
- 12. Cassandra 1.1 Node • Responsible for a single range of keys • Range determined by single token • One server = One token = One node ©2012 DataStax 10 Friday, February 15, 13
- 13. Cassandra 1.1 Node • Responsible for a single range of keys • Range determined by single token • One server = One token = One node ©2012 DataStax 10 Friday, February 15, 13
- 14. Cassandra 1.1 Node • Responsible for a single range of keys • Range determined by single token • One server = One token = One node Commodity node? ©2012 DataStax 10 Friday, February 15, 13
- 15. Cassandra 1.1 Node • Responsible for a single range of keys • Range determined by single token • One server = One token = One node Commodity node? ©2012 DataStax 10 Friday, February 15, 13
- 16. Cassandra 1.1 Node • Responsible for a single range of keys • Range determined by single token • One server = One token = One node Commodity node? What you really want. ©2012 DataStax 10 Friday, February 15, 13
- 17. Cassandra 1.1 Node • Responsible for a single range of keys • Range determined by single token • One server = One token = One node Commodity node? What you really want. ©2012 DataStax 10 Friday, February 15, 13
- 18. Time for a new plan • Hardware is only getting bigger • One node is responsible for more data • Token assignments are a pain ©2012 DataStax 11 Friday, February 15, 13
- 19. Token assignment (sucks) • Tokens need to be evenly spread • Growing a ring... not good options • Shrinking a ring... not good options • Tokens have to be added to each server config ©2012 DataStax 12 Friday, February 15, 13
- 20. Enter Virtual Nodes • One server should have many nodes • Each node should be small • Tokens should be automatic Version 1.1 Version 1.2 Server 1 Server 1 1 2 1-4 4 3 ©2012 DataStax 13 Friday, February 15, 13
- 21. Virtual Node Features • Default 256 Nodes per server • Auto assign tokens • Faster rebuilds of servers • Faster server add to cluster • New partitioner (More later) ©2012 DataStax 14 Friday, February 15, 13
- 22. Transitioning to vnodes Super easy! Find these lines in your cassandra.yaml file: #num_tokens: initial_token: <some big number> Change to: num_tokens: 256 initial_token: and restart. Repeat on all nodes in cluster ©2012 DataStax 15 Friday, February 15, 13
- 23. Transitioning to vnodes After all Cassandra instances have been reset Initialize a shuffle operation [patrick@cassandra0 ~]$ cassandra-shuffle create Enable shuffling [patrick@cassandra0 ~]$ cassandra-shuffle enable List pending relocations* [patrick@cassandra0 ~]$ cassandra-shuffle ls Let’s walk through it... *This is a slow op. Be patient. ©2012 DataStax 16 Friday, February 15, 13
- 24. Existing 1.1 cluster Server 1 Server 2 1-4 4-8 Server 4 Server 3 13-16 9-12 ©2012 DataStax Friday, February 15, 13
- 25. Set num_tokens and restart Server 1 Server 2 1-4 1-4 4-8 4-8 1-4 1-4 4-8 4-8 Server 4 Server 3 13-16 13-16 9-12 9-12 13-16 13-16 9-12 9-12 ©2012 DataStax 18 Friday, February 15, 13
- 26. Set num_tokens and restart Server 1 Server 2 1 2 5 6 3 4 7 8 Server 4 Server 3 13 14 9 10 15 16 11 12 ©2012 DataStax Initialize and Enable shuffling... 19 Friday, February 15, 13
- 27. Shuffle enable Server 1 Server 2 1 5 2 6 13 9 14 10 Server 4 Server 3 3 7 4 8 16 12 15 11 ©2012 DataStax 20 Friday, February 15, 13
- 28. Shuffle complete Server 1 Server 2 1 5 2 6 13 9 14 10 Server 4 Server 3 3 7 4 8 16 12 15 11 ©2012 DataStax 21 Friday, February 15, 13
- 29. Ops life with vnodes • Add any number of nodes • No token assignments! • Bigger server? Larger num_tokens • Decommission any number of nodes • New nodetool command: status One more time now! ©2012 DataStax 22 Friday, February 15, 13
- 30. Bonus new thing • New Partitioner: Murmur3Partitoner • Murmur3 replaces MD5 • Slightly faster than MD5 in certain cases • Go forward partitioner for NEW clusters • No need to convert More details here: https://issues.apache.org/jira/browse/CASSANDRA-3772 ©2012 DataStax 23 Friday, February 15, 13
- 31. In conclusion... Go out and try some vnode love today! Download Cassandra 1.2 now http://www.datastax.com/download/community http://cassandra.apache.org/download/ ©2012 DataStax 24 Friday, February 15, 13
- 32. Some handy references http://www.datastax.com/dev/blog/virtual-nodes-in-cassandra-1-2 http://www.datastax.com/dev/blog/upgrading-an-existing-cluster-to-vnodes Follow me on Twitter for more: @PatrickMcFadin ©2012 DataStax 25 Friday, February 15, 13
- 33. We power the apps that transform business. ©2012 DataStax 26 Friday, February 15, 13