Cassandra Community Webinar | Introduction to Apache Cassandra 1.2
11 likes2,444 views
This document is a presentation on Apache Cassandra, focusing on its architecture, data model, and consistency mechanisms. It covers key concepts such as clusters, nodes, replication strategies, and data storage methods alongside examples using CQL for data manipulation. Additionally, it highlights the upcoming Cassandra Summit in June 2013.
Cassandra Community Webinar | Introduction to Apache Cassandra 1.2
- 1. CASSANDRA COMMUNITY WEBINARS APRIL 2013 INTRODUCTION TO APACHE CASSANDRA 1.2 Aaron Morton Apache Cassandra Committer, Data Stax MVP for Apache Cassandra @aaronmorton www.thelastpickle.com Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License
- 2. Cassandra Summit 2013 June 11 & 12 San Francisco Use SFSummit25 for 25% off
- 3. Cassandra Summit 2013 DataStax Ac*ademy Free certification during the summit.
- 4. Overview The Cluster The Node The Data Model
- 5. Cassandra - Started at Facebook - Open sourced in 2008 - Top Level Apache project since 2010.
- 7. Inspiration - Google Big Table (2006) - Amazon Dynamo (2007)
- 8. Why Cassandra? - Scale - Operations - Data Model
- 9. Overview The Cluster The Node The Data Model
- 10. Store ‘foo’ key with Replication Factor 3. Node 1 - 'foo' Node 2 - 'foo'Node 4 Node 3 - 'foo'
- 11. Consistent Hashing. - Evenly map keys to nodes - Minimise key movements when nodes join or leave
- 12. Partitioner. RandomPartitioner transforms Keys to Tokens using MD5. (Default pre version 1.2.)
- 13. Partitioner. Murmur3Partitioner transforms Keys to Tokens using Murmur3. (Default in version 1.2.)
- 14. Keys andTokens? token 0 99 key 'fop' 'foo' 10 90
- 15. Token Ring. 'foo' token: 90 'fop' token: 10 99 0
- 16. Token Ranges pre v1.2. Node 1 token: 0 Node 2 token: 25 Node 4 token: 75 Node 3 token: 50 1-2576-0
- 17. Token Ranges withVirtual Nodes in v1.2. Node 1 Node 2 Node 3 Node 4
- 18. LocateToken Range. Node 1 Node 2 Node 3 Node 4 'foo' token: 90
- 19. Replication Strategy selects Replication Factor number of nodes for a row.
- 20. SimpleStrategy with RF 3. Node 1 Node 2 Node 3 Node 4 'foo' token: 90
- 21. NetworkTopologyStrategy uses a Replication Factor per Data Centre. (Default.)
- 22. Multi DC Replication with RF 3 and RF 2. Node 1 Node 2 Node 3 Node 4 'foo' token: 90 Node 1 Node 2 Node 3 Node 4 West DC East DC
- 23. The Snitch knows which Data Centre and Rack the Node is in.
- 24. SimpleSnitch. Places all nodes in the same DC and Rack. (Default, there are others.)
- 25. EC2Snitch. DC is set to AWS Region and a Rack to Availability Zone.
- 26. The Client and the Coordinator. Node 1 Node 2 Node 3 Node 4 'foo' token: 90 Client
- 27. Multi DC Client and the Coordinator. Node 1 Node 2 Node 3 Node 4 'foo' token: 90 Client Node 10 Node 20 Node 30 Node 40
- 28. Gossip. Nodes share information with a small number of neighbours. Who share information with a small number of neigh..
- 29. Consistency Level (CL). - Specified for each request - Number of nodes to wait for.
- 30. Consistency Level (CL) - Any* - One,Two Three - QUORUM - LOCAL_QUORUM, EACH_QUOURM*
- 31. QUOURM at Replication Factor... Replication Factor QUOURM 2 or 3 4 or 5 6 or 7 2 3 4
- 32. Write ‘foo’ at QUOURM with Hinted Handoff. Node 1 Node 2 Node 3 Node 4 'foo' for #3 'foo' token: 90 Client
- 33. Read ‘foo’ at QUOURM. Node 1 Node 2 Node 3 Node 4 'foo' token: 90 Client
- 34. Column Timestamps used to resolve differences.
- 35. Resolving differences. Column Node 1 Node 2 Node 3 purple cromulent (timestamp 10) cromulent (timestamp 10) <missing> monkey embiggens (timestamp 10) embiggens (timestamp 10) debigulator (timestamp 5) dishwasher tomato (timestamp 10) tomato (timestamp 10) tomacco (timestamp 15)
- 36. Consistent read for ‘foo’ at QUOURM. Node 1 Node 2 Node 3 Node 4 Client cromulent cromulent <empty> Node 1 Node 2 Node 3 Node 4 Client cromulent cromulent
- 37. Strong Consistency W + R > N (#Write Nodes + #Read Nodes> Replication Factor)
- 38. Achieving Strong Consistency. - QUOURM Read + QUORUM Write - ALL Read + ONE Write - ONE Read + ALL Write
- 39. Achieving Consistency - Consistency Level - Hinted Handoff - Read Repair - Anti Entropy
- 40. Overview The Cluster The Node The Data Model
- 42. Write path Append to Write Ahead Log. (fsync every 10s by default, other options available)
- 43. Write path... Merge Columns into Memtable. (Lock free, always in memory.)
- 44. (Later.) Asynchronously flush Memtable to new files. (May be 10’s or 100’s of MB in size.)
- 45. Data is stored in immutable SSTables. (Sorted String table.)
- 47. SSTables. SSTable 1 foo: dishwasher (ts 10): tomato purple (ts 10): cromulent SSTable 2 foo: frink (ts 20): flayven monkey (ts 10): embiggins SSTable 3 SSTable 4 foo: dishwasher (ts 15): tomacco SSTable 5
- 48. Read purple, monkey, dishwasher. SSTable 1-Data.db foo: dishwasher (ts 10): tomato purple (ts 10): cromulent SSTable 2-Data.db foo: frink (ts 20): flayven monkey (ts 10): embiggins SSTable 3-Data.db SSTable 4-Data.db foo: dishwasher (ts 15): tomacco SSTable 5-Data.db Bloom Filter Index Sample SSTable 1-Index.db Bloom Filter Index Sample SSTable 2-Index.db Bloom Filter Index Sample SSTable 3-Index.db Bloom Filter Index Sample SSTable 4-Index.db Bloom Filter Index Sample SSTable 5-Index.db Memory Disk
- 49. Key Cache caches row key position in *-Data.db file. (Removes up to1disk seek per SSTable.)
- 50. Read with Key Cache. SSTable 1-Data.db foo: dishwasher (ts 10): tomato purple (ts 10): cromulent SSTable 2-Data.db foo: frink (ts 20): flayven monkey (ts 10): embiggins SSTable 3-Data.db SSTable 4-Data.db foo: dishwasher (ts 15): tomacco SSTable 5-Data.db Key Cache Index Sample SSTable 1-Index.db Key Cache Index Sample SSTable 2-Index.db Key Cache Index Sample SSTable 3-Index.db Key Cache Index Sample SSTable 4-Index.db Key Cache Index Sample SSTable 5-Index.db Memory Disk Bloom Filter Bloom Filter Bloom Filter Bloom Filter Bloom Filter
- 51. Row Cache caches entire row. (Removes all disk IO.)
- 52. Read with Row Cache. Row Cache SSTable 1-Data.db foo: dishwasher (ts 10): tomato purple (ts 10): cromulent SSTable 2-Data.db foo: frink (ts 20): flayven monkey (ts 10): embiggins SSTable 3-Data.db SSTable 4-Data.db foo: dishwasher (ts 15): tomacco SSTable 5-Data.db Key Cache Index Sample SSTable 1-Index.db Key Cache Index Sample SSTable 2-Index.db Key Cache Index Sample SSTable 3-Index.db Key Cache Index Sample SSTable 4-Index.db Key Cache Index Sample SSTable 5-Index.db Memory Disk Bloom Filter Bloom Filter Bloom Filter Bloom Filter Bloom Filter
- 53. Compaction merges truth from multiple SSTables into one SSTable with the same truth. (Manual and continuous background process.)
- 54. Compaction. Column SSTable 1 SSTable 2 SSTable 4 New purple cromulent (timestamp 10) <tombstone> (timestamp 15) <tombstone> (timestamp 15) monkey embiggens (timestamp 10) embiggens (timestamp 10) dishwasher tomato (timestamp 10) tomacco (timestamp 15) tomacco (timestamp 15)
- 55. Overview The Cluster The Node The Data Model
- 56. Cassandra is good at reading data from a row in the order it is stored.
- 57. Typically an efficient data model will denormalize data and use the storage engine order.
- 58. To create a good data model understand the queries your application requires.
- 59. API Choice Thrift Original and still fully supported API.
- 60. API Choice CQL3 New and fully supported API.
- 61. CQL 3 A Table Orientated, Schema Driven, Data Model and Query language similar to SQL.
- 62. CQL 3 A Table Orientated, Schema Driven, Data Model and Query language similar to SQL.
- 63. Twitter clone using CQL 3 via the cqlsh tool. bin/cqlsh
- 64. Queries? - Post Tweet to Followers - Get Tweet by ID - List Tweets by User - List Tweets in User Timeline - List Followers
- 66. Our Keyspace CREATE KEYSPACE cass_community WITH replication = {'class':'NetworkTopologyStrategy', 'datacenter1':1};
- 67. Table A sparse collection of well known, ordered columns.
- 68. FirstTable CREATE TABLE User ( user_name text, password text, real_name text, PRIMARY KEY (user_name) );
- 69. Some users cqlsh:cass_community> INSERT INTO User ... (user_name, password, real_name) ... VALUES ... ('fred', 'sekr8t', 'Mr Foo'); cqlsh:cass_community> select * from User; user_name | password | real_name -----------+----------+----------- fred | sekr8t | Mr Foo
- 70. Some users cqlsh:cass_community> INSERT INTO User ... (user_name, password) ... VALUES ... ('bob', 'pwd'); cqlsh:cass_community> select * from User where user_name = 'bob'; user_name | password | real_name -----------+----------+----------- bob | pwd | null
- 71. Data Model (so far) Table / Value User user_name Primary Key
- 72. TweetTable CREATE TABLE Tweet ( tweet_id bigint, body text, user_name text, timestamp timestamp, PRIMARY KEY (tweet_id) );
- 73. TweetTable... cqlsh:cass_community> INSERT INTO Tweet ... (tweet_id, body, user_name, timestamp) ... VALUES ... (1, 'The Tweet','fred',1352150816917); cqlsh:cass_community> select * from Tweet where tweet_id = 1; tweet_id | body | timestamp | user_name ----------+-----------+--------------------------+----------- 1 | The Tweet | 2012-11-06 10:26:56+1300 | fred
- 74. Data Model (so far) Table / Value User Tweet user_name Primary Key Field tweet_id Primary Key
- 75. UserTweetsTable CREATE TABLE UserTweets ( tweet_id bigint, user_name text, body text, timestamp timestamp, PRIMARY KEY (user_name, tweet_id) );
- 76. UserTweetsTable... cqlsh:cass_community> INSERT INTO UserTweets ... (tweet_id, body, user_name, timestamp) ... VALUES ... (1, 'The Tweet','fred',1352150816917); cqlsh:cass_community> select * from UserTweets where user_name='fred'; user_name | tweet_id | body | timestamp -----------+----------+-----------+-------------------------- fred | 1 | The Tweet | 2012-11-06 10:26:56+1300
- 77. UserTweetsTable... cqlsh:cass_community> select * from UserTweets where user_name='fred' and tweet_id=1; user_name | tweet_id | body | timestamp -----------+----------+-----------+-------------------------- fred | 1 | The Tweet | 2012-11-06 10:26:56+1300
- 78. UserTweetsTable... cqlsh:cass_community> INSERT INTO UserTweets ... (tweet_id, body, user_name, timestamp) ... VALUES ... (2, 'Second Tweet', 'fred', 1352150816918); cqlsh:cass_community> select * from UserTweets where user_name = 'fred'; user_name | tweet_id | body | timestamp -----------+----------+--------------+-------------------------- fred | 1 | The Tweet | 2012-11-06 10:26:56+1300 fred | 2 | Second Tweet | 2012-11-06 10:26:56+1300
- 79. UserTweetsTable... cqlsh:cass_community> select * from UserTweets where user_name = 'fred' order by tweet_id desc; user_name | tweet_id | body | timestamp -----------+----------+--------------+-------------------------- fred | 2 | Second Tweet | 2012-11-06 10:26:56+1300 fred | 1 | The Tweet | 2012-11-06 10:26:56+1300
- 80. UserTimeline CREATE TABLE UserTimeline ( user_name text, tweet_id bigint, tweet_user text, body text, timestamp timestamp, PRIMARY KEY (user_name, tweet_id) ) WITH CLUSTERING ORDER BY (tweet_id DESC);
- 81. UserTimeline cqlsh:cass_community> INSERT INTO UserTimeline ... (user_name, tweet_id, tweet_user, body, timestamp) ... VALUES ... ('fred', 1, 'fred', 'The Tweet',1352150816917); cqlsh:cass_community> INSERT INTO UserTimeline ... (user_name, tweet_id, tweet_user, body, timestamp) ... VALUES ... ('fred', 100, 'bob', 'My Tweet',1352150846917);
- 82. UserTimeline cqlsh:cass_community> select * from UserTimeline where user_name = 'fred'; user_name | tweet_id | body | timestamp | tweet_user -----------+----------+-----------+--------------------------+------------ fred | 100 | My Tweet | 2012-11-06 10:27:26+1300 | bob fred | 1 | The Tweet | 2012-11-06 10:26:56+1300 | fred
- 83. Data Model (so far) Table / Value User Tweet User Tweets User Timeline user_name Primary Key Field Primary Key Primary Key tweet_id Primary Key Primary Key Component Primary Key Component
- 84. UserMetricsTable CREATE TABLE UserMetrics ( user_name text, tweets counter, followers counter, following counter, PRIMARY KEY (user_name) );
- 85. UserMetricsTable... cqlsh:cass_community> UPDATE ... UserMetrics ... SET ... tweets = tweets + 1 ... WHERE ... user_name = 'fred'; cqlsh:cass_community> select * from UserMetrics where user_name = 'fred'; user_name | followers | following | tweets -----------+-----------+-----------+-------- fred | null | null | 1
- 86. Data Model (so far) Table / Value User Tweet User Tweets User Timeline User Metrics user_name Primary Key Field Primary Key Primary Key Primary Key tweet_id Primary Key Primary Key Component Primary Key Component
- 87. Relationships CREATE TABLE Followers ( user_name text, follower text, timestamp timestamp, PRIMARY KEY (user_name, follower) ); CREATE TABLE Following ( user_name text, following text, timestamp timestamp, PRIMARY KEY (user_name, following) );
- 88. Relationships cqlsh:cass_community> INSERT INTO ... Following ... (user_name, following, timestamp) ... VALUES ... ('bob', 'fred', 1352247749161); cqlsh:cass_community> INSERT INTO ... Followers ... (user_name, follower, timestamp) ... VALUES ... ('fred', 'bob', 1352247749161);
- 89. Relationships cqlsh:cass_community> select * from Following; user_name | following | timestamp -----------+-----------+-------------------------- bob | fred | 2012-11-07 13:22:29+1300 cqlsh:cass_community> select * from Followers; user_name | follower | timestamp -----------+----------+-------------------------- fred | bob | 2012-11-07 13:22:29+1300
- 90. Data Model Table / Value User Tweet User Tweets User Timeline User Metrics Follows Followers user_name Primary Key Field Primary Key Primary Key Primary Key Primary Key tweet_id Primary Key Primary Key Component Primary Key Component
- 91. Cassandra Summit 2013 June 11 & 12 San Francisco Use SFSummit25 for 25% off
- 92. Thanks.
- 93. Aaron Morton @aaronmorton www.thelastpickle.com Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License