CQL3 and Data Modeling 101 with Apache Cassandra
1 like1,155 views
This document provides an overview of Cassandra data modeling concepts using CQL. It discusses Cassandra properties like being column-oriented and distributed. It explains key Cassandra containers like keyspaces, column families, rows and columns. Examples are given of modeling user, handle and event data using tables with composite keys and columns. The document emphasizes designing data models for idempotency and reading performance.
![How do I create these?
[default@userdb] create column family usersCF;
5ecec19a-3a43-3490-8c9a-3eb2901e2e97
Waiting for schema agreement...
... schemas agree across the cluster
[default@userdb] create column family handleCF;
df82135c-eb1f-3abf-b9df-02c605d571d5
Waiting for schema agreement...
... schemas agree across the cluster](https://image.slidesharecdn.com/cql3anddatamodeling101withapachecassandra-sdcassndrameetup-20140227-140228010638-phpapp01/85/CQL3-and-Data-Modeling-101-with-Apache-Cassandra-23-320.jpg)
![How do I insert data?
[default@userdb] set handleCF[utf8(âbb@DALâ)]
⊠[utf8(âNAMEâ)] = utf8('Bobtholomew');
Value inserted.
Elapsed time: 22 msec(s).
[default@userdb] set handleCF[utf8(âbb@DAL')]
⊠[utf8(âEMAILâ)] = utf8(âbb@example.comâ);
Value inserted.
Elapsed time: 3.43 msec(s).](https://image.slidesharecdn.com/cql3anddatamodeling101withapachecassandra-sdcassndrameetup-20140227-140228010638-phpapp01/85/CQL3-and-Data-Modeling-101-with-Apache-Cassandra-24-320.jpg)
![âTypesâ
[default@userdb] set handleCF[utf8(âbb@DALâ)]
!
âą
Key Validator
âą
(Column) Comparator
âą
(Column Value) Default Validator, Metadata
âą
BytesType, AsciiType, UTF8Type, IntegerType,
Int32Type, LongType, UUIDType, TimeUUIDType,
DateType, BooleanType, FloatType, DoubleType,
DecimalType, CounterColumnType (, CompositeTypes)](https://image.slidesharecdn.com/cql3anddatamodeling101withapachecassandra-sdcassndrameetup-20140227-140228010638-phpapp01/85/CQL3-and-Data-Modeling-101-with-Apache-Cassandra-27-320.jpg)
![CQL
âą
Thrift alternative
âą
Simpler API
âą
Hides the structure of the
internal storage
âą
3 generations
âą
Only looking at CQL3 here
âą
cqlsh [-3]](https://image.slidesharecdn.com/cql3anddatamodeling101withapachecassandra-sdcassndrameetup-20140227-140228010638-phpapp01/85/CQL3-and-Data-Modeling-101-with-Apache-Cassandra-32-320.jpg)
CQL3 and Data Modeling 101 with Apache Cassandra
- 1. CQL3 & Data Modeling 101 with Apache Cassandra San Diego Cassandra Meetup Feb 27, 2014
- 2. Not how youâve done it beforeâŠ
- 3. In the beginning⊠There was the Row, and the Column And the Row was fast to ïŹnd and scale, And the Column was fast to order.
- 6. Column Oriented âą Columns actually hold the data âą Key/Value pair âą Name can be used to store meaning as well
- 7. Distributed Database âą Rows are used to distribute âą C* pulls the entire row into memory âą Can pull out individual parts or write to individual parts, but itâs still considered together
- 8. Log Structured Updates âą Commitlog and sstables are log structured âą Oriented around appending (streaming at a know location) âą ==> Writes quickly âą And you want to avoid rewrites
- 9. Random Reads âą Data is scattered around the store (have to get location and random read to look it up) âą Some indexing, and hopefully itâs in the vfs page cache, but still. âą ==> Reads âslowerâ
- 10. General Rules of Thumb âą De-normalize Everything âą Duplicate your data âą Organize it for reading
- 11. Containers
- 12. Keyspace âą For modeling, not much âą All data lives inside of Keyspaces
- 13. Columnfamily âą aka Table âą Grouping of similar data âą Has unique key/row space âą Where some structure is applied
- 14. Row âą Unique inside of a column family âą Key/Value where the Value is all of the columns in the row âą Can handle some additional meaning to the row name âą Typically âbucketingâ
- 15. Column âą Holds data values âą Key/Value âą Can have meaning in the column name as well
- 16. Thrift Interface âą Operates on the raw rows and columns âą Many different language drivers âą Can use cassandra-cli to do this on the command line
- 17. Data Patterns
- 19. Lookup by chat handle ICQ CF IRC CF EMAIL 89403270 EMAIL bb@DAL bb@example .com [email protected] EMAIL steve@ DARK steve@my .net
- 20. Lookup by chat handle ICQ CF EMAIL IRC CF Name 89403270 EMAIL Name bb@example .com Bobtholomew EMAIL Name steve@my .net Steve bb@DAL [email protected] John steve@ DARK
- 23. How do I create these? [default@userdb] create column family usersCF; 5ecec19a-3a43-3490-8c9a-3eb2901e2e97 Waiting for schema agreement... ... schemas agree across the cluster [default@userdb] create column family handleCF; df82135c-eb1f-3abf-b9df-02c605d571d5 Waiting for schema agreement... ... schemas agree across the cluster
- 24. How do I insert data? [default@userdb] set handleCF[utf8(âbb@DALâ)] ⊠[utf8(âNAMEâ)] = utf8('Bobtholomew'); Value inserted. Elapsed time: 22 msec(s). [default@userdb] set handleCF[utf8(âbb@DAL')] ⊠[utf8(âEMAILâ)] = utf8(â[email protected]â); Value inserted. Elapsed time: 3.43 msec(s).
- 25. Users CF - TAGS mac@mac .com NAME TWITTER TAGS mac @macmceniry admin,super,cool jsmith@mac .com NAME ICQ Employer Hobby John 89403270 Smithco Miniature Horses bb@example .com NAME IRC Bobtholomew bb@DAL NAME TWITTER Food TAGS Elizabeth @liz Cheesecake admin NAME IRC Steven steve@DARK liz@example .com steve@my .net
- 26. Users CF - TAGS mac@mac .com jsmith@mac .com bb@example .com liz@example .com steve@my .net NAME TWITTER TAGS:admin TAGS:cool mac @macmceniry NAME ICQ Employer Hobby John 89403270 Smithco Miniature Horses NAME IRC Bobtholomew bb@DAL NAME TWITTER Food TAGS:admin Elizabeth @liz Cheesecake NAME IRC Steven steve@DARK TAGS:super
- 27. âTypesâ [default@userdb] set handleCF[utf8(âbb@DALâ)] ! âą Key Validator âą (Column) Comparator âą (Column Value) Default Validator, Metadata âą BytesType, AsciiType, UTF8Type, IntegerType, Int32Type, LongType, UUIDType, TimeUUIDType, DateType, BooleanType, FloatType, DoubleType, DecimalType, CounterColumnType (, CompositeTypes)
- 28. Whatâs in a name? âą Can use row names and column names to add meaning âą Row name meaning creates a new distribution bin âą Column name meaning can create a data hierarchy âą No real change to the column family creation in the thrift interface (well, types depending on what youâre doing)
- 30. That was then, this isâŠ
- 31. Now âą Same underlying structure none of that has changed âą âą âą Rows - reference quickly use for searching Columns - scan quickly user for ordering But now have usage patterns âą Some have been codiïŹed into CQL
- 32. CQL âą Thrift alternative âą Simpler API âą Hides the structure of the internal storage âą 3 generations âą Only looking at CQL3 here âą cqlsh [-3]
- 33. How does handle look here? cqlsh:userdb> CREATE TABLE handles ( ⊠handlename VARCHAR, ⊠email VARCHAR, ⊠name VARCHAR, ⊠PRIMARY KEY (handlename) ⊠); cqlsh:userdb> INSERT INTO handles ⊠(handlename, email, name) VALUES ⊠(âbb@DALâ, â[email protected]â, âBobtholomewâ);
- 35. How does handle look here? cqlsh:userdb> SELECT * FROM handles; handlename | email | name ââââââââââââ|ââââââââââââââââ|âââââââââââââ bb@DAL | [email protected] | Bobtholomew cqlsh:userdb> SELECT * FROM handles WHERE ⊠handlename = âbb@DALâ; handlename | email | name ââââââââââââ|ââââââââââââââââ|âââââââââââââ bb@DAL | [email protected] | Bobtholomew
- 36. How do I change it? cqlsh:userdb> UPDATE handles SET email=ânoneâ ⊠WHERE handlename = âbb@DALâ; cqlsh:userdb> SELECT * FROM handles; handlename | email | name ââââââââââââ|ââââââââââââââââ|âââââââââââââ bb@DAL | none | Bobtholomew
- 37. upsert âą Update instead of Insert âą âą Does the same thing (as long as itâs not a key) Insert instead of Update âą Overwrites data if itâs already there
- 38. What about our event buckets from earlier? âą Can do the same thing âą Creating a composite key âą âą USERNAME:DATE Creating a composite column âą hh:mm:ss
- 40. cqlsh:userdb> CREATE TABLE events ( ⊠username VARCHAR, ⊠d VARCHAR, ⊠hr INT, ⊠min INT, ⊠sec INT, ⊠event VARCHAR, ⊠PRIMARY KEY ( (username,d), hr, min, sec ) );
- 41. ⊠PRIMARY KEY ( (username,d), hr, min, sec ) ); ROW NAME (C* 1.2) COLUMN NAME
- 42. Tags âą CQL has collections âą map, list, set âą Collections are build similar to small/special composite columns âą Can add to our existing handle table
- 43. cqlsh:userdb> ALTER TABLE handles ADD tags SET; cqlsh:userdb> UPDATE TABLE handles ⊠SET tags = (âadminâ, âfooâ); email name [email protected] Bobtholomew bb@DAL tags:admin tags:foo
- 44. Design the data model so that itâs idempotent (eBay) âą Counter versus Collection (what question is being asked?) Count 100 Count 200 Count 300 Likes A Likes B Likes C Likes A Likes B Likes C +user11 1393287359 +user12 1393287359 +user11 1393287359 +user12 1393280912 -user11 1393281942 +user13 1393212345 1393287100 +user12 1393287100 1393287100 +user13 1393287100 1393287100 +user14 1393287100
- 45. Go Forth and Model! Thank You! PS⊠Sony Network is hiring!