Cassandra Day Atlanta 2015: Data Modeling 101

©2013 DataStax Conﬁdential. Do not distribute without consent.
@PatrickMcFadin
Patrick McFadin 
Chief Evangelist for Apache Cassandra
Introduction to Data Modeling
1

My Background
…ran into this problem

Gave it my best shot
shard 1 shard 2 shard 3 shard 4
router
client
Patrick,
All your wildest
dreams will come
true.

ACID vs CAP
ACID
CAP - Pick two
Atomic - All or none
Consistency - Only valid data is written
Isolation - One operation at a time
Durability - Once committed, it stays that way
Consistency - All data on cluster
Availability - Cluster always accepts writes
Partition tolerance - Nodes in cluster can’t talk to each other
Cassandra let’s you tune this

Relational Data Models
• 5 normal forms
• Foreign Keys
• Joins
deptId First Last
1 Edgar Codd
2 Raymond Boyce
id Dept
1 Engineering
2 Math
Employees
Department

Relational Modeling
Data
Models
Application

Cassandra Modeling
Data
Models
Application

CQL vs SQL
• No joins
• No aggregations
deptId First Last
1 Edgar Codd
2 Raymond Boyce
id Dept
1 Engineering
2 Math
Employees
Department
SELECT e.First, e.Last, d.Dept
FROM Department d, Employees e
WHERE ‘Codd’ = e.Last
AND e.deptId = d.id

Denormalization
• Combine table columns into a single view
• No joins
SELECT First, Last, Dept
FROM employees
WHERE id = ‘1’
id First Last Dept
1 Edgar Codd Engineering
2 Raymond Boyce Math
Employees

No more sequences
• Great for auto-creation of Ids
• Guaranteed unique
• Needs ACID to work. (Sorry. No sharding)
INSERT INTO user (id, firstName, LastName)
VALUES (seq.nextVal(), ‘Ted’, ‘Codd’)

No sequences???
• Almost impossible in a distributed system
• Couple of great choices
• Natural Key - Unique values like email
• Surrogate Key - UUID
• Universal Unique ID
• 128 bit number represented in character form
• Easily generated on the client
• Same as GUID for the MS folks
99051fe9-6a9c-46c2-b949-38ef78858dd0

KillrVideo.com
• Hosted on Azure
• Code on GitHub
• Also on your USB
• Data Model for examples

Entity Table
• Simple view of a single
user
• UUID used for ID
• Simple primary key
// Users keyed by id
CREATE TABLE users (
userid uuid,
firstname text,
lastname text,
email text,
created_date timestamp,
PRIMARY KEY (userid)
);
SELECT firstname, lastname
FROM user
WHERE userId = 99051fe9-6a9c-46c2-b949-38ef78858dd0

CQL Collections
• Meant to be dynamic part of table
• Update syntax is very different from insert
• Reads require all of collection to be read

CQL Set
• Set is sorted by CQL type comparator
INSERT INTO collections_example (id, set_example)
VALUES(1, {'1-one', '2-two'});
set_example set<text>
Collection name Collection type CQLType

CQL Set Operations
• Adding an element to the set
• After adding this element, it will sort to the beginning.
• Removing an element from the set
UPDATE collections_example
SET set_example = set_example + {'3-three'} WHERE id = 1;
SET set_example = set_example + {'0-zero'} WHERE id = 1;
SET set_example = set_example - {'3-three'} WHERE id = 1;

CQL List
• Ordered by insertion
• Use with caution
list_example list<text>
Collection name Collection type
INSERT INTO collections_example (id, list_example)
VALUES(1, ['1-one', '2-two']);
CQLType

CQL List Operations
• Adding an element to the end of a list
• Adding an element to the beginning of a list
• Deleting an element from a list
SET list_example = list_example + ['3-three']
WHERE id = 1;
SET list_example = ['0-zero'] + list_example
WHERE id = 1;
SET list_example = list_example - ['3-three'] WHERE id = 1;

CQL Map
• Key and value
• Key is sorted by CQL type comparator
INSERT INTO collections_example (id, map_example)
VALUES(1, { 1 : 'one', 2 : 'two' });
map_example map<int,text>
Collection name Collection type Value CQLTypeKey CQLType

CQL Map Operations
• Add an element to the map
• Update an existing element in the map
• Delete an element in the map
SET map_example[3] = 'three'
WHERE id = 1;
SET map_example[3] = 'tres'
WHERE id = 1;
DELETE map_example[3]
FROM collections_example
WHERE id = 1;

Entity with collections
• Same type of entity
• SET type for dynamic data
• tags for each video
// Videos by id
CREATE TABLE videos (
videoid uuid,
userid uuid,
name text,
description text,
location text,
location_type int,
preview_image_location text,
tags set<text>,
added_date timestamp,
PRIMARY KEY (videoid)
);

Index (or lookup) tables
• Table arranged to find data
• Denormalized for speed
• Find videos for a user
// One-to-many from user point of view (lookup table)
CREATE TABLE user_videos (
userid uuid,
videoid uuid,
name text,
PRIMARY KEY (userid, added_date, videoid)
) WITH CLUSTERING ORDER BY (added_date DESC, videoid ASC);

Primary Key
• First column name is the Partition Key
• Subsequent are the Clustering Columns
• Videos will be ordered by added_date and
userid uuid,
videoid uuid,
name text,

Primary key relationship
PRIMARY KEY (userId,added_date,videoId)

Partition Key

Partition Key Clustering Columns

A12378E55F5A32

2005:12:1:102005:12:1:92005:12:1:82005:12:1:7
5F22A0BC
F2B3652CFFB3652D7AB3652C
A12378E55F5A32
SELECT videoId FROM user_videos
WHERE userId = A12378E55F5A32
AND added_date = ‘2005-12-1’
AND videoId = 5F22A0BC

Clustering Order
• Clustering Columns have default order
• Use to specify order
• Bonus: Sorts on disk for speed
userid uuid,
videoid uuid,
name text,

Multiple Lookups
• Same data
• Different lookup pattern // Index for tag keywords
CREATE TABLE videos_by_tag (
tag text,
videoid uuid,
name text,
tagged_date timestamp,
PRIMARY KEY (tag, videoid)
);
// Index for tags by first letter in the tag
CREATE TABLE tags_by_letter (
first_letter text,
tag text,
PRIMARY KEY (first_letter, tag)
);

Many to Many Relationships
• Two views
• Different directions
• Insert data in a batch
// Comments for a given video
CREATE TABLE comments_by_video (
videoid uuid,
commentid timeuuid,
userid uuid,
comment text,
PRIMARY KEY (videoid, commentid)
) WITH CLUSTERING ORDER BY (commentid DESC);
// Comments for a given user
CREATE TABLE comments_by_user (
userid uuid,
commentid timeuuid,
videoid uuid,
comment text,
PRIMARY KEY (userid, commentid)
) WITH CLUSTERING ORDER BY (commentid DESC);

Example 1: Weather Station
• Weather station collects data
• Cassandra stores in sequence
• Application reads in sequence

Use case
• Store data per weather station
• Store time series in order: first to last
• Get all data for one weather station
• Get data for a single date and time
• Get data for a range of dates and times
Needed Queries
Data Model to support queries

Data Model
• Weather Station Id and Time
are unique
• Store as many as needed
CREATE TABLE temperature (
weather_station text,
year int,
month int,
day int,
hour int,
temperature double,
PRIMARY KEY (weather_station,year,month,day,hour)
);
INSERT INTO temperature(weather_station,year,month,day,hour,temperature)
VALUES (‘10010:99999’,2005,12,1,7,-5.6);
VALUES (‘10010:99999’,2005,12,1,8,-5.1);
VALUES (‘10010:99999’,2005,12,1,9,-4.9);
VALUES (‘10010:99999’,2005,12,1,10,-5.3);

Storage Model - Logical View
2005:12:1:7
-5.6
2005:12:1:8
-5.1
2005:12:1:9
-4.9
SELECT weather_station,hour,temperature
FROM temperature
WHERE weatherstation_id='10010:99999';
10010:99999
10010:99999
10010:99999
weather_station hour temperature
2005:12:1:10
-5.3
10010:99999

2005:12:1:12
-5.4
2005:12:1:11
-4.9-5.3-4.9-5.1
2005:12:1:7
-5.6
Storage Model - Disk Layout
2005:12:1:8 2005:12:1:9
10010:99999
2005:12:1:10
Merged, Sorted and Stored Sequentially
SELECT weather_station,hour,temperature
FROM temperature
WHERE weatherstation_id='10010:99999';

Query patterns
• Range queries
• “Slice” operation on disk
SELECT weatherstation,hour,temperature
FROM temperature
WHERE weatherstation=‘10010:99999'
AND year = 2005 AND month = 12 AND day = 1
AND hour >= 7 AND hour <= 10;
Single seek on disk
2005:12:1:12
-5.4
2005:12:1:11
-4.9-5.3-4.9-5.1
2005:12:1:7
-5.6
2005:12:1:8 2005:12:1:9
10010:99999
2005:12:1:10
Partition key for locality

Query patterns
• Range queries
• “Slice” operation on disk
Programmers like this
Sorted by event_time
2005:12:1:7
-5.6
2005:12:1:8
-5.1
2005:12:1:9
-4.9
10010:99999
10010:99999
10010:99999
weather_station hour temperature
2005:12:1:10
-5.3
10010:99999
SELECT weatherstation,hour,temperature
FROM temperature
WHERE weatherstation=‘10010:99999'
AND year = 2005 AND month = 12 AND day = 1
AND hour >= 7 AND hour <= 10;

Thank you!
Bring the questions
Follow me on twitter
@PatrickMcFadin

Cassandra Day Atlanta 2015: Data Modeling 101

More Related Content

What's hot (19)

Similar to Cassandra Day Atlanta 2015: Data Modeling 101 (20)

More from DataStax Academy (20)

Recently uploaded (20)

Cassandra Day Atlanta 2015: Data Modeling 101