SlideShare a Scribd company logo

Schema Design for Riak (Take 2)

Download as KEY, PDF
••
Sean Cribbs, profile picture
Sean Cribbs

The document discusses schema design principles for Riak, emphasizing the transition from relational to non-relational paradigms. It highlights the advantages of flexible data structures, reduced operational complexity, and improved scaling while acknowledging losses associated with traditional relational features like foreign keys and ACID properties. Key recommendations include focusing on common access patterns, denormalizing data, and utilizing caches for efficiency.

Technology
1 of 143
Downloaded 101 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
Schema Design for Riak Sean Cribbs Developer Advocate b a sho
Thinking Non-Relationally b a sho
“There is no spoon schema.” b a sho
There is an application. b a sho
What you lose b a sho
What you lose •Tables b a sho
What you lose •Tables •Foreign keys and constraints b a sho
What you lose •Tables •Foreign keys and constraints •ACID b a sho
What you lose •Tables •Foreign keys and constraints •ACID •Sophisticated query planners b a sho
What you lose •Tables •Foreign keys and constraints •ACID •Sophisticated query planners •Declarative query language (SQL) b a sho
What you gain b a sho
What you gain •More flexible, fluid designs b a sho
What you gain •More flexible, fluid designs •More natural data representations b a sho
What you gain •More flexible, fluid designs •More natural data representations •Scaling without pain b a sho
What you gain •More flexible, fluid designs •More natural data representations •Scaling without pain •Reduced operational complexity b a sho
Walking without Relational crutches b a sho
Walking without Relational crutches •Sparse data (optional/multi-value fields) b a sho
Walking without Relational crutches •Sparse data (optional/multi-value fields) •Richer data structures b a sho
Walking without Relational crutches •Sparse data (optional/multi-value fields) •Richer data structures •Meaningful identifiers b a sho
Walking without Relational crutches •Sparse data (optional/multi-value fields) •Richer data structures •Meaningful identifiers •Innovative access patterns b a sho
Know your data b a sho
Make your Top 10 list b a sho
Make your Top 10 list •Frequently requested pages/screens b a sho
Make your Top 10 list •Frequently requested pages/screens •Slow queries b a sho
Make your Top 10 list •Frequently requested pages/screens •Slow queries •Secondary indexes b a sho
Make your Top 10 list •Frequently requested pages/screens •Slow queries •Secondary indexes •Complicated joins or aggregations b a sho
Analyze b a sho
Analyze •Interdependencies, coupling b a sho
Analyze •Interdependencies, coupling •Cardinalities of relationships b a sho
Analyze •Interdependencies, coupling •Cardinalities of relationships •Access pattern b a sho
Analyze •Interdependencies, coupling •Cardinalities of relationships •Access pattern •Shoehorned structures b a sho
Example b a sho
Radiant CMS radiantcms.org b a sho
The “Styled Blog” Template b a sho
Date-organized pages The “Styled Blog” Template b a sho
Static sidebar content Date-organized pages The “Styled Blog” Template b a sho
Administration UI b a sho
hierarchical site structure Administration UI b a sho
add new pages by parent hierarchical site structure Administration UI b a sho
add new pages by parent special “virtual” pages hierarchical site structure Administration UI b a sho
b a sho
slug, breadcrumb b a sho
slug, breadcrumb content blocks b a sho
slug, breadcrumb content blocks layout b a sho
b a sho
template tags b a sho
Content-Type template tags b a sho
Relational Schema pages b a sho
Relational Schema parent pages b a sho
Relational Schema parent pages page_parts b a sho
Relational Schema parent pages page_parts layouts b a sho
Relational Schema parent pages page_parts layouts snippets b a sho
Relational Schema parent pages page_parts layouts users snippets b a sho
Relational Schema parent pages page_parts created/modified by layouts users snippets b a sho
Converting to Riak b a sho
Layouts and Snippets b a sho
Layouts and Snippets • Layouts and Snippets are accessed by name b a sho
Layouts and Snippets • Layouts and Snippets are accessed by name • SQL: WHERE name=? b a sho
Layouts and Snippets • Layouts and Snippets are accessed by name layouts/Main • SQL: WHERE name=? • Simple access by key snippets/top-nav b a sho
Layouts and Snippets • Layouts and Snippets are accessed by name layouts/Main • SQL: WHERE name=? • Simple access by key snippets/top-nav • Simple value structure (content + metadata) b a sho
Users Normally accessed by login or email: WHERE login=? OR email=? b a sho
Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key b a sho
Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • Easy lookup on one, manual index other b a sho
Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • Easy lookup on one, manual index other • #2: Arbitrary key b a sho
Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • Easy lookup on one, manual index other • #2: Arbitrary key • Independent of email/login changes b a sho
Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • Easy lookup on one, manual index other • #2: Arbitrary key • Independent of email/login changes • Manually index both b a sho
Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • #3: Users-list object • Easy lookup on one, manual index other • #2: Arbitrary key • Independent of email/login changes • Manually index both b a sho
Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • #3: Users-list object • Easy lookup on one, • Fits “small teams” manual index other philosophy • #2: Arbitrary key • Independent of email/login changes • Manually index both b a sho
Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • #3: Users-list object • Easy lookup on one, • Fits “small teams” manual index other philosophy • #2: Arbitrary key • Quick lookup (one fetch) • Independent of email/login changes • Manually index both b a sho
Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • #3: Users-list object • Easy lookup on one, • Fits “small teams” manual index other philosophy • #2: Arbitrary key • Quick lookup (one fetch) • Independent of email/login changes • Bad for large list, still need unique IDs • Manually index both b a sho
The Bigger Problems b a sho
Page Rendering b a sho
Page Rendering • Load and render layout in page context b a sho
Page Rendering • Load and render layout in page context • Render parts, snippets via template tags b a sho
Page Rendering • Load and render layout in page context • Render parts, snippets via template tags • Iterate over sections of page hierarchy b a sho
Rendering Page Parts b a sho
Rendering Page Parts •Parts are dependent on the page b a sho
Rendering Page Parts •Parts are dependent on the page •Compose / Denormalize (part of page) b a sho
Rendering Page Parts •Parts are dependent on the page •Compose / Denormalize (part of page) •Reduces lookups, conceptual unity, no index needed b a sho
Rendering Page Parts •Parts are dependent on the page •Compose / Denormalize (part of page) •Reduces lookups, conceptual unity, no index needed • Larger objects, no lazy fetching b a sho
Denormalized Page { title:”Home Page”, parts:[ {name:”body”, content:”...”}, {name:”sidebar”, content:”...”}, ], // ... } b a sho
Denormalized Page { title:”Home Page”, parts:[ {name:”body”, content:”...”}, {name:”sidebar”, content:”...”}, ], // ... } dependent objects inline b a sho
Site Hierarchy ...or, how to find a page (currently) b a sho
Site Hierarchy ...or, how to find a page (currently) • Start at the root: WHERE parent_id IS NULL b a sho
Site Hierarchy ...or, how to find a page (currently) • Start at the root: WHERE parent_id IS NULL • Check if the current page matches URL b a sho
Site Hierarchy ...or, how to find a page (currently) • Start at the root: WHERE parent_id IS NULL • Check if the current page matches URL • Check children for matching path segment, recurse b a sho
Site Hierarchy ...or, how to find a page (currently) • Start at the root: WHERE parent_id IS NULL • Check if the current page matches URL • Check children for matching path segment, recurse • Return “not found” page or 404 b a sho
Site Hierarchy ...or, how to find a page (analysis) b a sho
Site Hierarchy ...or, how to find a page (analysis) • O(log N) queries to find requested page b a sho
Site Hierarchy ...or, how to find a page (analysis) • O(log N) queries to find requested page • Index on parent_id speeds retrieval b a sho
Site Hierarchy ...or, how to find a page (analysis) • O(log N) queries to find requested page • Index on parent_id speeds retrieval • Traversing/iterating for content generation b a sho
Site Hierarchy ...or, how to find a page (analysis) • O(log N) queries to find requested page • Index on parent_id speeds retrieval • Traversing/iterating for content generation • Generating URL paths b a sho
Site Hierarchy ...or, how to find a page (in Riak) b a sho
Site Hierarchy ...or, how to find a page (in Riak) • Blog post: http://ow.ly/3jDAO b a sho
Site Hierarchy ...or, how to find a page (in Riak) • Blog post: http://ow.ly/3jDAO • #1 Parent/child links b a sho
Site Hierarchy ...or, how to find a page (in Riak) • Blog post: http://ow.ly/3jDAO • #1 Parent/child links • #2 Material path key b a sho
Site Hierarchy ...or, how to find a page (in Riak) • Blog post: http://ow.ly/3jDAO • #1 Parent/child links • #2 Material path key • #3 Tree object b a sho
#1: Parent/Child Links A Natural Tree A B C D E F b a sho
#1: Parent/Child Links A Natural Tree • Easy to understand & A implement B C D E F b a sho
#1: Parent/Child Links A Natural Tree </riak/pages/C>; riaktag=”child” • Easy to understand & A implement </riak/pages/A>; riaktag=”parent” • Doubly-linked for easy traversal in either B C direction D E F b a sho
#1: Parent/Child Links A Natural Tree </riak/pages/C>; riaktag=”child” • Easy to understand & A implement </riak/pages/A>; riaktag=”parent” • Doubly-linked for easy traversal in either B C direction • Easy to move entire subtrees D E F b a sho
#1: Parent/Child Links A Natural Tree A B C D E F b a sho
#1: Parent/Child Links A Natural Tree A • Child order is arbitrary B C D E F b a sho
#1: Parent/Child Links A Natural Tree A • Child order is arbitrary • Still O(log N) traversal B C D E F b a sho
#1: Parent/Child Links A Natural Tree A • Child order is arbitrary • Still O(log N) traversal B C • Two writes to add or update D E F b a sho
#2: Material Path Key “Best Guess” discovery _root_ (A) B C B/D C/E “pages” bucket b a sho
#2: Material Path Key “Best Guess” discovery _root_ (A) • Use path-to-page as key B C B/D C/E “pages” bucket b a sho
#2: Material Path Key “Best Guess” discovery _root_ (A) • Use path-to-page as key B • Best case: one lookup C to find requested page B/D C/E “pages” bucket b a sho
#2: Material Path Key “Best Guess” discovery _root_ (A) • Use path-to-page as key B • Best case: one lookup C to find requested page B/D • Ancestor pages listed inside object C/E “pages” bucket b a sho
#2: Material Path Key “Best Guess” discovery { _root_ (A) • Use path-to-page as title:”D”, key ancestors:[ B “B”, • Best case: one lookup “_root_” C to find requested page ] B/D • Ancestor pages listed inside object C/E “pages” bucket b a sho
#2: Material Path Key “Best Guess” discovery _root_ (A) B C B/D C/E “pages” bucket b a sho
#2: Material Path Key “Best Guess” discovery • Large update cost for _root_ (A) internal nodes B C B/D C/E “pages” bucket b a sho
#2: Material Path Key “Best Guess” discovery • Large update cost for _root_ (A) internal nodes • Dynamic URLs hard B C B/D C/E “pages” bucket b a sho
#2: Material Path Key “Best Guess” discovery • Large update cost for _root_ (A) internal nodes • Dynamic URLs hard B • Key-filters (0.14) C needed for efficient B/D child lists C/E “pages” bucket b a sho
#2: Material Path Key “Best Guess” discovery • Large update cost for _root_ (A) internal nodes • Dynamic URLs hard B • Key-filters (0.14) C needed for efficient B/D child lists • Need fallback for C/E “misses” “pages” bucket b a sho
#3: Tree Object “Branches and Leaves” { key:”A”, children:[ {key:”B”, children:[“D”]}, {key:”C”, children:[“E”,”F”]} ] } b a sho
#3: Tree Object “Branches and Leaves” • One request to get site structure { key:”A”, children:[ {key:”B”, children:[“D”]}, {key:”C”, children:[“E”,”F”]} ] } b a sho
#3: Tree Object “Branches and Leaves” • One request to get site structure { key:”A”, • Separates content from children:[ {key:”B”, organization children:[“D”]}, {key:”C”, children:[“E”,”F”]} ] } b a sho
#3: Tree Object “Branches and Leaves” • One request to get site structure { key:”A”, • Separates content from children:[ {key:”B”, organization children:[“D”]}, {key:”C”, • Intrinsic ordering ] children:[“E”,”F”]} } b a sho
#3: Tree Object “Branches and Leaves” • One request to get site structure { key:”A”, • Separates content from children:[ {key:”B”, organization children:[“D”]}, {key:”C”, • Intrinsic ordering ] children:[“E”,”F”]} } • Massive structure changes are quick b a sho
#3: Tree Object “Branches and Leaves” { key:”A”, children:[ {key:”B”, children:[“D”]}, {key:”C”, children:[“E”,”F”]} ] } b a sho
#3: Tree Object “Branches and Leaves” • Large site = large tree object (expensive to transfer) { key:”A”, children:[ {key:”B”, children:[“D”]}, {key:”C”, children:[“E”,”F”]} ] } b a sho
#3: Tree Object “Branches and Leaves” • Large site = large tree object (expensive to transfer) { key:”A”, • Some metadata children:[ {key:”B”, (slug?)will need to be children:[“D”]}, {key:”C”, in-tree for efficient children:[“E”,”F”]} traversal ] } b a sho
#3: Tree Object “Branches and Leaves” • Large site = large tree object (expensive to transfer) { key:”A”, • Some metadata children:[ {key:”B”, (slug?)will need to be children:[“D”]}, {key:”C”, in-tree for efficient children:[“E”,”F”]} traversal ] } • Multiple writers problematic b a sho
Hybrid Solutions TIMTOWTDI b a sho
Hybrid Solutions TIMTOWTDI •Material paths for quick lookups, links for relative traversal b a sho
Hybrid Solutions TIMTOWTDI •Material paths for quick lookups, links for relative traversal •Tree object for relative lookups, secondary index for material paths b a sho
Key Takeaways b a sho
Key Takeaways •Design for most common access pattern: the key is your index b a sho
Key Takeaways •Design for most common access pattern: the key is your index •Denormalize dependent data types b a sho
Key Takeaways •Design for most common access pattern: the key is your index •Denormalize dependent data types •Build richer (or simpler) data structures b a sho
Key Takeaways •Design for most common access pattern: the key is your index •Denormalize dependent data types •Build richer (or simpler) data structures •Use links to connect normalized or independent types b a sho
Useful Tips b a sho
Useful Tips •Use query or identity caches to reduce duplicate fetches b a sho
Useful Tips •Use query or identity caches to reduce duplicate fetches •Store data in JSON, XML, or Erlang terms for MapReduce b a sho
Useful Tips •Use query or identity caches to reduce duplicate fetches •Store data in JSON, XML, or Erlang terms for MapReduce •Use Riak Search where appropriate to reduce complexity b a sho
Review b a sho
Review •Analyze your relational model b a sho
Review •Analyze your relational model •Identify pain points, take stats b a sho
Review •Analyze your relational model •Identify pain points, take stats •Design some alternatives b a sho
Review •Analyze your relational model •Identify pain points, take stats •Design some alternatives •Test, Measure, Repeat! b a sho
Plug Interested in learning about support, consulting, or Enterprise features?   Email info@basho.com or go to http://www.basho.com/contact.html to talk with us. www.basho.com b a sho
Plug Interested in learning about support, consulting, or Enterprise features?   Email info@basho.com or go to http://www.basho.com/contact.html to talk with us. www.basho.com sean@basho.com @seancribbs b a sho

More Related Content

KEY
Schema Design for Riak
Sean Cribbs
 
PPTX
Search for Overview for SC Upstate SP users
Mike Brannon
 
PPTX
Re-Experience SharePoint - Ripping Apart the Interface on SharePoint 2010
EndUserSharePoint
 
PDF
URL Design with Lasso
macsolve
 
KEY
Color Me Flexible
Steven Merrill
 
PDF
What is new in Riak 2.0
Philipp Fehre
 
PDF
Riak: A friendly key/value store for the web.
codefluency
 
PDF
NoSQL Overview
adesso AG
 
Schema Design for Riak
Sean Cribbs
 
Search for Overview for SC Upstate SP users
Mike Brannon
 
Re-Experience SharePoint - Ripping Apart the Interface on SharePoint 2010
EndUserSharePoint
 
URL Design with Lasso
macsolve
 
Color Me Flexible
Steven Merrill
 
What is new in Riak 2.0
Philipp Fehre
 
Riak: A friendly key/value store for the web.
codefluency
 
NoSQL Overview
adesso AG
 

Similar to Schema Design for Riak (Take 2) (20)

PDF
Access2007 m1
jigeno
 
PPTX
Polyglot persistence with no sql
Michael Lehmann
 
PPTX
Database Management System
Muhd Dembo
 
PDF
Introduction to database
Suleman Memon
 
PPTX
lecture 6 DIGITAL TOOLBOX SEMESTER 1.PPTX
t2133457
 
PPTX
No sql introduction_v1.1.1
Fan Ang
 
PDF
Introduction to Database Management Systems: Structure, Applications, and Key...
Mahmud Hasan Tanvir
 
PDF
10 sql tips to speed up your database cats whocode.com
Kaing Menglieng
 
KEY
Advanced querying
strmpnk
 
PDF
My sql102
Dave Stokes
 
PDF
Exalead managing terrabytes
Jérémie BORDIER
 
PPTX
Relational database concept and technology
Ducat
 
PPTX
Different type of databases
Shwe Yee
 
PPTX
Couchbase at the academic bisilim, Turkey
sharonyb
 
PDF
Ten Ways to Destroy Your Database
PostgreSQL Experts, Inc.
 
KEY
NoSQL databases and managing big data
Steven Francia
 
PDF
Transition from relational to NoSQL Philly DAMA Day
Dipti Borkar
 
PDF
Navigating the Transition from relational to NoSQL - CloudCon Expo 2012
Dipti Borkar
 
PDF
Data in your SOA: From SQL to NoSQL and Beyond
WSO2
 
PPTX
Week3 adb
David Sherlock
 
Access2007 m1
jigeno
 
Polyglot persistence with no sql
Michael Lehmann
 
Database Management System
Muhd Dembo
 
Introduction to database
Suleman Memon
 
lecture 6 DIGITAL TOOLBOX SEMESTER 1.PPTX
t2133457
 
No sql introduction_v1.1.1
Fan Ang
 
Introduction to Database Management Systems: Structure, Applications, and Key...
Mahmud Hasan Tanvir
 
10 sql tips to speed up your database cats whocode.com
Kaing Menglieng
 
Advanced querying
strmpnk
 
My sql102
Dave Stokes
 
Exalead managing terrabytes
Jérémie BORDIER
 
Relational database concept and technology
Ducat
 
Different type of databases
Shwe Yee
 
Couchbase at the academic bisilim, Turkey
sharonyb
 
Ten Ways to Destroy Your Database
PostgreSQL Experts, Inc.
 
NoSQL databases and managing big data
Steven Francia
 
Transition from relational to NoSQL Philly DAMA Day
Dipti Borkar
 
Navigating the Transition from relational to NoSQL - CloudCon Expo 2012
Dipti Borkar
 
Data in your SOA: From SQL to NoSQL and Beyond
WSO2
 
Week3 adb
David Sherlock
 
Ad

More from Sean Cribbs (18)

KEY
Eventually Consistent Data Structures (from strangeloop12)
Sean Cribbs
 
KEY
Eventually-Consistent Data Structures
Sean Cribbs
 
KEY
A Case of Accidental Concurrency
Sean Cribbs
 
KEY
Embrace NoSQL and Eventual Consistency with Ripple
Sean Cribbs
 
KEY
Riak with node.js
Sean Cribbs
 
PDF
Riak (Øredev nosql day)
Sean Cribbs
 
PDF
Riak Tutorial (Øredev)
Sean Cribbs
 
PDF
The Radiant Ethic
Sean Cribbs
 
KEY
Introduction to Riak and Ripple (KC.rb)
Sean Cribbs
 
KEY
Riak with Rails
Sean Cribbs
 
KEY
Introduction to Riak - Red Dirt Ruby Conf Training
Sean Cribbs
 
PDF
Introducing Riak and Ripple
Sean Cribbs
 
ZIP
Round PEG, Round Hole - Parsing Functionally
Sean Cribbs
 
PDF
Story Driven Development With Cucumber
Sean Cribbs
 
KEY
Achieving Parsing Sanity In Erlang
Sean Cribbs
 
PDF
Of Rats And Dragons
Sean Cribbs
 
KEY
Erlang/OTP for Rubyists
Sean Cribbs
 
PDF
Content Management That Won't Rot Your Brain
Sean Cribbs
 
Eventually Consistent Data Structures (from strangeloop12)
Sean Cribbs
 
Eventually-Consistent Data Structures
Sean Cribbs
 
A Case of Accidental Concurrency
Sean Cribbs
 
Embrace NoSQL and Eventual Consistency with Ripple
Sean Cribbs
 
Riak with node.js
Sean Cribbs
 
Riak (Øredev nosql day)
Sean Cribbs
 
Riak Tutorial (Øredev)
Sean Cribbs
 
The Radiant Ethic
Sean Cribbs
 
Introduction to Riak and Ripple (KC.rb)
Sean Cribbs
 
Riak with Rails
Sean Cribbs
 
Introduction to Riak - Red Dirt Ruby Conf Training
Sean Cribbs
 
Introducing Riak and Ripple
Sean Cribbs
 
Round PEG, Round Hole - Parsing Functionally
Sean Cribbs
 
Story Driven Development With Cucumber
Sean Cribbs
 
Achieving Parsing Sanity In Erlang
Sean Cribbs
 
Of Rats And Dragons
Sean Cribbs
 
Erlang/OTP for Rubyists
Sean Cribbs
 
Content Management That Won't Rot Your Brain
Sean Cribbs
 
Ad

Recently uploaded (20)

PPTX
What-is-the-World-Wide-Web -- Introduction
tonifi9488
 
PDF
Economic Impact of Data Centres to the Malaysian Economy
flintglobalapac
 
PPTX
Dev Dives: Automate, test, and deploy in one place—with Unified Developer Exp...
AndreeaTom
 
PPTX
OA presentation.pptx OA presentation.pptx
pateldhruv002338
 
PPTX
Agile Chennai 18-19 July 2025 Ideathon | AI Powered Microfinance Literacy Gui...
AgileNetwork
 
PDF
Security features in Dell, HP, and Lenovo PC systems: A research-based compar...
Principled Technologies
 
PPTX
cloud computing vai.pptx for the project
vaibhavdobariyal79
 
PPTX
Simple and concise overview about Quantum computing..pptx
mughal641
 
PDF
How Open Source Changed My Career by abdelrahman ismail
a0m0rajab1
 
PDF
Accelerating Oracle Database 23ai Troubleshooting with Oracle AHF Fleet Insig...
Sandesh Rao
 
PPTX
AI in Daily Life: How Artificial Intelligence Helps Us Every Day
vanshrpatil7
 
PDF
OFFOFFBOX™ – A New Era for African Film | Startup Presentation
ambaicciwalkerbrian
 
PDF
Unlocking the Future- AI Agents Meet Oracle Database 23ai - AIOUG Yatra 2025.pdf
Sandesh Rao
 
PDF
Research-Fundamentals-and-Topic-Development.pdf
ayesha butalia
 
PPTX
Agile Chennai 18-19 July 2025 | Emerging patterns in Agentic AI by Bharani Su...
AgileNetwork
 
PDF
How ETL Control Logic Keeps Your Pipelines Safe and Reliable.pdf
Stryv Solutions Pvt. Ltd.
 
PPTX
Introduction to Flutter by Ayush Desai.pptx
ayushdesai204
 
PDF
Responsible AI and AI Ethics - By Sylvester Ebhonu
Sylvester Ebhonu
 
PDF
Brief History of Internet - Early Days of Internet
sutharharshit158
 
PDF
GDG Cloud Munich - Intro - Luiz Carneiro - #BuildWithAI - July - Abdel.pdf
Luiz Carneiro
 
What-is-the-World-Wide-Web -- Introduction
tonifi9488
 
Economic Impact of Data Centres to the Malaysian Economy
flintglobalapac
 
Dev Dives: Automate, test, and deploy in one place—with Unified Developer Exp...
AndreeaTom
 
OA presentation.pptx OA presentation.pptx
pateldhruv002338
 
Agile Chennai 18-19 July 2025 Ideathon | AI Powered Microfinance Literacy Gui...
AgileNetwork
 
Security features in Dell, HP, and Lenovo PC systems: A research-based compar...
Principled Technologies
 
cloud computing vai.pptx for the project
vaibhavdobariyal79
 
Simple and concise overview about Quantum computing..pptx
mughal641
 
How Open Source Changed My Career by abdelrahman ismail
a0m0rajab1
 
Accelerating Oracle Database 23ai Troubleshooting with Oracle AHF Fleet Insig...
Sandesh Rao
 
AI in Daily Life: How Artificial Intelligence Helps Us Every Day
vanshrpatil7
 
OFFOFFBOX™ – A New Era for African Film | Startup Presentation
ambaicciwalkerbrian
 
Unlocking the Future- AI Agents Meet Oracle Database 23ai - AIOUG Yatra 2025.pdf
Sandesh Rao
 
Research-Fundamentals-and-Topic-Development.pdf
ayesha butalia
 
Agile Chennai 18-19 July 2025 | Emerging patterns in Agentic AI by Bharani Su...
AgileNetwork
 
How ETL Control Logic Keeps Your Pipelines Safe and Reliable.pdf
Stryv Solutions Pvt. Ltd.
 
Introduction to Flutter by Ayush Desai.pptx
ayushdesai204
 
Responsible AI and AI Ethics - By Sylvester Ebhonu
Sylvester Ebhonu
 
Brief History of Internet - Early Days of Internet
sutharharshit158
 
GDG Cloud Munich - Intro - Luiz Carneiro - #BuildWithAI - July - Abdel.pdf
Luiz Carneiro
 

Schema Design for Riak (Take 2)

  • 1. Schema Design for Riak Sean Cribbs Developer Advocate b a sho
  • 2. Thinking Non-Relationally b a sho
  • 3. “There is no spoon schema.” b a sho
  • 4. There is an application. b a sho
  • 6. What you lose •Tables b a sho
  • 7. What you lose •Tables •Foreign keys and constraints b a sho
  • 8. What you lose •Tables •Foreign keys and constraints •ACID b a sho
  • 9. What you lose •Tables •Foreign keys and constraints •ACID •Sophisticated query planners b a sho
  • 10. What you lose •Tables •Foreign keys and constraints •ACID •Sophisticated query planners •Declarative query language (SQL) b a sho
  • 12. What you gain •More flexible, fluid designs b a sho
  • 13. What you gain •More flexible, fluid designs •More natural data representations b a sho
  • 14. What you gain •More flexible, fluid designs •More natural data representations •Scaling without pain b a sho
  • 15. What you gain •More flexible, fluid designs •More natural data representations •Scaling without pain •Reduced operational complexity b a sho
  • 16. Walking without Relational crutches b a sho
  • 17. Walking without Relational crutches •Sparse data (optional/multi-value fields) b a sho
  • 18. Walking without Relational crutches •Sparse data (optional/multi-value fields) •Richer data structures b a sho
  • 19. Walking without Relational crutches •Sparse data (optional/multi-value fields) •Richer data structures •Meaningful identifiers b a sho
  • 20. Walking without Relational crutches •Sparse data (optional/multi-value fields) •Richer data structures •Meaningful identifiers •Innovative access patterns b a sho
  • 22. Make your Top 10 list b a sho
  • 23. Make your Top 10 list •Frequently requested pages/screens b a sho
  • 24. Make your Top 10 list •Frequently requested pages/screens •Slow queries b a sho
  • 25. Make your Top 10 list •Frequently requested pages/screens •Slow queries •Secondary indexes b a sho
  • 26. Make your Top 10 list •Frequently requested pages/screens •Slow queries •Secondary indexes •Complicated joins or aggregations b a sho
  • 28. Analyze •Interdependencies, coupling b a sho
  • 29. Analyze •Interdependencies, coupling •Cardinalities of relationships b a sho
  • 30. Analyze •Interdependencies, coupling •Cardinalities of relationships •Access pattern b a sho
  • 31. Analyze •Interdependencies, coupling •Cardinalities of relationships •Access pattern •Shoehorned structures b a sho
  • 34. The “Styled Blog” Template b a sho
  • 35. Date-organized pages The “Styled Blog” Template b a sho
  • 36. Static sidebar content Date-organized pages The “Styled Blog” Template b a sho
  • 38. hierarchical site structure Administration UI b a sho
  • 39. add new pages by parent hierarchical site structure Administration UI b a sho
  • 40. add new pages by parent special “virtual” pages hierarchical site structure Administration UI b a sho
  • 43. slug, breadcrumb content blocks b a sho
  • 44. slug, breadcrumb content blocks layout b a sho
  • 47. Content-Type template tags b a sho
  • 48. Relational Schema pages b a sho
  • 49. Relational Schema parent pages b a sho
  • 50. Relational Schema parent pages page_parts b a sho
  • 51. Relational Schema parent pages page_parts layouts b a sho
  • 52. Relational Schema parent pages page_parts layouts snippets b a sho
  • 53. Relational Schema parent pages page_parts layouts users snippets b a sho
  • 54. Relational Schema parent pages page_parts created/modified by layouts users snippets b a sho
  • 57. Layouts and Snippets • Layouts and Snippets are accessed by name b a sho
  • 58. Layouts and Snippets • Layouts and Snippets are accessed by name • SQL: WHERE name=? b a sho
  • 59. Layouts and Snippets • Layouts and Snippets are accessed by name layouts/Main • SQL: WHERE name=? • Simple access by key snippets/top-nav b a sho
  • 60. Layouts and Snippets • Layouts and Snippets are accessed by name layouts/Main • SQL: WHERE name=? • Simple access by key snippets/top-nav • Simple value structure (content + metadata) b a sho
  • 61. Users Normally accessed by login or email: WHERE login=? OR email=? b a sho
  • 62. Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key b a sho
  • 63. Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • Easy lookup on one, manual index other b a sho
  • 64. Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • Easy lookup on one, manual index other • #2: Arbitrary key b a sho
  • 65. Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • Easy lookup on one, manual index other • #2: Arbitrary key • Independent of email/login changes b a sho
  • 66. Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • Easy lookup on one, manual index other • #2: Arbitrary key • Independent of email/login changes • Manually index both b a sho
  • 67. Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • #3: Users-list object • Easy lookup on one, manual index other • #2: Arbitrary key • Independent of email/login changes • Manually index both b a sho
  • 68. Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • #3: Users-list object • Easy lookup on one, • Fits “small teams” manual index other philosophy • #2: Arbitrary key • Independent of email/login changes • Manually index both b a sho
  • 69. Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • #3: Users-list object • Easy lookup on one, • Fits “small teams” manual index other philosophy • #2: Arbitrary key • Quick lookup (one fetch) • Independent of email/login changes • Manually index both b a sho
  • 70. Users Normally accessed by login or email: WHERE login=? OR email=? • #1: Login or Email key • #3: Users-list object • Easy lookup on one, • Fits “small teams” manual index other philosophy • #2: Arbitrary key • Quick lookup (one fetch) • Independent of email/login changes • Bad for large list, still need unique IDs • Manually index both b a sho
  • 73. Page Rendering • Load and render layout in page context b a sho
  • 74. Page Rendering • Load and render layout in page context • Render parts, snippets via template tags b a sho
  • 75. Page Rendering • Load and render layout in page context • Render parts, snippets via template tags • Iterate over sections of page hierarchy b a sho
  • 77. Rendering Page Parts •Parts are dependent on the page b a sho
  • 78. Rendering Page Parts •Parts are dependent on the page •Compose / Denormalize (part of page) b a sho
  • 79. Rendering Page Parts •Parts are dependent on the page •Compose / Denormalize (part of page) •Reduces lookups, conceptual unity, no index needed b a sho
  • 80. Rendering Page Parts •Parts are dependent on the page •Compose / Denormalize (part of page) •Reduces lookups, conceptual unity, no index needed • Larger objects, no lazy fetching b a sho
  • 81. Denormalized Page { title:”Home Page”, parts:[ {name:”body”, content:”...”}, {name:”sidebar”, content:”...”}, ], // ... } b a sho
  • 82. Denormalized Page { title:”Home Page”, parts:[ {name:”body”, content:”...”}, {name:”sidebar”, content:”...”}, ], // ... } dependent objects inline b a sho
  • 83. Site Hierarchy ...or, how to find a page (currently) b a sho
  • 84. Site Hierarchy ...or, how to find a page (currently) • Start at the root: WHERE parent_id IS NULL b a sho
  • 85. Site Hierarchy ...or, how to find a page (currently) • Start at the root: WHERE parent_id IS NULL • Check if the current page matches URL b a sho
  • 86. Site Hierarchy ...or, how to find a page (currently) • Start at the root: WHERE parent_id IS NULL • Check if the current page matches URL • Check children for matching path segment, recurse b a sho
  • 87. Site Hierarchy ...or, how to find a page (currently) • Start at the root: WHERE parent_id IS NULL • Check if the current page matches URL • Check children for matching path segment, recurse • Return “not found” page or 404 b a sho
  • 88. Site Hierarchy ...or, how to find a page (analysis) b a sho
  • 89. Site Hierarchy ...or, how to find a page (analysis) • O(log N) queries to find requested page b a sho
  • 90. Site Hierarchy ...or, how to find a page (analysis) • O(log N) queries to find requested page • Index on parent_id speeds retrieval b a sho
  • 91. Site Hierarchy ...or, how to find a page (analysis) • O(log N) queries to find requested page • Index on parent_id speeds retrieval • Traversing/iterating for content generation b a sho
  • 92. Site Hierarchy ...or, how to find a page (analysis) • O(log N) queries to find requested page • Index on parent_id speeds retrieval • Traversing/iterating for content generation • Generating URL paths b a sho
  • 93. Site Hierarchy ...or, how to find a page (in Riak) b a sho
  • 94. Site Hierarchy ...or, how to find a page (in Riak) • Blog post: http://ow.ly/3jDAO b a sho
  • 95. Site Hierarchy ...or, how to find a page (in Riak) • Blog post: http://ow.ly/3jDAO • #1 Parent/child links b a sho
  • 96. Site Hierarchy ...or, how to find a page (in Riak) • Blog post: http://ow.ly/3jDAO • #1 Parent/child links • #2 Material path key b a sho
  • 97. Site Hierarchy ...or, how to find a page (in Riak) • Blog post: http://ow.ly/3jDAO • #1 Parent/child links • #2 Material path key • #3 Tree object b a sho
  • 98. #1: Parent/Child Links A Natural Tree A B C D E F b a sho
  • 99. #1: Parent/Child Links A Natural Tree • Easy to understand & A implement B C D E F b a sho
  • 100. #1: Parent/Child Links A Natural Tree </riak/pages/C>; riaktag=”child” • Easy to understand & A implement </riak/pages/A>; riaktag=”parent” • Doubly-linked for easy traversal in either B C direction D E F b a sho
  • 101. #1: Parent/Child Links A Natural Tree </riak/pages/C>; riaktag=”child” • Easy to understand & A implement </riak/pages/A>; riaktag=”parent” • Doubly-linked for easy traversal in either B C direction • Easy to move entire subtrees D E F b a sho
  • 102. #1: Parent/Child Links A Natural Tree A B C D E F b a sho
  • 103. #1: Parent/Child Links A Natural Tree A • Child order is arbitrary B C D E F b a sho
  • 104. #1: Parent/Child Links A Natural Tree A • Child order is arbitrary • Still O(log N) traversal B C D E F b a sho
  • 105. #1: Parent/Child Links A Natural Tree A • Child order is arbitrary • Still O(log N) traversal B C • Two writes to add or update D E F b a sho
  • 106. #2: Material Path Key “Best Guess” discovery _root_ (A) B C B/D C/E “pages” bucket b a sho
  • 107. #2: Material Path Key “Best Guess” discovery _root_ (A) • Use path-to-page as key B C B/D C/E “pages” bucket b a sho
  • 108. #2: Material Path Key “Best Guess” discovery _root_ (A) • Use path-to-page as key B • Best case: one lookup C to find requested page B/D C/E “pages” bucket b a sho
  • 109. #2: Material Path Key “Best Guess” discovery _root_ (A) • Use path-to-page as key B • Best case: one lookup C to find requested page B/D • Ancestor pages listed inside object C/E “pages” bucket b a sho
  • 110. #2: Material Path Key “Best Guess” discovery { _root_ (A) • Use path-to-page as title:”D”, key ancestors:[ B “B”, • Best case: one lookup “_root_” C to find requested page ] B/D • Ancestor pages listed inside object C/E “pages” bucket b a sho
  • 111. #2: Material Path Key “Best Guess” discovery _root_ (A) B C B/D C/E “pages” bucket b a sho
  • 112. #2: Material Path Key “Best Guess” discovery • Large update cost for _root_ (A) internal nodes B C B/D C/E “pages” bucket b a sho
  • 113. #2: Material Path Key “Best Guess” discovery • Large update cost for _root_ (A) internal nodes • Dynamic URLs hard B C B/D C/E “pages” bucket b a sho
  • 114. #2: Material Path Key “Best Guess” discovery • Large update cost for _root_ (A) internal nodes • Dynamic URLs hard B • Key-filters (0.14) C needed for efficient B/D child lists C/E “pages” bucket b a sho
  • 115. #2: Material Path Key “Best Guess” discovery • Large update cost for _root_ (A) internal nodes • Dynamic URLs hard B • Key-filters (0.14) C needed for efficient B/D child lists • Need fallback for C/E “misses” “pages” bucket b a sho
  • 116. #3: Tree Object “Branches and Leaves” { key:”A”, children:[ {key:”B”, children:[“D”]}, {key:”C”, children:[“E”,”F”]} ] } b a sho
  • 117. #3: Tree Object “Branches and Leaves” • One request to get site structure { key:”A”, children:[ {key:”B”, children:[“D”]}, {key:”C”, children:[“E”,”F”]} ] } b a sho
  • 118. #3: Tree Object “Branches and Leaves” • One request to get site structure { key:”A”, • Separates content from children:[ {key:”B”, organization children:[“D”]}, {key:”C”, children:[“E”,”F”]} ] } b a sho
  • 119. #3: Tree Object “Branches and Leaves” • One request to get site structure { key:”A”, • Separates content from children:[ {key:”B”, organization children:[“D”]}, {key:”C”, • Intrinsic ordering ] children:[“E”,”F”]} } b a sho
  • 120. #3: Tree Object “Branches and Leaves” • One request to get site structure { key:”A”, • Separates content from children:[ {key:”B”, organization children:[“D”]}, {key:”C”, • Intrinsic ordering ] children:[“E”,”F”]} } • Massive structure changes are quick b a sho
  • 121. #3: Tree Object “Branches and Leaves” { key:”A”, children:[ {key:”B”, children:[“D”]}, {key:”C”, children:[“E”,”F”]} ] } b a sho
  • 122. #3: Tree Object “Branches and Leaves” • Large site = large tree object (expensive to transfer) { key:”A”, children:[ {key:”B”, children:[“D”]}, {key:”C”, children:[“E”,”F”]} ] } b a sho
  • 123. #3: Tree Object “Branches and Leaves” • Large site = large tree object (expensive to transfer) { key:”A”, • Some metadata children:[ {key:”B”, (slug?)will need to be children:[“D”]}, {key:”C”, in-tree for efficient children:[“E”,”F”]} traversal ] } b a sho
  • 124. #3: Tree Object “Branches and Leaves” • Large site = large tree object (expensive to transfer) { key:”A”, • Some metadata children:[ {key:”B”, (slug?)will need to be children:[“D”]}, {key:”C”, in-tree for efficient children:[“E”,”F”]} traversal ] } • Multiple writers problematic b a sho
  • 125. Hybrid Solutions TIMTOWTDI b a sho
  • 126. Hybrid Solutions TIMTOWTDI •Material paths for quick lookups, links for relative traversal b a sho
  • 127. Hybrid Solutions TIMTOWTDI •Material paths for quick lookups, links for relative traversal •Tree object for relative lookups, secondary index for material paths b a sho
  • 129. Key Takeaways •Design for most common access pattern: the key is your index b a sho
  • 130. Key Takeaways •Design for most common access pattern: the key is your index •Denormalize dependent data types b a sho
  • 131. Key Takeaways •Design for most common access pattern: the key is your index •Denormalize dependent data types •Build richer (or simpler) data structures b a sho
  • 132. Key Takeaways •Design for most common access pattern: the key is your index •Denormalize dependent data types •Build richer (or simpler) data structures •Use links to connect normalized or independent types b a sho
  • 134. Useful Tips •Use query or identity caches to reduce duplicate fetches b a sho
  • 135. Useful Tips •Use query or identity caches to reduce duplicate fetches •Store data in JSON, XML, or Erlang terms for MapReduce b a sho
  • 136. Useful Tips •Use query or identity caches to reduce duplicate fetches •Store data in JSON, XML, or Erlang terms for MapReduce •Use Riak Search where appropriate to reduce complexity b a sho
  • 138. Review •Analyze your relational model b a sho
  • 139. Review •Analyze your relational model •Identify pain points, take stats b a sho
  • 140. Review •Analyze your relational model •Identify pain points, take stats •Design some alternatives b a sho
  • 141. Review •Analyze your relational model •Identify pain points, take stats •Design some alternatives •Test, Measure, Repeat! b a sho
  • 142. Plug Interested in learning about support, consulting, or Enterprise features?   Email [email protected] or go to http://www.basho.com/contact.html to talk with us. www.basho.com b a sho
  • 143. Plug Interested in learning about support, consulting, or Enterprise features?   Email [email protected] or go to http://www.basho.com/contact.html to talk with us. www.basho.com [email protected] @seancribbs b a sho

Editor's Notes