SlideShare a Scribd company logo
Table Partitioning:
Secret Weapon for Big Data Problems

October 15-18, 2013
Charlotte, NC

John Sterrett, Sr. Database Admin Advisor
DELL
About John

2
How to contact me?
http://johnsterrett.com/go/partition

Blog: http://JohnSterrett.com/
Twitter: @JohnSterrett
LinkedIn: http://linkedin.com/in/johnsterrett

3
Agenda
Big Data starting to slow you down? Data growth putting your
maintenance tasks in danger of not meeting your SLAs? Wish you
could archive old data with minimal impact to your tables during the
archive process or that you could eliminate most of the data in your
tables when you query them? If so, it’s time you consider implementing
table partitioning to help with general performance and reduce your
window for completing maintenance tasks.

•  Learn if Partitioning can help you manage big data.
•  Understand how partitioning works.
•  Learn how to maintain table partitioning with an automated
sliding window.

4
Assumptions…
•  You are not familiar with Table Partitioning and want
to see how it works.
•  You have Enterprise Edition
•  Highly Transactional Tables with over 50 GB of data
•  General Maintenance Tasks are in danger of not meeting
your SLA
•  Purge and/or Archive process is slowing you down.

5
The Big Question…
HOW CAN TABLE PARTITIONG MAKE MY LIFE
EASIER?

6
How partitioning helps me?
•  Reduce Maintenance Tasks

7
How partitioning helps me?
•  Reduce Maintenance Tasks
•  Improves Purging and/or Archiving

8
How partitioning helps me?
•  Reduce Maintenance Tasks
•  Improves Purging and/or Archiving
•  Improves Performance

9
Big Question…
HOW DOES TABLE PARTITIONING WORK?

10
High Level…

11
High Level…
Partition	
  Table

Partition	
  Scheme

Partition	
  Function

12
Selecting The Partition Column
Only get one column, use it wisely!
Column should be a highly used filter.
•  Review index usage statistics
•  Review Missing index statistics
•  Review Queries and Talk to developers ;-)

Column must be part of clustered index

13
Partition Function

Defines the data type used to distribute data into partitions.
Assigns boundary values to split data between partitions.
Assigns the RANGE for boundary values
Partitions = Boundary values + 1
NULL values go to left most partition

14
Partition Function
Partition functions are not static. They can change over
time with SPLIT and MERGE statements.
Range LEFT is used by default.

15
Partition Function - Range

{min….100}, {101…200}, {201...300}, {301…max}

{min…99}, {100…199}, {200…299}, {300…max}

16
Partition Scheme

Assigns a partition function to a partition scheme
Assigns filegroups to partitions

17
Partitioning
DEMO

October 15-18, 2013 | Charlotte, NC
Big Question…
HOW DOES TABLE PARTITIONING IMPROVE MY
MAINTENANCE TASKS?

19
Improving Maintenance Tasks
Backup and restore filegroups based on business priorities.
Index Maintenance by partition
New Features in SQL 2014
•  Rebuild index online by partition
•  Incremental Statistics by partition (SQL 2014 CTP 2)

20
Incremental Statistics
Just added in SQL Server 2014 CTP2
From Books Online:
“When ON, the statistics are recreated as per partition
statistics.”
Cannot be used for the following:
•  Statistics created with indexes that are not partitionaligned with the base table
•  Filtered Indexes

21
HOW DOES TABLE PARTITIONING IMPROVE
PERFORMANCE?

22
Partition Elimination

23
Skip-Scan: Seek keys

24
Database Compression by Partition

25
Partitioning
DEMO

October 15-18, 2013 | Charlotte, NC
HOW DOES PARTITIONING IMPROVE ARCHIVING AND
PURGING?

27
Real World Example:

28
Quickest way to move billions of rows

Meta data swap is quickest way to move billions of rows
assuming you can get a schema lock.

29
Sliding Window Goals
SPLIT and MERGE with empty partitions
Use SWITCH to do meta-data swaps
Minimize all physical data movement

30
Range Right - MERGE

31
Range Left - MERGE

32
Range Right - SPLIT

33
Range Left - SPLIT

34
Visual Sliding Window Example

35
Visual Sliding Window Example

Partiton1
(EMPTY)

Partiton2
(DATA)

Partiton3
(DATA)

Partiton4
(DATA)

Partiton5
(DATA)

FG1

FG2

FG3

FG4

FG5
Visual Sliding Window Example

Partiton1
(EMPTY)

Partiton2
(DATA)

Staging	
  
Table	
  	
  
(EMPTY)

FG2

Partiton4
(DATA)

Partiton5
(DATA)

FG3

FG1

Partiton3
(DATA)

FG4

FG5
Visual Sliding Window Example

Partiton1
(EMPTY)

FG1

Partiton2
(EMPTY)

Staging	
  
Table	
  	
  (DATA)

FG2

Partiton4
(DATA)

Partiton5
(DATA)

FG3

SWAP

Partiton3
(DATA)

FG4

FG5
Visual Sliding Window Example

Partiton1
(EMPTY)

MERGE

Partiton2
(EMPTY)

Staging	
  
Table	
  	
  (DATA)

FG2

Partiton4
(DATA)

Partiton5
(DATA)

FG3

FG1

Partiton3
(DATA)

FG4

FG5
Visual Sliding Window Example

Partiton1
(EMPTY)

Partiton2
(DATA)

Partiton3
(DATA)

Partiton4
(DATA)

FG1

FG3

FG4

FG5

Staging	
  
Table	
  	
  (DATA)
FG2
Visual Sliding Window Example

Partiton1
(EMPTY)

Partiton2
(DATA)

Partiton3
(DATA)

Partiton4
(DATA)

FG1

FG3

FG4

FG5

NEXT	
  USED

Staging	
  
Table	
  	
  
(EMPTY)
FG2
Visual Sliding Window Example

Partiton1
(EMPTY)

Partiton2
(DATA)

Partiton3
(DATA)

Partiton4
(DATA)

Partiton5
(DATA)

FG1

FG3

FG4

FG5

FG2
Sliding Window Steps...
Create partition swap table (if it doesn’t exist)
1.  Insert partition swap meta data
2.  Create staging table
3.  Meta-data swap (partition 2 with staging table)
4.  Merge Partitions #1 and #2
5.  Mark next used partition
6.  Split to create new partition
7.  Update processed partition swap meta data.

43
Partitioning
DEMO

October 15-18, 2013 | Charlotte, NC
Questions…
http://johnsterrett.com/go/partition

Blog: http://JohnSterrett.com/
Twitter: @JohnSterrett
LinkedIn: http://linkedin.com/in/johnsterrett

45
Thank you
for attending this session and the
2013 PASS Summit in Charlotte, NC

46

October 15-18, 2013 | Charlotte, NC

More Related Content

Similar to Table Partitioning: Secret Weapon for Big Data Problems (20)

PPT
The thinking persons guide to data warehouse design
Calpont
 
PDF
PHP Detroit -- MySQL 8 A New Beginning (updated presentation)
Dave Stokes
 
PDF
PostgreSQL Table Partitioning / Sharding
Amir Reza Hashemi
 
PDF
2017 AWS DB Day | Amazon Redshift 소개 및 실습
Amazon Web Services Korea
 
PPTX
MySQL 8.0 Featured for Developers
Dave Stokes
 
PDF
Alluxio Product School Webinar - Boosting Trino Performance.
Alluxio, Inc.
 
PPT
SQL Server 2008 Performance Enhancements
infusiondev
 
PPTX
MySQL 8 - UKOUG Techfest Brighton December 2nd, 2019
Dave Stokes
 
PDF
Collecting and Making Sense of Diverse Data at WayUp
Harlan Harris
 
PPTX
Vldb Yaron Moshe
sqlserver.co.il
 
PPT
Scalable Machine Learning: The Role of Stratified Data Sharding
inside-BigData.com
 
PPT
ORACLE 12C-New-Features
Navneet Upneja
 
PDF
overview of_data_processing
FEG
 
PPTX
very large database
Kazem Taghandiky
 
PPTX
Hive: Loading Data
Benjamin Leonhardi
 
PPT
Stack It And Unpack It
Jeff Moss
 
PDF
Oracle 12 c new-features
Navneet Upneja
 
PDF
Cs437 lecture 1-6
Aneeb_Khawar
 
PPTX
Database Performance
Boris Hristov
 
PDF
Common Strategies for Improving Performance on Your Delta Lakehouse
Databricks
 
The thinking persons guide to data warehouse design
Calpont
 
PHP Detroit -- MySQL 8 A New Beginning (updated presentation)
Dave Stokes
 
PostgreSQL Table Partitioning / Sharding
Amir Reza Hashemi
 
2017 AWS DB Day | Amazon Redshift 소개 및 실습
Amazon Web Services Korea
 
MySQL 8.0 Featured for Developers
Dave Stokes
 
Alluxio Product School Webinar - Boosting Trino Performance.
Alluxio, Inc.
 
SQL Server 2008 Performance Enhancements
infusiondev
 
MySQL 8 - UKOUG Techfest Brighton December 2nd, 2019
Dave Stokes
 
Collecting and Making Sense of Diverse Data at WayUp
Harlan Harris
 
Vldb Yaron Moshe
sqlserver.co.il
 
Scalable Machine Learning: The Role of Stratified Data Sharding
inside-BigData.com
 
ORACLE 12C-New-Features
Navneet Upneja
 
overview of_data_processing
FEG
 
very large database
Kazem Taghandiky
 
Hive: Loading Data
Benjamin Leonhardi
 
Stack It And Unpack It
Jeff Moss
 
Oracle 12 c new-features
Navneet Upneja
 
Cs437 lecture 1-6
Aneeb_Khawar
 
Database Performance
Boris Hristov
 
Common Strategies for Improving Performance on Your Delta Lakehouse
Databricks
 

More from John Sterrett (12)

PDF
Database Management Myths for Developers
John Sterrett
 
PDF
DBA Basics: Getting Started with Performance Tuning.pdf
John Sterrett
 
PDF
Getting Started with SQL Server Performance Tuning.pdf
John Sterrett
 
PDF
Workload Replay in the Cloud: Secret Weapon for Cloud Migrations
John Sterrett
 
PDF
PowerPivot for DBAs
John Sterrett
 
PPTX
Introduction to High Availability with SQL Server
John Sterrett
 
PPTX
Introduction to PowerShell for DBA's
John Sterrett
 
PPTX
SQL Server Performance Root Cause Analysis in 10 Minutes
John Sterrett
 
PPTX
Can You Host a SQL Saturday?
John Sterrett
 
PPTX
Evaluating Daily Checklist Against 1000 Servers using Policy Based Management
John Sterrett
 
PPTX
SQL Server 2008 For Developers
John Sterrett
 
PPTX
Evaluate Daily Checklist with PBM and CMS
John Sterrett
 
Database Management Myths for Developers
John Sterrett
 
DBA Basics: Getting Started with Performance Tuning.pdf
John Sterrett
 
Getting Started with SQL Server Performance Tuning.pdf
John Sterrett
 
Workload Replay in the Cloud: Secret Weapon for Cloud Migrations
John Sterrett
 
PowerPivot for DBAs
John Sterrett
 
Introduction to High Availability with SQL Server
John Sterrett
 
Introduction to PowerShell for DBA's
John Sterrett
 
SQL Server Performance Root Cause Analysis in 10 Minutes
John Sterrett
 
Can You Host a SQL Saturday?
John Sterrett
 
Evaluating Daily Checklist Against 1000 Servers using Policy Based Management
John Sterrett
 
SQL Server 2008 For Developers
John Sterrett
 
Evaluate Daily Checklist with PBM and CMS
John Sterrett
 
Ad

Recently uploaded (20)

PDF
BÀI TẬP TEST BỔ TRỢ THEO TỪNG CHỦ ĐỀ CỦA TỪNG UNIT KÈM BÀI TẬP NGHE - TIẾNG A...
Nguyen Thanh Tu Collection
 
PDF
John Keats introduction and list of his important works
vatsalacpr
 
PPTX
20250924 Navigating the Future: How to tell the difference between an emergen...
McGuinness Institute
 
PDF
Module 2: Public Health History [Tutorial Slides]
JonathanHallett4
 
PPTX
LDP-2 UNIT 4 Presentation for practical.pptx
abhaypanchal2525
 
PPTX
How to Close Subscription in Odoo 18 - Odoo Slides
Celine George
 
DOCX
Unit 5: Speech-language and swallowing disorders
JELLA VISHNU DURGA PRASAD
 
PPTX
Command Palatte in Odoo 18.1 Spreadsheet - Odoo Slides
Celine George
 
PPTX
K-Circle-Weekly-Quiz12121212-May2025.pptx
Pankaj Rodey
 
PPTX
Applied-Statistics-1.pptx hardiba zalaaa
hardizala899
 
PDF
Virat Kohli- the Pride of Indian cricket
kushpar147
 
PPTX
PROTIEN ENERGY MALNUTRITION: NURSING MANAGEMENT.pptx
PRADEEP ABOTHU
 
PPTX
Gupta Art & Architecture Temple and Sculptures.pptx
Virag Sontakke
 
PPT
DRUGS USED IN THERAPY OF SHOCK, Shock Therapy, Treatment or management of shock
Rajshri Ghogare
 
PPTX
Virus sequence retrieval from NCBI database
yamunaK13
 
PPTX
Introduction to pediatric nursing in 5th Sem..pptx
AneetaSharma15
 
PPTX
HEALTH CARE DELIVERY SYSTEM - UNIT 2 - GNM 3RD YEAR.pptx
Priyanshu Anand
 
PPTX
Basics and rules of probability with real-life uses
ravatkaran694
 
PPTX
Continental Accounting in Odoo 18 - Odoo Slides
Celine George
 
PPTX
Unlock the Power of Cursor AI: MuleSoft Integrations
Veera Pallapu
 
BÀI TẬP TEST BỔ TRỢ THEO TỪNG CHỦ ĐỀ CỦA TỪNG UNIT KÈM BÀI TẬP NGHE - TIẾNG A...
Nguyen Thanh Tu Collection
 
John Keats introduction and list of his important works
vatsalacpr
 
20250924 Navigating the Future: How to tell the difference between an emergen...
McGuinness Institute
 
Module 2: Public Health History [Tutorial Slides]
JonathanHallett4
 
LDP-2 UNIT 4 Presentation for practical.pptx
abhaypanchal2525
 
How to Close Subscription in Odoo 18 - Odoo Slides
Celine George
 
Unit 5: Speech-language and swallowing disorders
JELLA VISHNU DURGA PRASAD
 
Command Palatte in Odoo 18.1 Spreadsheet - Odoo Slides
Celine George
 
K-Circle-Weekly-Quiz12121212-May2025.pptx
Pankaj Rodey
 
Applied-Statistics-1.pptx hardiba zalaaa
hardizala899
 
Virat Kohli- the Pride of Indian cricket
kushpar147
 
PROTIEN ENERGY MALNUTRITION: NURSING MANAGEMENT.pptx
PRADEEP ABOTHU
 
Gupta Art & Architecture Temple and Sculptures.pptx
Virag Sontakke
 
DRUGS USED IN THERAPY OF SHOCK, Shock Therapy, Treatment or management of shock
Rajshri Ghogare
 
Virus sequence retrieval from NCBI database
yamunaK13
 
Introduction to pediatric nursing in 5th Sem..pptx
AneetaSharma15
 
HEALTH CARE DELIVERY SYSTEM - UNIT 2 - GNM 3RD YEAR.pptx
Priyanshu Anand
 
Basics and rules of probability with real-life uses
ravatkaran694
 
Continental Accounting in Odoo 18 - Odoo Slides
Celine George
 
Unlock the Power of Cursor AI: MuleSoft Integrations
Veera Pallapu
 
Ad

Table Partitioning: Secret Weapon for Big Data Problems