SlideShare a Scribd company logo
Secondary Index Search in MySQL
Mijin An
meeeeejin@gmail.com
8/23/16 1
Contents
8/23/16 2
• Secondary/Primary Key Record Format
• Secondary Index Search
• Proposal for Improvement
• Improved Secondary Index Search
Secondary Key Record Format
8/23/16 3
= Primary Key Value
Primary Key Record Format
8/23/16 4
Secondary Index Search
8/23/16 5
① Start a mini-transaction
② Searches an index tree, then positions a tree cursor(pcur)
on a record corresponding to the index
(btr_pcur_open_with_no_init())
③ Get the record that pcur indicates(rec)
④ Go to rec_loop:
row_search_for_mysql()
rec_loop:
8/23/16 6
a. Get the record that pcur indicates(rec)
b. If record is infimum or supremum, go to next_rec:
Look for matching records in a loop
next_rec:
8/23/16 7
a. Stores current position of pcur
b. Commit mini-transaction
c. If record is infimum è move pcur to the next record
Else if record is supremum è move pcur to the first
record of next page
Else è move pcur according to the search direction
d. If we may need to process the record the cursor is now
on, go to rec_loop:
Move cursor to the next record, then search again
rec_loop:
8/23/16 8
a. Get the record that pcur indicates(rec)
b. If record is infimum or supremum, go to next_rec:
c. Caculates the offsets to each field in the record (offsets)
d. If select_lock_type == LOCK_S or LOCK_X,
- try to place a lock on the index record
e. Go to locks_ok:
If index != clust_index && need_to_access_clustered,
- go to requires_clust_rec:
Look for matching records in a loop
requires_clust_rec:
8/23/16 9
a. Retrieves the clustered record(clust_rec) corresponding
to a record in a secondary index (row_sel_get_clust_rec_for_mysql())
Get clustered record using secondary index
row_sel_get_clust_rec_for_mysql()
8/23/16 10
a. Get the clust_index from sec_index
b. Searches an index tree, then positions a tree cursor
(clust_pcur) on a record corresponding to the index
(btr_pcur_open_with_no_init())
c. Get the record that clust_pcur indicates (clust_rec)
d. Caculate the offsets to each field in the record (offsets)
e. If select_lock_type == LOCK_S or LOCK_X,
- try to place a lock on the index record
Get clustered record using secondary index
row_sel_get_clust_rec_for_mysql()
8/23/16 11
f. Go to func_exit:
g. In func_exit:, store the current clust_pcur position if
select_lock_type is LOCK_S or LOCK_X
Get clustered record using secondary index
requires_clust_rec:
8/23/16 12
a. Retrieves the clustered record(clust_rec) corresponding
to a record in a secondary index (row_sel_get_clust_rec_for_mysql())
b. If clust_rec is delete marked, skip it è next_rec:
c. Store clust_rec into the result_rec
d. Go to normal_return:
Get clustered record using secondary index
Secondary Index Search
8/23/16 13
⑤ Commit mini-transaction
⑥ Go to func_exit:
⑦ In func_exit:, frees the space occupied by a memory
heap then exit
row_search_for_mysql() è normal_return: è func_exit:
Data records
Secondary Index Search
8/23/16 14
Primary key
Secondary index tree Primary index tree
Primary index tree
Data records
Data records
Secondary Index Search
8/23/16 15
Primary key
Secondary index tree Primary index tree
Primary index tree
Data records
Data records
Secondary Index Search
8/23/16 16
Primary key
Secondary index tree Primary index tree
Primary index tree
Data records
Data records
Proposal: Prefetching in MySQL
8/23/16 17
Primary key
Secondary index tree Primary index tree
P1
Primary key list(unsorted)
Data records
8/23/16 18
Primary key
Secondary index tree Primary index tree
P1 P2
Primary key list(unsorted)
Proposal: Prefetching in MySQL
Data records
8/23/16 19
Primary key
Secondary index tree Primary index tree
P1 P2 P3
Primary key list(unsorted)
Proposal: Prefetching in MySQL
P2
P3
Data records
8/23/16 20
Primary key
Secondary index tree Primary index tree
P1 P2 P3
Primary key list(unsorted)
Primary key list(sorted)
Proposal: Prefetching in MySQL
P2
P3
Data records
8/23/16 21
Secondary index tree Primary index tree
P1
Primary key list(sorted)
Primary index tree
Primary index tree
Data records
Data records
Proposal: Prefetching in MySQL
P2
P3
Data records
Proposal: Prefetching in MySQL
8/23/16 22
Secondary index tree Primary index tree
P1
Primary key list(sorted)
Primary index tree
Primary index tree
Data records
Data records
P2
P3
Data records
Proposal: Prefetching in MySQL
8/23/16 23
Secondary index tree Primary index tree
P1
Primary key list(sorted)
Primary index tree
Primary index tree
Data records
Data records
Improved Secondary Index Search
8/23/16 24
① Search a data record using secondary index
② Get primary key and insert it into primary key list, then
repeat ①~② process until there is no record to search
using secondary index
③ Sort the primary key list
④ Search primary index tree sequentially using sorted
primary key list
⑤ Access data record sequentially
Reference
8/23/16 25
[1] “MySQL 5.6 Reference Manual”, MySQL, https://dev.mysql.com/doc/refman/5.6/en/
[2] Jeremy Cole, “InnoDB”, https://blog.jcole.us/innodb/

More Related Content

What's hot (20)

PPTX
Linux MMAP & Ioremap introduction
Gene Chang
 
PDF
Best practices for MySQL High Availability
Colin Charles
 
PDF
Spark + Parquet In Depth: Spark Summit East Talk by Emily Curtin and Robbie S...
Spark Summit
 
PPTX
Linux memory-management-kamal
Kamal Maiti
 
PDF
spinlock.pdf
Adrian Huang
 
PDF
semaphore & mutex.pdf
Adrian Huang
 
PDF
ClickHouse Deep Dive, by Aleksei Milovidov
Altinity Ltd
 
PDF
Memory management in Linux kernel
Vadim Nikitin
 
PDF
M|18 Deep Dive: InnoDB Transactions and Write Paths
MariaDB plc
 
PDF
Process Address Space: The way to create virtual address (page table) of user...
Adrian Huang
 
PDF
[2018] MySQL 이중화 진화기
NHN FORWARD
 
PDF
Introduction VAUUM, Freezing, XID wraparound
Masahiko Sawada
 
PDF
Physical Memory Management.pdf
Adrian Huang
 
PDF
Physical Memory Models.pdf
Adrian Huang
 
PDF
Memory Mapping Implementation (mmap) in Linux Kernel
Adrian Huang
 
PPT
Linux kernel memory allocators
Hao-Ran Liu
 
PDF
kexec / kdump implementation in Linux Kernel and Xen hypervisor
The Linux Foundation
 
PDF
Vmlinux: anatomy of bzimage and how x86 64 processor is booted
Adrian Huang
 
PDF
Online MySQL Backups with Percona XtraBackup
Kenny Gryp
 
PPTX
Demystifying flink memory allocation and tuning - Roshan Naik, Uber
Flink Forward
 
Linux MMAP & Ioremap introduction
Gene Chang
 
Best practices for MySQL High Availability
Colin Charles
 
Spark + Parquet In Depth: Spark Summit East Talk by Emily Curtin and Robbie S...
Spark Summit
 
Linux memory-management-kamal
Kamal Maiti
 
spinlock.pdf
Adrian Huang
 
semaphore & mutex.pdf
Adrian Huang
 
ClickHouse Deep Dive, by Aleksei Milovidov
Altinity Ltd
 
Memory management in Linux kernel
Vadim Nikitin
 
M|18 Deep Dive: InnoDB Transactions and Write Paths
MariaDB plc
 
Process Address Space: The way to create virtual address (page table) of user...
Adrian Huang
 
[2018] MySQL 이중화 진화기
NHN FORWARD
 
Introduction VAUUM, Freezing, XID wraparound
Masahiko Sawada
 
Physical Memory Management.pdf
Adrian Huang
 
Physical Memory Models.pdf
Adrian Huang
 
Memory Mapping Implementation (mmap) in Linux Kernel
Adrian Huang
 
Linux kernel memory allocators
Hao-Ran Liu
 
kexec / kdump implementation in Linux Kernel and Xen hypervisor
The Linux Foundation
 
Vmlinux: anatomy of bzimage and how x86 64 processor is booted
Adrian Huang
 
Online MySQL Backups with Percona XtraBackup
Kenny Gryp
 
Demystifying flink memory allocation and tuning - Roshan Naik, Uber
Flink Forward
 

Similar to Secondary Index Search in InnoDB (20)

PDF
MySQL Indexing
BADR
 
PDF
MySQL Indexing
Hoang Long
 
PDF
RivieraJUG - MySQL 8.0 - What's new for developers.pdf
Frederic Descamps
 
PPT
PHP tips by a MYSQL DBA
Amit Kumar Singh
 
PDF
MySQL Interview Questions and Answers PDF By ScholarHat.pdf
Scholarhat
 
PDF
Scaling MySQL Strategies for Developers
Jonathan Levin
 
PPTX
DB LECTURE 4 INDEXINGS PPT NOTES.pptx
grahamoyigo19
 
PDF
MySQL Query And Index Tuning
Manikanda kumar
 
PDF
Mysql tracing
Anis Berejeb
 
PDF
Mysql tracing
Anis Berejeb
 
PPT
Mysql Introduction
hemant meena
 
PDF
Quick Wins
HighLoad2009
 
ODP
InnoDB: архитектура транзакционного хранилища (Константин Осипов)
Ontico
 
PDF
Bt0075 rdbms with mysql 1
Techglyphs
 
PDF
Practical my sql performance optimization
Marian Marinov
 
PDF
How to analyze and tune sql queries for better performance percona15
oysteing
 
PDF
My sql 5.6&MySQL Cluster 7.3
Oleksii(Alexey) Porytskyi
 
PDF
The MySQL SYS Schema
Mark Leith
 
PDF
Covering indexes
MYXPLAIN
 
PPTX
Работа с индексами - лучшие практики для MySQL 5.6, Петр Зайцев (Percona)
Ontico
 
MySQL Indexing
BADR
 
MySQL Indexing
Hoang Long
 
RivieraJUG - MySQL 8.0 - What's new for developers.pdf
Frederic Descamps
 
PHP tips by a MYSQL DBA
Amit Kumar Singh
 
MySQL Interview Questions and Answers PDF By ScholarHat.pdf
Scholarhat
 
Scaling MySQL Strategies for Developers
Jonathan Levin
 
DB LECTURE 4 INDEXINGS PPT NOTES.pptx
grahamoyigo19
 
MySQL Query And Index Tuning
Manikanda kumar
 
Mysql tracing
Anis Berejeb
 
Mysql tracing
Anis Berejeb
 
Mysql Introduction
hemant meena
 
Quick Wins
HighLoad2009
 
InnoDB: архитектура транзакционного хранилища (Константин Осипов)
Ontico
 
Bt0075 rdbms with mysql 1
Techglyphs
 
Practical my sql performance optimization
Marian Marinov
 
How to analyze and tune sql queries for better performance percona15
oysteing
 
My sql 5.6&MySQL Cluster 7.3
Oleksii(Alexey) Porytskyi
 
The MySQL SYS Schema
Mark Leith
 
Covering indexes
MYXPLAIN
 
Работа с индексами - лучшие практики для MySQL 5.6, Петр Зайцев (Percona)
Ontico
 
Ad

Recently uploaded (20)

PPTX
How Apagen Empowered an EPC Company with Engineering ERP Software
SatishKumar2651
 
PDF
Streamline Contractor Lifecycle- TECH EHS Solution
TECH EHS Solution
 
PDF
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
PDF
Build It, Buy It, or Already Got It? Make Smarter Martech Decisions
bbedford2
 
PDF
Alarm in Android-Scheduling Timed Tasks Using AlarmManager in Android.pdf
Nabin Dhakal
 
PDF
HiHelloHR – Simplify HR Operations for Modern Workplaces
HiHelloHR
 
PPTX
Tally_Basic_Operations_Presentation.pptx
AditiBansal54083
 
PPTX
A Complete Guide to Salesforce SMS Integrations Build Scalable Messaging With...
360 SMS APP
 
PDF
Thread In Android-Mastering Concurrency for Responsive Apps.pdf
Nabin Dhakal
 
PDF
Alexander Marshalov - How to use AI Assistants with your Monitoring system Q2...
VictoriaMetrics
 
PPTX
3uTools Full Crack Free Version Download [Latest] 2025
muhammadgurbazkhan
 
PPTX
Why Businesses Are Switching to Open Source Alternatives to Crystal Reports.pptx
Varsha Nayak
 
PPTX
Engineering the Java Web Application (MVC)
abhishekoza1981
 
PDF
iTop VPN With Crack Lifetime Activation Key-CODE
utfefguu
 
PPTX
Migrating Millions of Users with Debezium, Apache Kafka, and an Acyclic Synch...
MD Sayem Ahmed
 
PDF
GetOnCRM Speeds Up Agentforce 3 Deployment for Enterprise AI Wins.pdf
GetOnCRM Solutions
 
PPTX
Fundamentals_of_Microservices_Architecture.pptx
MuhammadUzair504018
 
PDF
Beyond Binaries: Understanding Diversity and Allyship in a Global Workplace -...
Imma Valls Bernaus
 
PPTX
An Introduction to ZAP by Checkmarx - Official Version
Simon Bennetts
 
PPTX
Java Native Memory Leaks: The Hidden Villain Behind JVM Performance Issues
Tier1 app
 
How Apagen Empowered an EPC Company with Engineering ERP Software
SatishKumar2651
 
Streamline Contractor Lifecycle- TECH EHS Solution
TECH EHS Solution
 
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
Build It, Buy It, or Already Got It? Make Smarter Martech Decisions
bbedford2
 
Alarm in Android-Scheduling Timed Tasks Using AlarmManager in Android.pdf
Nabin Dhakal
 
HiHelloHR – Simplify HR Operations for Modern Workplaces
HiHelloHR
 
Tally_Basic_Operations_Presentation.pptx
AditiBansal54083
 
A Complete Guide to Salesforce SMS Integrations Build Scalable Messaging With...
360 SMS APP
 
Thread In Android-Mastering Concurrency for Responsive Apps.pdf
Nabin Dhakal
 
Alexander Marshalov - How to use AI Assistants with your Monitoring system Q2...
VictoriaMetrics
 
3uTools Full Crack Free Version Download [Latest] 2025
muhammadgurbazkhan
 
Why Businesses Are Switching to Open Source Alternatives to Crystal Reports.pptx
Varsha Nayak
 
Engineering the Java Web Application (MVC)
abhishekoza1981
 
iTop VPN With Crack Lifetime Activation Key-CODE
utfefguu
 
Migrating Millions of Users with Debezium, Apache Kafka, and an Acyclic Synch...
MD Sayem Ahmed
 
GetOnCRM Speeds Up Agentforce 3 Deployment for Enterprise AI Wins.pdf
GetOnCRM Solutions
 
Fundamentals_of_Microservices_Architecture.pptx
MuhammadUzair504018
 
Beyond Binaries: Understanding Diversity and Allyship in a Global Workplace -...
Imma Valls Bernaus
 
An Introduction to ZAP by Checkmarx - Official Version
Simon Bennetts
 
Java Native Memory Leaks: The Hidden Villain Behind JVM Performance Issues
Tier1 app
 
Ad

Secondary Index Search in InnoDB

  • 1. Secondary Index Search in MySQL Mijin An [email protected] 8/23/16 1
  • 2. Contents 8/23/16 2 • Secondary/Primary Key Record Format • Secondary Index Search • Proposal for Improvement • Improved Secondary Index Search
  • 3. Secondary Key Record Format 8/23/16 3 = Primary Key Value
  • 4. Primary Key Record Format 8/23/16 4
  • 5. Secondary Index Search 8/23/16 5 ① Start a mini-transaction ② Searches an index tree, then positions a tree cursor(pcur) on a record corresponding to the index (btr_pcur_open_with_no_init()) ③ Get the record that pcur indicates(rec) ④ Go to rec_loop: row_search_for_mysql()
  • 6. rec_loop: 8/23/16 6 a. Get the record that pcur indicates(rec) b. If record is infimum or supremum, go to next_rec: Look for matching records in a loop
  • 7. next_rec: 8/23/16 7 a. Stores current position of pcur b. Commit mini-transaction c. If record is infimum è move pcur to the next record Else if record is supremum è move pcur to the first record of next page Else è move pcur according to the search direction d. If we may need to process the record the cursor is now on, go to rec_loop: Move cursor to the next record, then search again
  • 8. rec_loop: 8/23/16 8 a. Get the record that pcur indicates(rec) b. If record is infimum or supremum, go to next_rec: c. Caculates the offsets to each field in the record (offsets) d. If select_lock_type == LOCK_S or LOCK_X, - try to place a lock on the index record e. Go to locks_ok: If index != clust_index && need_to_access_clustered, - go to requires_clust_rec: Look for matching records in a loop
  • 9. requires_clust_rec: 8/23/16 9 a. Retrieves the clustered record(clust_rec) corresponding to a record in a secondary index (row_sel_get_clust_rec_for_mysql()) Get clustered record using secondary index
  • 10. row_sel_get_clust_rec_for_mysql() 8/23/16 10 a. Get the clust_index from sec_index b. Searches an index tree, then positions a tree cursor (clust_pcur) on a record corresponding to the index (btr_pcur_open_with_no_init()) c. Get the record that clust_pcur indicates (clust_rec) d. Caculate the offsets to each field in the record (offsets) e. If select_lock_type == LOCK_S or LOCK_X, - try to place a lock on the index record Get clustered record using secondary index
  • 11. row_sel_get_clust_rec_for_mysql() 8/23/16 11 f. Go to func_exit: g. In func_exit:, store the current clust_pcur position if select_lock_type is LOCK_S or LOCK_X Get clustered record using secondary index
  • 12. requires_clust_rec: 8/23/16 12 a. Retrieves the clustered record(clust_rec) corresponding to a record in a secondary index (row_sel_get_clust_rec_for_mysql()) b. If clust_rec is delete marked, skip it è next_rec: c. Store clust_rec into the result_rec d. Go to normal_return: Get clustered record using secondary index
  • 13. Secondary Index Search 8/23/16 13 ⑤ Commit mini-transaction ⑥ Go to func_exit: ⑦ In func_exit:, frees the space occupied by a memory heap then exit row_search_for_mysql() è normal_return: è func_exit:
  • 14. Data records Secondary Index Search 8/23/16 14 Primary key Secondary index tree Primary index tree Primary index tree Data records
  • 15. Data records Secondary Index Search 8/23/16 15 Primary key Secondary index tree Primary index tree Primary index tree Data records
  • 16. Data records Secondary Index Search 8/23/16 16 Primary key Secondary index tree Primary index tree Primary index tree Data records
  • 17. Data records Proposal: Prefetching in MySQL 8/23/16 17 Primary key Secondary index tree Primary index tree P1 Primary key list(unsorted)
  • 18. Data records 8/23/16 18 Primary key Secondary index tree Primary index tree P1 P2 Primary key list(unsorted) Proposal: Prefetching in MySQL
  • 19. Data records 8/23/16 19 Primary key Secondary index tree Primary index tree P1 P2 P3 Primary key list(unsorted) Proposal: Prefetching in MySQL
  • 20. P2 P3 Data records 8/23/16 20 Primary key Secondary index tree Primary index tree P1 P2 P3 Primary key list(unsorted) Primary key list(sorted) Proposal: Prefetching in MySQL
  • 21. P2 P3 Data records 8/23/16 21 Secondary index tree Primary index tree P1 Primary key list(sorted) Primary index tree Primary index tree Data records Data records Proposal: Prefetching in MySQL
  • 22. P2 P3 Data records Proposal: Prefetching in MySQL 8/23/16 22 Secondary index tree Primary index tree P1 Primary key list(sorted) Primary index tree Primary index tree Data records Data records
  • 23. P2 P3 Data records Proposal: Prefetching in MySQL 8/23/16 23 Secondary index tree Primary index tree P1 Primary key list(sorted) Primary index tree Primary index tree Data records Data records
  • 24. Improved Secondary Index Search 8/23/16 24 ① Search a data record using secondary index ② Get primary key and insert it into primary key list, then repeat ①~② process until there is no record to search using secondary index ③ Sort the primary key list ④ Search primary index tree sequentially using sorted primary key list ⑤ Access data record sequentially
  • 25. Reference 8/23/16 25 [1] “MySQL 5.6 Reference Manual”, MySQL, https://dev.mysql.com/doc/refman/5.6/en/ [2] Jeremy Cole, “InnoDB”, https://blog.jcole.us/innodb/