Advanced MySQL Query Optimizations

Advanced MySQL
Query Optimization
PHP World
November 16th, 2017

Safe Harbor Agreement
THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT
DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND
MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A
COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY,
AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS.
THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR
FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE
SOLE DISCRETION OF ORACLE.
2

Intro
So you know how to add indexes to speed queries and maybe can use EXPLAIN. But why do your
queries still stink? This session covers how the MySQL Optimizer looks at your SQL statements,
where it has to throw up its hands in frustration, and how to tame it. Plus, there are some minor
data architecture tweaks you need to know.
3

Ground Rules
The only dumb question is the one you don’t ask!
This is your time and I hope this is a wise investment!
Many people are needlessly frightened of databases.
They do not bite …
..but they will chew your butt if not careful!
Practice good programming practices & stress clarity
4

7
COST
MODEL
Hueristics
Storage Engine
Optimzer
Index
Info
MySQL Server
SELECT
City.name,
Country.name
FROM city
JOIN
(city.countryCode =
country.code)
MySQL Server Architecture

Indexes
Indexes greatly speed the lookup of records
But there is overhead
Insertion of new records
Updates of existing records
Removal of existing records
Disk space & Memory
The speed of record lookup should faster that the
speed of the overhead
8

9
General Idea
Of cost based query
optimization
1. Assign Costs to operations
2. Compute costs of alternative
plans
3. Search for lowest cost plan
Cost Based Optimizations:
Access Method
Join Order
Subquery Strategy

Cost Inputs
What goes into the cost model
IO Cost
Cost of number pages to read to
get data -- both INDEX and DATA
Schema
Length of records & keys
Uniqueness of indexes
NULLable
Statistics
Number or rows in table
Number of records in index range
Key Distribution
Index Cardinality
(Avg number of records per key)
10

Example Query
SELECT City.Name, Country.Name
FROM City
JOIN Country
ON (City.Countrycode = Country.code);
Read the City file, print the name of the city
and print the corresponding country name.
Quiz: Does having Country.Code a ‘PRI’
help in this query’s speed?
11
DESC City;
DESC Country;

Example Query
SELECT city.name as 'City',
country.name as 'Country'
FROM city
JOIN country ON
(city.countryCode=country.Code)
12

EXPLAIN
What is this saying?
1. We have to read ALL (100 %) of the City table, 4079 rows
2. For each of the rows (loop) in #1 we have to read one row in the Country table,
Will need to read 100% of that table (but one line at a time),
There is a key to use,
The PRIMARY will be used with eq_ref on world_x.City.CountryCode
Quiz: Explain the key_len length
13

Example Query
country.name as
'Country'
FROM city
JOIN country ON
(city.countryCode=country.
Code)
14

15
Query Trace
{
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "5231.03"
},
"nested_loop": [
{
"table": {
"table_name": "country",
"access_type": "ALL",
"possible_keys": [
"PRIMARY"
],
"rows_examined_per_scan": 239,
"rows_produced_per_join": 239,
"filtered": "100.00",
"cost_info": {
"read_cost": "6.00",
"eval_cost": "47.80",
"prefix_cost": "53.80",
"data_read_per_join": "61K"
},
"used_columns": [
"Code",
"Name"
]
}
},
{
"table": {
"table_name": "city",
"access_type": "ref",
"possible_keys": [
"CountryCode"
],
"key": "CountryCode",
"used_key_parts": [
"CountryCode"
],
"key_length": "3",
"ref": [
"world.country.Code"
],
"filtered": "100.00",
"cost_info": {
"read_cost": "4314.36",
"eval_cost": "862.87",
"data_read_per_join": "303K"
},
"used_columns": [
"Name",
"CountryCode"
]
}
}
]
}
}

Cost Examples
TABLE SCAN
IO-Cost -- Number of Pages in table
CPU Cost -- Number of ROWS *
ROW_EVALUATE_COST
16
RANGE SCAN
IO-Cost -- Number of Pages in Index + Number of
Rows in Range
CPU Cost -- Number Rows in range *
ROW_EVALUATE_COST

Slightly More
Complex
Example
select a,b from ab
where a > 10
and a < 25
and a NOT IN (11,19)
and (b < 5 or b > 10);
{
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "2.40"
},
"table": {
"table_name": "ab",
"access_type": "ALL",
"filtered": "14.29",
"cost_info": {
"read_cost": "2.20",
"eval_cost": "0.20",
"data_read_per_join": "15"
},
"used_columns": [
"a",
"b"
],
"attached_condition": "((`preso`.àb`.à` > 10) and
(`preso`.àb`.à` < 25) and (`preso`.àb`.à` not in (11,19))
and ((`preso`.àb`.`b` < 5) or (`preso`.àb`.`b` > 10)))"
}
}
}
17

RTFM
All you will here today is in the MySQL manual. I am just pointing out some of the stuff
that should be more obvious or need to be pointed out.
18

Optimizing Queries
with EXPLAIN
https://dev.mysql.com/doc/refman/5.7/en/using-explain.html
19
Things we could speed a week on easily!!
READ THIS LATER FOR BACKGROUND!

Use Only Index
In some cases, a query can be
optimized to retrieve values without
consulting the data rows. (An index
that provides all the necessary results
for a query is called a covering
index.) If a query uses from a table
only columns that are included in
some index, the selected values can
be retrieved from the index tree for
greater speed
https://dev.mysql.com/doc/refman/5.7/en/mysql-indexes.html
21

Example
CREATE TABLE ionly (a int, b int, c int);
CREATE INDEX idxonly ON ionly(a,c,b);
INSERT INTO ionly VALUES
(1,10,100),(2,20,200),(3,30,300),(4,40,400) ….
SELECT c FROM ionly where a=1;
22

Faster
This allows the server to ONLY read the index, not the index THEN the data
23

Note:
To eliminate rows from
consideration. If there is a choice
between multiple indexes, MySQL
normally uses the index that finds
the smallest number of rows (the
most selective index).
24
This may bite you where you
least want to be bitten

Multiple Column
Index
If the table has a multiple-column
index, any leftmost prefix of the index
can be used by the optimizer to look
up rows. For example, if you have a
three-column index on (col1, col2,
col3), you have indexed search
capabilities on (col1), (col1, col2), and
(col1, col2, col3). For more
information, see Section 8.3.5,
“Multiple-Column Indexes”.
25
Where to be careful!

Hashed Indexes
As an alternative to a composite
index, you can introduce a column
that is “hashed” based on information
from other columns.
If this column is short, reasonably
unique, and indexed, it might be
faster than a “wide” index on many
columns. In MySQL, it is very easy to
use this extra column
26
SELECT * FROM tbl_name
WHERE
hash_col=MD5(CONCAT(val1,val2))
AND col1=val1 AND col2=val2;

Compare Apples and Oranges
MySQL can use indexes on columns more efficiently if they are declared as the same type and
size. In this context, VARCHAR and CHAR are considered the same if they are declared as the same
size. For example, VARCHAR(10) and CHAR(10) are the same size, but VARCHAR(10) and CHAR(15)
are not.
For comparisons between nonbinary string columns, both columns should use the same character
set. For example, comparing a utf8 column with a latin1 column precludes use of an index.
Comparison of dissimilar columns (comparing a string column to a temporal or numeric column, for
example) may prevent use of indexes if values cannot be compared directly without conversion.
For a given value such as 1 in the numeric column, it might compare equal to any number of values in
the string column such as '1', ' 1', '00001', or '01.e1'. This rules out use of any indexes for the string
column.
https://dev.mysql.com/doc/refman/5.7/en/mysql-indexes.html
27

Where Indexes won’t work
● Indexed column used as argument to a function
YEAR(birthdate) < 2007
● Suffix searches
name LIKE ‘%ington’
● Mismatched type
last_name = 10
28

Chapter 2 Avoid Full Table Scans
(if you can)
29

Example w/WHERE
country.name as 'Country'
FROM city
JOIN country ON
(city.countryCode=country.Code)
WHERE country.Code = 'USA'
31

Finishing Full Table Scans
For small tables, a table scan often is appropriate and the performance impact is
negligible. For large tables, try the following techniques to avoid having the
optimizer incorrectly choose a table scan:
● Use ANALYZE TABLE tbl_name to update the key distributions for the
scanned table. See Section 13.7.2.1, “ANALYZE TABLE Syntax”.
● Use FORCE INDEX for the scanned table to tell MySQL that table scans
are very expensive compared to using the given index:
● SELECT * FROM t1, t2 FORCE INDEX (index_for_column)
WHERE t1.col_name=t2.col_name;
32

Query Plan
Some databases let you ‘lock down’ the query plan. MySQL ain’t one of them.
33

Index Hints
Index hints give the optimizer
information about how to choose
indexes during query processing.
Index hints are specified following
a table name.
SELECT * FROM t
USE INDEX (index1)
IGNORE INDEX (index1) FOR ORDER
BY
IGNORE INDEX (index1) FOR GROUP
BY
WHERE ... IN BOOLEAN MODE ... ;
SELECT * FROM t
USE INDEX (index1)
WHERE ... IN BOOLEAN MODE ... ; 34

Optimizer Hints
optimizer hints apply on a
per-statement basis
SELECT /*+ NO_RANGE_OPTIMIZATION(t3 PRIMARY, f2_idx) */ f1
FROM t3 WHERE f1 > 30 AND f1 < 33;
SELECT /*+ BKA(t1) NO_BKA(t2) */ * FROM t1 INNER JOIN t2 WHERE ...;
SELECT /*+ NO_ICP(t1, t2) */ * FROM t1 INNER JOIN t2 WHERE ...;
SELECT /*+ SEMIJOIN(FIRSTMATCH, LOOSESCAN) */ * FROM t1 ...;
EXPLAIN SELECT /*+ NO_ICP(t1) */ * FROM t1 WHERE ...;
35

Why Use Foreign Keys
37
If a table has many columns, and you query many different combinations of columns, it
might be efficient to split the less-frequently used data into separate tables with a few
columns each, and relate them back to the main table by duplicating the numeric ID
column from the main table. That way, each small table can have a primary key for fast
lookups of its data, and you can query just the set of columns that you need using a join
operation.
Depending on how the data is distributed, the queries might perform less I/O and take
up less cache memory because the relevant columns are packed together on disk. (To
maximize performance, queries try to read as few data blocks as possible from disk;
tables with only a few columns can fit more rows in each data block.)
https://dev.mysql.com/doc/refman/5.7/en/foreign-key-optimization.html

FKs
Foreign keys let you
cross-reference related data across
tables, and foreign key constraints
help keep this spread-out data
consistent.
MySQL supports ON UPDATE and
ON DELETE foreign key references
in CREATE TABLE and ALTER
TABLE statements. The available
referential actions are RESTRICT
(the default), CASCADE, SET
NULL, and NO ACTION.
38
MySQL supports foreign keys, which let you
cross-reference related data across tables, and
foreign key constraints, which help keep this
spread-out data consistent.

Use FKs
To automatically delete or update child tables.
39

Example
CREATE TABLE parent (
id INT NOT NULL,
PRIMARY KEY (id)
) ENGINE=INNODB;
40
CREATE TABLE child (
id INT,
parent_id INT,
INDEX par_ind (parent_id),
FOREIGN KEY (parent_id)
REFERENCES parent(id)
ON DELETE CASCADE
) ENGINE=INNODB;

PK Example
select parent.id as 'parent',
child.id as 'child'
from parent
join child on
(parent.id=child.parent_id);
delete from parent where
id=20;
The PK definition removed
the corresponding child
record -- you do not have to
remove it! 41

MySQL 8
MySQL 8 fixes a lot of legacy issues
(Bug 199, ALTER TABLE, etc.) and
introduces some cool features!
42
A lot of cool stuff is on the way

NOWAIT
NOWAIT informs the server to return immediately IF the desired rows can not be locked.
How To Use NOWAIT
Start a connection to your MySQL 8 server, start a transaction, and make a query to lock up part of the
data.
mysql>START TRANSACTION;
mysql>SELECT * FROM city WHERE countryCode = 'USA' FOR UPDATE;
On a second connection, start a transaction and issue a SELECT query with NOWAIT.
mysql>SELECT * FROM city WHERE countryCode = 'USA' FOR UPDATE NOWAIT;
The second connection will get a message saying 'statement aborted because lock(s) could not be
acquired immediately and NOWAIT is SET
43

SKIP LOCKED
SKIP LOCKED allows you to not wait about for locked records and work on what is available -- the unlocked records.
The MySQL world database has 274 records in the city table where the countryCode field equals 'USA' there are 274 records. From past
interactions, we somehow know there are some records with the ID field greater than 4000.
On MySQL shell number 1, start a transaction and lock some records
mysql>SELECT * FROM city WHERE ID > 4000 and countryCode = 'USA';
There will be 66 records.
On MySQL shell number number 2, start a transaction and lets try to lock the records starting at IS 3990 and up.
mysql>SELECT FROM city WHERE id > 3990 and countryCode='USA' FOR UPDATE SKIPPED LOCKED;
There will be 10 records. The records 4000 and up are locked by the transaction on the other shell.
44

More in 8MySQL is a major revolution, as is the new Document Store. Oracle is working very hard
to provide a much improved product with each release.
45

Another Option?
The MySQL Document Store allows
you to use MySQL as a Document
Database -- No schemas, no tables, no
data normalization -- Store data in
JSON
46
MySQL without the SQL
Oh My!!

MySQL Document Store
47
Built on the new X DevAPI and new Protocol
Allows for async calls and used for InnoDB Cluster
management
Write data to collections
But can still talk to ol’ SQL style relational tables
JavaScript, Node.JS, Java, Python, C/C++ and PHP

PHP PECL mysql_xdevapi
#!/usr/bin/php
<?PHP
// Connection Parameters
$user = 'root';
$passwd = 'S3cret#';
$host = 'localhost';
$port = '33060';
$connection_uri = 'mysqlx://'.$user.':'.$passwd.'@'.$host.':'.$port;
$nodeSession = mysql_xdevapigetNodeSession($connection_uri);
$schema = $nodeSession->getSchema("world_x");
$collection = $schema->getCollection("countryinfo");
$result = $collection->find('_id = "USA"')->execute();
$data = $result->fetchAll();
var_dump($data);
?>
48

Final point
Too much in this subject for a
mere hour!
49

“Now it’s computers and more computers
and soon everybody will have one,
3-year-olds will have computers
and everybody will know everything
about everybody else
long before they meet them.
nobody will want to meet anybody
else ever again
and everybody will be
a recluse
like I am now.”
- Charles Bukowski
August 16, 1920 – March 9, 1994
50

MySQL Optimization Details
https://www.slideshare.net/oysteing/how-to-analyze-and-tune-sql-queries-f
or-better-performance-percona15?qid=598b4222-0f0d-4a01-9661-d679093e431
0&v=&b=&from_search=3
MySQL SYS Schema
https://dev.mysql.com/doc/refman/5.7/en/sys-schema.html
MySQL Blogs
https://planet.mysql.com/
51

Thanks!
david.stokes@oracle.com
@Stoker
https://elephantdolphin.blogspot.co.uk/
https://joind.in/talk/688b0
52

Advanced MySQL Query Optimizations

More Related Content

What's hot(20)

Viewers also liked(17)

Similar to Advanced MySQL Query Optimizations(20)

More from Dave Stokes(20)

Recently uploaded(20)

Advanced MySQL Query Optimizations