MySQL 5.7 New Features for Developers

Zohar Elkayam
www.realdbamagic.com
Twitter: @realmgic
MySQL 5.7 New Features
For Developers

Who am I?
• Zohar Elkayam, CTO at Brillix
• Programmer, DBA, team leader, database trainer, public
speaker, and a senior consultant for over 18 years
• Oracle ACE Associate
• Part of ilOUG – Israel Oracle User Group
• Blogger – www.realdbamagic.com and www.ilDBA.co.il
2 http://brillix.co.il

About Brillix
• We offer complete, integrated end-to-end solutions based on best-of-
breed innovations in database, security and big data technologies
• We provide complete end-to-end 24x7 expert remote database
services
• We offer professional customized on-site trainings, delivered by our
top-notch world recognized instructors
3

Some of Our Customers
http://brillix.co.il4

Agenda
•Optimizer, Performance and InnoDB changes
•Native JSON datatype
•The Performance Schema and SYS Schema
•Other features we should know
•What are we waiting for in MySQL 8.0?

Versions Guide
•MySQL 5.7 released in October 2015
•Current version 5.7.16 (released October 2016)
• Over 200+ new features!
•Next major version will be MySQL 8.0
(Currently, DMR – a Development Milestone Release)

Optimizer, Performance and
InnoDB Changes

How Does a Query Run?

Optimizer Changes
• Parser and Optimizer refactoring
• Readability, maintainability and stability
• Separate parsing, optimizing, execution stages
• Easier feature additions, with lessened risk
• New hint framework: easier to manage, new hints
• Cost-based optimizer
• Configurable and tunable: mysql.server_cost and
mysql.engine_cost
• API for where data resides: on disk or in cache

MySQL Optimizer

Cost-based Query Optimization
•Assign cost to operations
•Assign cost to partial or alternative plans
•Search for plan with lowest cost (“best plan”)
•Cost base optimization control
• Access method
• Join order
• Subquery strategy

Optimizer Cost Model

0
20
40
60
80
100
Q3 Q7 Q8 Q9 Q12
Executiontimerelativeto5.6(%)
5 out of 22 queries get a much improved query plan (others remain the same)
MySQL 5.6
MySQL 5.7
13
Source: MySQL 5.7: 20 Years in the Making! By Geir Høydalsvik, Sr. Director, MySQL Engineering
Optimizer Cost Model: Performance Improvements
DBT-3 (Size Factor 10, CPU Bound)

0
20
40
60
80
100
Q2 Q18
CPU bound
5.6
5.7
Optimizer Cost Model: Performance Improvements
DBT-3 (Size Factor 10)
2 out of 22 queries get a significantly improved query plan (others remain the same)
0
20
40
60
80
100
Q2 Q18
Disk bound
5.6
5.7
14 Source: MySQL 5.7: 20 Years in the Making! By Geir Høydalsvik, Sr. Director, MySQL Engineering

Adjustable Cost Constants (Experimental!)
• We can change the costs factors by changing system
tables:
• Use mysql.engine_cost and mysql.server_costs to
change default values
• Use FLUSH command to make server aware of new values
(only new connections will see updated cost constants)
update mysql.engine_cost set cost_value=2
where cost_name='io_block_read_cost';
flush optimizer_costs;

The Query Rewrite Plugin
• New pre and post parse query rewrite APIs
• Users can write their own plug-ins
• Provides a post-parse query plugin
• Rewrite problematic queries without the need to make application
changes
• Add hints
• Modify join order
• Improve problematic queries from ORMs, third party apps, etc.
• Eliminates many legacy use cases for proxies

InnoDB Improvements: Temporary Tables
• Temp tables are no longer stored in normal system tables
• Definitions can be maintained in-memory (without persisting
to the disk)
• Locking constraints can be relaxed since only one client can
see these tables.

InnoDB Improvements: Native Partitioning
•Eliminates previous limitations
•Eliminates resource usage problems
• No longer using the ha_partition handler
• Reduce memory usage by 90%
• MySQL 5.7.9 will try to upgrade old partitions to native or
Use ALTER TABLE ... UPGRADE PARTITIONING
command
•Transportable tablespace support (5.7.4)

InnoDB Bulk Load Performance
•Bulk loading is now used when creating or rebuilding
indexes
•Much faster INDEX creation:
• 2-3 x performance improvement for ADD/CREATE INDEX
operations
• 2-5% improvement for standard INSERT operations

InnoDB Page level Compression
•Not the same as Table Compression (5.1 feature)
•Reduces IO for better performance
•The compressed data is written to disk, where the hole
punching mechanism then releases empty blocks from
the end of the page.
•If compression fails, data is written out as-is.
•Supports Zlib and LZ4 compressions

InnoDB Online Operations
•Resize the InnoDB Buffer Pool online
•More online ALTER operations
• Rename indexes
• Enlarge Varchar columns
• Optimize Table (5.7.4)
•Truncate undo logs
•More dynamic configuration variables

Native JSON Support

The JSON Native Datatype
• New native data type: JSON
• Supports all JSON internal types
• Numbers, strings, bool
• Objects, JSON arrays
• Supports also extended data types: date, time, datetime, and
timestamp
• Efficient access: optimized for read intensive workload
• Performance: fast access to array cells by creating indexes
• MySQL Document Store (5.7.12)

Why Not Just Use TEXT/Varchar?
•Document validation: parse and validation on insert
•Efficient binary format
• Allows quicker access to object members and array
elements
• Binary format of JSON type is very efficient at searching
• Storing as TEXT performs over 10x worse at traversal
•Built-in handling functions

Built-in JSON Functions
•5.7 has built in functions to CREATE, SEARCH,
MODIFY and RETURN JSON documents and JSON
values
•For a complete list of the JSON functions:
https://dev.mysql.com/doc/refman/5.7/en/json-function-
reference.html

Create JSON_OBJECT and JSON_ARRAY
•Use JSON_OBJECT to create JSON from tables:
•JSON_ARRAY is a function that generates a
JSON array from a list of values
mysql> select JSON_OBJECT('id', id, 'firstName', first_name, 'lastName',
last_name) from employees_old;
+-----------------------------------------------------------------------+
| JSON_OBJECT('id', id, 'firstName', first_name, 'lastName', last_name) |
+-----------------------------------------------------------------------+
| {"id": 1, "lastName": "Elkayam", "firstName": "Tamar"} |
| {"id": 2, "lastName": "Elkayam", "firstName": "Efrat"} |
| {"id": 3, "lastName": "Elkayam", "firstName": "Zohar"} |
| {"id": 4, "lastName": "Elkayam", "firstName": "Ido"} |
+-----------------------------------------------------------------------+
4 rows in set (0.00 sec)

Extracting a Value: JSON_EXTRACT
•JSON_EXTRACT – Extracts data from the JSON
• Uses JSON Path
• supports two shorthand operators:
JSON_EXTACT (column_name, “$.type”(
column_name->"$.type" (extract)
column_name->>"$.type" (extract + unquote)

Searching In JSON
•Use JSON_SEARCH to find paths with certain data
•Use JSON_CONTAINS and JSON_CONTAIN_PATH
to know if data exists in the document
•Use the extract shorthand syntax to locate the rows
containing data
JSON_SEARCH (column_name, ‘one-or-all’, ‘value’(
WHERE column_name->"$.type" = ‘value’

Modifying JSON Documents
•We can modify existing JSON with various functions
• JSON_APPEND/JSON_ARRAY_APPEND: appends
value/JSON array to a JSON documents
• JSON_INSERT/JSON_ARRAY_INSERT: adds data/JSON
array to a JSON
• JSON_MERGE: merge JSONs
• JSON_REPLACE: replace values at path
• JSON_SET: replace value at path or adds it if not exist

Performance: Functional Indexes with JSON
•Indexing of Documents using Generated Columns
•InnoDB supports indexes on both STORED and
VIRTUAL (default) Generated Columns
•New expression analyzer automatically uses the best
“functional” index available
alter table employees add id numeric as (data->>"$.id");
create index emp_id on employees(id);

Performance Schema and
SYS Schema

Performance Schema
•Performance monitoring schema
•Include various table to help us with instrumentation
and performance monitoring
• Memory Instrumentation
• Statement Instrumentation
• Transactions and Locks
• Additional Information: replication, stored routines, etc.

What’s New in Performance Schema
•User variables, status variables (session/global)
•Reduced footprint, memory usage and overhead
•Total of 35 new tables – 5.6 had 52, 5.7 has 87 (with
983 instrumentations!)
•But using the performance schema might be a bit too
complicated….

The SYS Schema
• A set of objects that helps DBAs and developers interpret
data collected by the Performance Schema
• Provides helper objects that answer common performance,
health, usage, and monitoring questions
• Introduced in 5.6 as part of an extension to MySQL
(available at GitHub: https://github.com/mysql/mysql-sys)
• MySQL 5.7.7+ includes the SYS schema by default

MySQL SYS Summary Views
• Reference set of views solving various administrator use cases
• Simple views, you can create/copy your own, sys is not “locked
down”
• Build upon both performance_schema and
INFORMATION_SCHEMA
• Both formatted and raw views are available
• All raw views are prefixed with x$
• Raw views are there for tools to poll
• Formatted views are for humans and the command line

What Data Can We See In Summary Views?
•SYS schema comes with multiple dimensions for
summary views:
• User/Host summary views
• IO summary views
• Schema analysis
• Wait and Lock wait analysis
• Statement analysis
• Memory views

Formatter and Helper Functions
• SYS Schema also provides formatter and helper functions:
• Make output human readable
• Format time appropriately
• Format bytes appropriately
• Truncate output for large width values for CLI
• Extract object names
• Check instrumentation state
• Performance Schema Config Helper Procedures

Useful SYS Functions
• Formatter:
• format_time
• format_bytes
• format_path
• format_statement
• Extractor
• extract_table_from_file_name
• extract_schema_from_file_name
• Performance Schema States
• ps_is_account_enabled
• ps_is_consumer_enabled
• ps_is_instrument_default_enabled
• ps_is_instrument_default_timed
• ps_is_thread_instrumented

Short Demo for SYS Schema

Other Features and MySQL
8.0

Generated Columns (5.7.5)
•Virtual Columns – functional columns calculated on the
fly OR stored result
•Generated column can use JSON extracts
•Support Indexes – Functional Indexes
•Example by Percona:
https://www.percona.com/blog/2015/04/29/generated-
virtual-columns-in-mysql-5-7-labs/

Server Side Timeouts (5.7.5)
•Interrupting the execution of a statement when it takes
longer to complete
• Defined at the Global for server, per session, or for
individual SELECT statements
•Example
SET GLOBAL MAX_STATEMENT_TIME=1000;
SET SESSION MAX_STATEMENT_TIME=2000;
SELECT /* MAX_STATEMENT_TIME=1000 */ * FROM table;

What Are We Waiting For In MySQL 8.0?
•Improve with UTF8 support: default will be utf8mb4
instead of latin1
•Transactional data dictionary
•Common Table Expression (CTE)
•Invisible mode for Indexes and other object types

What Are We Waiting For In MySQL 8.0?
(cont.)
•Performance changes
• Cost based optimizer statistics
• Remove buffer pool mutexes
• Redo log improvements for scalability
• Memcache improvement to support multiple get and range
scan
•Bug Fixes: for example, persistent auto increment (bug
#199, created on 27 March 2003)

What Features Did We NOT Talk About?
•MySQL Document Store (The X Plugin and X Protocol)
•Security Improvements
•GIS Improvements
•Replication Improvements
•More (200+ new features!):
see http://www.thecompletelistoffeatures.com/
for more details!

Q&A

Summary
•Performance, Optimizer and other InnoDB features are
really important
•Use JSON and other document store features to make
more with MySQL
•Learn how to use SYS Schema to leverage internal
knowledge to your advantage

Thank You
Zohar Elkayam
twitter: @realmgic
Zohar@Brillix.co.il
www.realdbamagic.com

MySQL 5.7 New Features for Developers

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to MySQL 5.7 New Features for Developers (20)

More from Zohar Elkayam (14)

Recently uploaded (20)

MySQL 5.7 New Features for Developers

Editor's Notes