MySQL 8.0 EXPLAIN ANALYZE
- 1. Norvald H. Ryeng Software Development Director MySQL Optimizer Team January 30, 2020 MySQL 8.0 EXPLAIN ANALYZE New in 8.0.18
- 2. Safe harbor statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation.
- 3. Program agenda 1 Quick demo 2 The MySQL query executor 3 EXPLAIN FORMAT=TREE 4 EXPLAIN ANALYZE
- 6. The MySQL query executor TurtlesIterators all the way down
- 7. Refactoring the executor Refactor iterator interfaces From many to one interface From C style function pointers to C++ classes Possible because phases were separated Much more modular exeuctor One common iterator interface for all operations Each operation is contained within an iterator Able to put together plans in new ways Immediate benefit: Removes temporary tables in some cases Join is just an iterator Nested loop join is just an iterator Hash join is just an iterator Your favorite join method is just an iterator Parse Prepare Optimize Execute SQL Resolve Transform Abstract syntax tree Logical plan Physical plan
- 8. MySQL iterator executor Each operation is an iterator Execution loop reads from root node Row by row May trigger multiple read calls further down Common interface Init() Read() HashJoinIterator TableScanIterator TableScanIterator SELECT * FROM t1 JOIN t2 ON t1.a = t2.a; t1 t2
- 9. Old MySQL executor vs. iterator executor Old executor Nested loop focused Hard to extend Code for one operation spread out Different interfaces for each operation Combination of operations hard coded Iterator executor Modular Easy to extend Each iterator encapsulates one operation Same interface for all iterators All operations can be connected
- 10. MySQL 8.0 features based on the iterator executor Hash join Just another iterator type EXPLAIN FORMAT=TREE Print the iterator tree EXPLAIN ANALYZE 1. Insert intstrumentation nodes in the tree 2. Execute the query 3. Print the iterator tree Parse Prepare Optimize Execute SQL Resolve Transform Abstract syntax tree Logical plan Physical plan
- 11. EXPLAIN FORMAT=TREE Old EXPLAIN implementation Analyzes query plan and prints its interpretation of it Duplicates plan interpretation EXPLAIN interpretation Executor interpretation Still used for FORMAT=TRADITIONAL and FORMAT=JSON Hope to change to a single EXPLAIN implementation EXPLAIN FORMAT=TREE Direct dump of execution structures A single plan interpretation for both EXPLAIN and execution Plan to reuse same implementation for all formats HashJoinIterator TableScanIterator TableScanIterator t1 t2 EXPLAIN FORMAT=TREE SELECT * FROM t1 JOIN t2 ON t1.a = t2.a; -> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) -> Table scan on t2 (cost=0.35 rows=1) -> Hash -> Table scan on t1 (cost=0.35 rows=1)
- 12. EXPLAIN FORMAT=TREE Old EXPLAIN implementation Analyzes query plan and prints its interpretation of it Duplicates plan interpretation EXPLAIN interpretation Executor interpretation Still used for FORMAT=TRADITIONAL and FORMAT=JSON Hope to change to a single EXPLAIN implementation EXPLAIN FORMAT=TREE Direct dump of execution structures A single plan interpretation for both EXPLAIN and execution Plan to reuse same implementation for all formats HashJoinIterator TableScanIterator TableScanIterator t1 t2 EXPLAIN FORMAT=TREE SELECT * FROM t1 JOIN t2 ON t1.a = t2.a; -> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) -> Table scan on t2 (cost=0.35 rows=1) -> Hash -> Table scan on t1 (cost=0.35 rows=1)
- 13. EXPLAIN ANALYZE
- 14. TimingIteratorTimingIterator TimingIterator MySQL EXPLAIN ANALYZE Wrap iterators in instrumentation nodes Measurements Time (in ms) to first row Time (in ms) to last row Number of rows Number of loops Execute the query and dump the stats Built on EXPLAIN FORMAT=TREE HashJoinIterator TableScanIterator TableScanIterator t1 t2 EXPLAIN ANALYZE SELECT * FROM t1 JOIN t2 ON t1.a = t2.a; -> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) (actual time=0.441..0.441 rows=0 loops=1) -> Table scan on t2 (cost=0.35 rows=1) (never executed) -> Hash -> Table scan on t1 (cost=0.35 rows=1) (actual time=0.220..0.220 rows=0 loops=1)
- 15. TimingIteratorTimingIterator TimingIterator MySQL EXPLAIN ANALYZE Profiling of query execution Where does the executor spend time? What are the actual row counts? Low overhead Timing is close to normal execution But considered too expensive to enable for all queries HashJoinIterator TableScanIterator TableScanIterator t1 t2 EXPLAIN ANALYZE SELECT * FROM t1 JOIN t2 ON t1.a = t2.a; -> Inner hash join (t2.a = t1.a) (cost=0.70 rows=1) (actual time=0.441..0.441 rows=0 loops=1) -> Table scan on t2 (cost=0.35 rows=1) (never executed) -> Hash -> Table scan on t1 (cost=0.35 rows=1) (actual time=0.220..0.220 rows=0 loops=1)
- 16. 467x 1520x>1400x 1332x 968x What's wrong with Q2? SELECT s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment FROM part, supplier, partsupp, nation, region WHERE p_partkey = ps_partkey AND s_suppkey = ps_suppkey AND p_size = 4 AND p_type LIKE '%TIN' AND s_nationkey = n_nationkey AND n_regionkey = r_regionkey AND r_name = 'AMERICA' AND ps_supplycost = ( SELECT min(ps_supplycost) FROM partsupp, supplier, nation, region WHERE p_partkey = ps_partkey AND s_suppkey = ps_suppkey AND s_nationkey = n_nationkey AND n_regionkey = r_regionkey AND r_name = 'AMERICA' ) ORDER BY s_acctbal DESC, n_name, s_name, p_partkey LIMIT 100; MySQL EXPLAIN ANALYZE to the rescue!
- 17. -> Limit: 100 row(s) (actual time=591739.000..591739.018 rows=100 loops=1) -> Sort: <temporary>.s_acctbal DESC, <temporary>.n_name, <temporary>.s_name, <temporary>.p_partkey, limit input to 100 row(s) per chunk (actual time -> Stream results (actual time=591738.599..591738.772 rows=462 loops=1) -> Inner hash join (nation.n_regionkey = region.r_regionkey), (nation.n_nationkey = supplier.s_nationkey) (cost=2074295.37 rows=98) (actual -> Table scan on nation (cost=0.00 rows=25) (actual time=0.024..0.026 rows=25 loops=1) -> Hash -> Inner hash join (supplier.s_suppkey = partsupp.ps_suppkey) (cost=2074041.10 rows=98) (actual time=591735.554..591738.311 rows=46 -> Table scan on supplier (cost=0.06 rows=9760) (actual time=0.068..2.024 rows=10000 loops=1) -> Hash -> Filter: (partsupp.ps_supplycost = (select #2)) (cost=1977898.52 rows=98) (actual time=1282.855..591733.987 rows=462 loop -> Inner hash join (partsupp.ps_partkey = part.p_partkey) (cost=1977898.52 rows=98) (actual time=84.827..271.307 rows=3 -> Table scan on partsupp (cost=3.54 rows=796168) (actual time=0.034..108.684 rows=800000 loops=1) -> Hash -> Inner hash join (cost=20353.91 rows=24) (actual time=0.274..83.964 rows=780 loops=1) -> Filter: ((part.p_size = 4) and (part.p_type like '%TIN')) (cost=20353.16 rows=2204) (actual time=0.212.. -> Table scan on part (cost=20353.16 rows=198401) (actual time=0.160..63.957 rows=200000 loops=1) -> Hash -> Filter: (region.r_name = 'AMERICA') (cost=0.75 rows=1) (actual time=0.029..0.036 rows=1 loops=1) -> Table scan on region (cost=0.75 rows=5) (actual time=0.021..0.030 rows=5 loops=1) -> Select #2 (subquery in condition; dependent) -> Aggregate: min(partsupp.ps_supplycost) (actual time=189.566..189.566 rows=1 loops=3120) -> Inner hash join (nation.n_regionkey = region.r_regionkey), (nation.n_nationkey = supplier.s_nationkey) (cost -> Table scan on nation (cost=0.00 rows=25) (actual time=0.005..0.007 rows=25 loops=3120) -> Hash -> Inner hash join (supplier.s_suppkey = partsupp.ps_suppkey) (cost=77789257.28 rows=79617) (actual tim -> Table scan on supplier (cost=0.02 rows=9760) (actual time=0.014..1.237 rows=10000 loops=3120) -> Hash -> Filter: (part.p_partkey = partsupp.ps_partkey) (cost=81757.41 rows=79617) (actual time=92.08 -> Inner hash join (cost=81757.41 rows=79617) (actual time=0.018..155.118 rows=800000 loops -> Table scan on partsupp (cost=10101.54 rows=796168) (actual time=0.011..97.353 rows=8 -> Hash -> Filter: (region.r_name = 'AMERICA') (cost=0.75 rows=1) (actual time=0.004..0.005 -> Table scan on region (cost=0.75 rows=5) (actual time=0.003..0.003 rows=5 loo Time is in milliseconds
- 18. MySQL 8.0 query analysis toolbox Analysis: EXPLAIN Plan analysis Cost estimates EXPLAIN ANALYZE Query profiling Actual statistics Optimizer trace Debug tracing Optimizer decisions Mitigation: Optimizer hints Per query workarounds Enable/disable specific optimizations Ask for specific plan choices Access methods Join order Optimizer switch Per session workarounds Enable/disable specific optimizations New in 8.0.18
- 19. Feature descriptions and design details directly from the source https://mysqlserverteam.com/
- 20. Thank you Norvald H. Ryeng Software Development Director MySQL Optimizer Team