Window functions in MySQL 8.0
2 likes1,398 views
Window functions enable calculations across partitions of rows in a result set. This document discusses window function syntax, types of window functions available in MySQL 8.0 like RANK(), DENSE_RANK(), ROW_NUMBER(), and provides examples of queries using window functions to analyze and summarize data in partitions.
More Related Content
![[Pgday.Seoul 2020] SQL Tuning](https://cdn.slidesharecdn.com/ss_thumbnails/pgday-201117134901-thumbnail.jpg?width=560&fit=bounds)
Similar to Window functions in MySQL 8.0 (20)
Window functions in MySQL 8.0
- 1. Window functions in MySQL 8.0 Presented by Sri Sakthivel M.D. www.mydbops.com [email protected]
- 2. About Mydbops ● Founded in 2015, HQ in Bangalore India with 450+ customer base across the globe. ● Mydbops is on Database Consulting with core specialization on MySQL,MongoDB,PostgreSQL Administration and Support. ● We have expert team with 30+ certified DBA’s providing full time support and currently managing 400+ servers. ● Mydbops was created with a motto of developing a Devops model for Database administration offering 24*7 expert remote DBA support. ● We help organisations to architect and scale systems in MySQL/Mongo by implementing the advanced technologies in industry which are completely open source. ● We are leading solution provider in the market for all sort of cloud based database deployments and management.
- 3. About Me ● MySQL/MariaDB DBA Expert. ● AWS Solution Architect Associate ● Expertise in Gallera / PXC / InnoDB clusters ● Expertise in MySQL load balancers ProxySQL / Maxscale ● Active MySQL Blogger. ● Interested on DB Fine Tuning and SQL Load Balancers. ● Twitter : @hercules7sakthi ● Hobbies : Bike riding and like good Food.
- 4. Agenda ● What is Window function ? ● Window function vs MySQL ● Window function Syntax ● Types of Window function ● Query Example with Window function ● Conclusion
- 5. What is Window function ? Window functions enable users to perform calculations against partitions (i.e. subgroups or sections) of a result set . Window functions increase the efficiency and reduce the complexity of queries that analyze partitions (windows) of a data set by providing an alternative to more complex SQL concepts, e.g. derived queries. Window functions do not cause rows to become grouped into a single output row, the rows retain their separate identities and an aggregated value will be added to each row.
- 6. Window function vs MySQL ● Frequently requested feature for data analysis ● Supports since version MySQL 8.0 ● Supports aggregate & most non aggregate functions ● Solving analytical queries & Improves performance ● Flexible & ease to use
- 7. Window function Syntax ● Partition definition ● Order definition ● Frame definition
- 8. Window function Syntax Partition definition ( Syntax ) : ● Helps to create the partition window over the column . windows_function_name(expression) OVER({PARTITION BY <definition>}) example : root@localhost>select a.method,a.loanId,a.amt,a.Total from (select method,loanId,amt, sum(amt) over(partition by method) as Total from yp_payments) a group by a.method limit 3; +--------+---------+------+--------------------+ | method | loanId | amt | Total | +--------+---------+------+--------------------+ | 118 | 6768178 | 2233 | 12176312573.809387 | | 122 | 4953108 | 1655 | 1036464 | | 167 | 1975693 | 8235 | 634836 | +--------+---------+------+--------------------+
- 9. Window function Syntax Order definition (syntax) : ● Can use the ORDER BY inside the function OVER() windows_function_name(expression) OVER({ORDER BY ASC | DESC}) example : root@localhost:>select a.method,a.loanId,a.amt,a.Total from (select method,loanId,amt, sum(amt) over(order by method desc) as Total from yp_payments) a group by a.method limit 3; +--------+---------+------+--------------------+ | method | loanId | amt | Total | +--------+---------+------+--------------------+ | 994 | 7108021 | 2 | 67 | | 518 | 6040015 | 4118 | 5050588984.439701 | | 489 | 4703452 | 1784 | 10805334804.439701 | +--------+---------+------+--------------------+ 3 rows in set (26.15 sec)
- 10. Window function Syntax Frame definition : ● Can compute running totals for each row. ● Can compute rolling averages. Syntax : windows_function_name(expression) OVER(PARTITION BY <expression> ROWS UNBOUNDED PRECEDING) windows_function_name(expression) OVER(PARTITION BY <expression> ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING)
- 11. Window function Syntax Frame definition : Example : running incremental totals for each method , root@localhost:yp>select a.method,a.loanId,a.amt,a.incremental_Total from (select method,loanId,amt,sum(amt) over(partition by method rows unbounded preceding) as incremental_Total from yp_payments where method=489) a limit 5; +--------+---------+------+-------------------+ | method | loanId | amt | incremental_Total | +--------+---------+------+-------------------+ | 489 | 6758312 | 2233 | 2233 | | 489 | 6457319 | 2677 | 4910 | | 489 | 6094261 | 8387 | 13297 | | 489 | 6240342 | 5148 | 18445 | | 489 | 6734211 | 3604 | 22049 | +--------+---------+------+-------------------+
- 12. Window function Syntax Frame definition : Example : rolling averages for each method, root@localhost:>select a.method,a.loanId,a.amt,a.rolling_Avg from (select method,loanId,amt, avg(amt) over(partition by method rows between 1 preceding and 1 following) as rolling_Avg from yp_payments where method=489) a limit 5; +--------+---------+------+-------------------+ | method | loanId | amt | rolling_Avg | +--------+---------+------+-------------------+ | 489 | 6758312 | 2233 | 2455 | | 489 | 6457319 | 2677 | 4432.333333333333 | | 489 | 6094261 | 8387 | 5404 | | 489 | 6240342 | 5148 | 5713 | | 489 | 6734211 | 3604 | 4290 | +--------+---------+------+-------------------+ 5 rows in set (12.73 sec)
- 13. Types of Window function ● CUME_DIST() ● DENSE_RANK() ● RANK() ● FIRST_VALUE() ● LAST_VALUE() ● NTH_VALUE() ● NTILE() ● LEAD() ● LAG() ● PERCENT_RANK() ● ROW_NUMBER()
- 14. Types of Window functions CUM_DIST() : ● Returns the cumulative distribution of a value within a group of values . root@localhost>select method,loanId,amt,cume_dist() over( partition by method order by amt) as cummulative_dist from yp_payments where method=994 limit 10; +--------+---------+------+---------------------+ | method | loanId | amt | cummulative_dist | +--------+---------+------+---------------------+ | 994 | 6966384 | 1 | 0.23684210526315788 | | 994 | 6966668 | 1 | 0.23684210526315788 | | 994 | 7101985 | 1 | 0.23684210526315788 | | 994 | 7102975 | 1 | 0.23684210526315788 | | 994 | 7108441 | 1 | 0.23684210526315788 | | 994 | 7109122 | 1 | 0.23684210526315788 | | 994 | 7110529 | 1 | 0.23684210526315788 | | 994 | 7111399 | 1 | 0.23684210526315788 | | 994 | 7112577 | 1 | 0.23684210526315788 | | 994 | 6965596 | 2 | 1 | +--------+---------+------+---------------------+ 10 rows in set (0.00 sec)
- 15. Types of Window functions DENSE_RANK() : ● Rank of current row within its partition, without gaps root@localhost>select method,loanId,amt, dense_rank() over(order by amt) as dens_rank from yp_payments group by method limit 5; +--------+---------+------+------------------+ | method | loanId | amt | dens_rank | +--------+---------+------+------------------+ | 489 | 713157 | 2 | 1 | | 518 | 775470 | 2 | 1 | | 994 | 6965596 | 2 | 1 | | 167 | 11801 | 344 | 2 | | 122 | 28101 | 1075 | 3 | +--------+---------+------+------------------+ 5 rows in set (4.83 sec)
- 16. Types of Window functions RANK() : ● Rank of current row within its partition, with gaps root@localhost:yp>select method,loanId,amt, rank() over(order by amt) as _mrank from yp_payments group by method limit 5; +--------+---------+------+------------------+ | method | loanId | amt | _mrank | +--------+---------+------+------------------+ | 489 | 713157 | 2 | 1 | | 518 | 775470 | 2 | 1 | | 994 | 6965596 | 2 | 1 | | 167 | 11801 | 344 | 4 | | 122 | 28101 | 1075 | 5 | +--------+---------+------+------------------+ 5 rows in set (8.41 sec)
- 17. Types of Window functions FIRST_VALUE() : 1. Value of argument from first row of window frame root@localhost:yp>select method,loanId,amt, first_value(amt) over( partition by method order by amt) as 1st_value from yp_payments where method in (994,323); +--------+---------+------+------------------+ | method | loanId | amt | 1st_value | +--------+---------+------+------------------+ | 323 | 5217241 | 70 | 70 | | 323 | 769665 | 8118 | 70 | | 323 | 671604 | 8768 | 70 | | 994 | 6966384 | 1 | 1 | | 994 | 6966668 | 1 | 1 | | 994 | 7101985 | 1 | 1 | +--------+---------+------+------------------+
- 18. Types of Window functions LAST_VALUE() : Value of argument from last row of window frame root@localhost>select method,loanId,amt, last_value(amt) over( partition by method order by amt) as lst_value from yp_payments where method in (994,323); +--------+---------+------+------------------+ | method | loanId | amt | lst_value | +--------+---------+------+------------------+ | 323 | 5217241 | 8768 | 70 | | 323 | 769665 | 8118 | 70 | | 323 | 671604 | 70 | 70 | | 994 | 6966384 | 1 | 1 | | 994 | 6966668 | 1 | 1 | | 994 | 7101985 | 1 | 1 | +--------+---------+------+------------------+
- 19. Types of Window functions NTH_VALUE() : ● Value of argument from N-th row of window frame root@localhost>select method,loanId,amt, nth_value(amt,3) over( partition by method ) as n_value from yp_payments where method in (994,323); +--------+---------+------+------------------+ | method | loanId | amt | n_value | +--------+---------+------+------------------+ | 323 | 170645 | 5244 | 3200 | | 323 | 150974 | 3404 | 3200 | | 323 | 135282 | 3200 | 3200 | | 323 | 5217241 | 70 | 3200 | | 994 | 6965596 | 2 | 2 | | 994 | 6966384 | 1 | 2 | | 994 | 6966384 | 2 | 2 | +--------+---------+------+------------------+
- 20. Types of Window functions NTILE() : ● Bucket number of current row within its partition. root@localhost:yp>select method,loanId,amt, ntile(100) over( partition by method ) as tile_value from yp_payments where method in (994,323); +--------+---------+------+------------------+ | method | loanId | amt | tile_value | +--------+---------+------+------------------+ | 323 | 170645 | 5244 | 1 | | 323 | 150974 | 3404 | 2 | . . . . . . . . . | 323 | 5217241 | 70 | 79 | | 994 | 6965596 | 2 | 1 | | 994 | 6966384 | 1 | 2 | . . . . . . . . . | 994 | 7112577 | 2 | 38 | +--------+---------+------+------------------+
- 21. Types of Window functions LEAD() : ● Value of argument from row leading current row within partition root@localhost>select method,loanId,amt, lead(amt) over( partition by method ) as lead_smt, amt - lead(amt) over(partition by method) as lead_diff from yp_payments where method in (323); +--------+---------+------+----------+-----------+ | method | loanId | amt | lead_smt | lead_diff | +--------+---------+------+----------+-----------+ | 323 | 170645 | 5244 | 3404 | 1840 | | 323 | 150974 | 3404 | 3200 | 204 | | 323 | 135282 | 3200 | 5174 | -1974 | | 323 | 288222 | 5174 | 5699 | -525 | | 323 | 376684 | 5699 | 5489 | 210 | | 323 | 432680 | 5489 | 5074 | 415 | | 323 | 506159 | 5074 | 3500 | 1574 | . . . . . . . . . +--------+---------+------+----------+-----------+ 79 rows in set (0.00 sec)
- 22. Types of Window functions LAG() : ● Value of argument from row lagging current row within partition root@localhost>select method,loanId,amt, lag(amt) over( partition by method ) as lag_amt, amt - lag(amt) over(partition by method) as lag_diff from yp_payments where method in (323); +--------+---------+------+---------+----------+ | method | loanId | amt | lag_amt | lag_diff | +--------+---------+------+---------+----------+ | 323 | 170645 | 5244 | NULL | NULL | | 323 | 150974 | 3404 | 5244 | -1840 | | 323 | 135282 | 3200 | 3404 | -204 | | 323 | 288222 | 5174 | 3200 | 1974 | | 323 | 376684 | 5699 | 5174 | 525 | | 323 | 432680 | 5489 | 5699 | -210 | | 323 | 506159 | 5074 | 5489 | -415 | . . . . . . . +--------+---------+------+---------+----------+
- 23. Types of Window functions PERSENT_RANK() : ● Percentage rank value root@localhost>select method,loanId,amt, percent_rank() over( order by method) as rank_per from yp_payments where method in (994,323); +--------+---------+------+--------------------+ | method | loanId | amt | rank_per | +--------+---------+------+--------------------+ | 323 | 170645 | 5244 | 0 | . . . . . . . . . .. . . . . . . . | 323 | 5217241 | 70 | 0 | | 994 | 6965596 | 2 | 0.6810344827586207 | . . . . . . . . . .. . . . . . . . | 994 | 7112577 | 2 | 0.6810344827586207 | +--------+---------+------+--------------------+ 117 rows in set (0.00 sec)
- 24. Types of Window functions ROW_NUMBER() : ● Percentage rank value root@localhost>select method,loanId,amt, row_number() over( order by method ) as rw_num from yp_payments where method in (994,323); +--------+---------+------+------------------+ | method | loanId | amt | rw_num | +--------+---------+------+------------------+ | 323 | 170645 | 5244 | 1 | | 323 | 150974 | 3404 | 2 | | 323 | 135282 | 3200 | 3 | | 323 | 288222 | 5174 | 4 | | 323 | 376684 | 5699 | 5 | ..... . . . . . . . .. . . . . . | 994 | 7112577 | 2 | 117 | +--------+---------+------+------------------+ 117 rows in set (0.01 sec)
- 25. Query example with windows function ● Need top 2 methods ( column ) from yp_payments table, which having huge amount ● The amount should be above 50000 thousands ● Need to know the highest amount from both methods
- 26. Query example with windows function Using windows function : root@localhost> select a.method,a.loanId,a.amt from (select method,loanId,amt,rank() over(partition by method order by amt desc) as rank_gap from yp_payments where amt > 50000) a where a.rank_gap=1 order by a.amt desc limit 2; +--------+---------+--------+ | method | loanId | amt | +--------+---------+--------+ | 118 | 448 | 508698 | | 518 | 5377057 | 92839 | +--------+---------+--------+ 2 rows in set (3.02 sec) Execution Time : 3 seconds +----+-------------+-------------+------------+------+---------------+-------------+---------+-------+---------+----------+-----------------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------------+------------+------+---------------+-------------+---------+-------+---------+----------+-----------------------------+ | 1 | PRIMARY | <derived2> | NULL | ref | <auto_key0> | <auto_key0> | 8 | const | 10 | 100.00 | Using filesort | | 2 | DERIVED | yp_payments | NULL | ALL | NULL | NULL | NULL | NULL | 5549711 | 33.33 |Using where; Using filesort | +----+-------------+-------------+------------+------+---------------+-------------+---------+-------+---------+----------+-----------------------------+ 2 rows in set, 2 warnings (0.00 sec)
- 27. Query example with windows function normal query : root@localhost>select method,loanId,max(amt) from yp_payments where amt > 50000 group by method order by amt desc limit 2; +--------+--------+----------+ | method | loanId | max(amt) | +--------+--------+----------+ | 118 | 448 | 508698 | | 518 |5377057 | 92839 | +--------+--------+----------+ 2 rows in set (12.00 sec) Execution Time : 12 seconds +----+-------------+-------------+------------+-------+-----------------------------------+---------------------------+---------+------+---------+----------+----------------------------------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------------+------------+-------+-----------------------------------+---------------------------+---------+------+---------+----------+----------------------------------------------+ | 1 | SIMPLE | yp_payments | NULL | index | fk_yp_payments_method_idx,idx_met | fk_yp_payments_method_idx | 4 | NULL | 5549920 | 33.33 | Using where; Using temporary; Using filesort | +----+-------------+-------------+------------+-------+-----------------------------------+---------------------------+---------+------+---------+----------+----------------------------------------------+ 1 row in set, 1 warning (0.00 sec)
- 28. Query example with windows function ● GROUP BY & ORDER BY required for normal query to sort the results ● Query without windows function is creating temporary table ● Temporary tables are very critical ( disk temporary tables) on huge data set
- 29. For an enterprise class Database support and DB managed services reach Mydbops Email : [email protected]
- 30. Thank you !!!