High Performance Rails with MySQL
4 likes2,844 views
This document discusses optimizing performance for Rails applications using MySQL. It recommends exhausting application optimizations first through profiling tools. For MySQL, it suggests disabling the query cache, using indexes, optimizing schemas, and upgrading to newer versions. For Rails, it advises caching queries, avoiding N+1 queries through includes/joins, compressing transactions, and learning SQL. The goal is to squeeze performance from software before prematurely scaling hardware.
![What to Optimize - MySQL
• Use InnoDB
innodb_buffer_pool_size # keep hot data in memory
innodb_log_file_size # allow more IO buffer
innodb_flush_method = O_DIRECT # skip OS cache
innodb_[read|write]_io_threads > 4
innodb_io_capacity
innodb_adaptive_flushing_method](https://image.slidesharecdn.com/rubyconf-2014-high-performance-rails-and-mysql-140705202038-phpapp01/85/High-Performance-Rails-with-MySQL-29-320.jpg)
![What to Optimize - Rails
• counter_cache
• Good for InnoDB if you often SELECT COUNT(*)
• Indexes for find_by, where and family
@posts = Post.where(title: params[:keyword])
!
mysql> EXPLAIN SELECT `posts`.* FROM `posts` WHERE `posts`.`title` = 'LongTitles' G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: posts
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 2
Extra: Using where
1 row in set (0.00 sec)](https://image.slidesharecdn.com/rubyconf-2014-high-performance-rails-and-mysql-140705202038-phpapp01/85/High-Performance-Rails-with-MySQL-30-320.jpg)
High Performance Rails with MySQL
- 1. High Performance Rails with MySQL Jervin Real, March 2014
- 2. I am… • Consultant, Percona • @dotmanila • http://dotmanila.com/blog/ • http://www.mysqlperformanceblog.com/
- 6. Web Apps Performance • Powerful servers • CPUs, higher clock speeds • Lots of memory • Fast storage • Scale in the cloud • Agile development techniques
- 7. Why Not? • Premature scaling is expensive • Cost inefficient • Agile means less effective measurement to compensate for fast deployments
- 8. Squeeze the Software • Exhaust application optimizations first • You cannot optimize what you can’t measure • Cacti, NewRelic, Scout • NewRelic RPM Developer Mode, Rails Footnotes (2, 3, 4!), Google PerfTools for Ruby
- 9. • Dumb schemas and queries (somewhat) Characteristics of Rails :primary_key, :string, :text, :integer, :float, :decimal, :datetime, :timestamp, :time, :date, :binary, :boolean
- 10. • Dumb schemas • Likes to use SHOW FIELDS Characteristics of Rails
- 11. • Dumb schemas • Likes to use SHOW FIELDS • N+1 queries problem • Well documented and discouraged for use Characteristics of Rails
- 12. • Dumb schemas • Likes to use SHOW FIELDS • N+1 queries problem • Well documented and discouraged for use • Does SELECT FOR UPDATE • NOOP transactions still wrapped in BEGIN/COMMIT Characteristics of Rails
- 13. • FK relationships - logical - GOOD Rails - Good
- 14. • FK relationships - logical - GOOD • Knows how to use PRIMARY KEYs — VERY GOOD! Rails - Good
- 15. • FK relationships - logical - GOOD • Knows how to use PRIMARY KEYs — VERY GOOD! • Knows NOOP changes - GOOD Rails - Good
- 16. • FK relationships - logical - GOOD • Knows how to use PRIMARY KEYs — VERY GOOD! • Knows NOOP changes - GOOD • Database agnostic - GOOD Rails - Good
- 17. MySQL by Default • Assumes you have less powerful hardware • ironically on 5.5, assumes you have very powerful CPUs - innodb_thread_concurrency = 0
- 18. MySQL by Default • Assumes you have less powerful hardware • ironically on 5.5, assumes you have very powerful CPUs - innodb_thread_concurrency = 0 • Not optimized for faster storage
- 19. MySQL by Default • Assumes you have less powerful hardware • ironically on 5.5, assumes you have very powerful CPUs - innodb_thread_concurrency = 0 • Not optimized for faster storage • Still have bad configuration assumptions • Query cache • MyISAM < 5.5.5
- 20. MySQL by Default • Assumes you have less powerful hardware • ironically on 5.5, assumes you have very powerful CPUs - innodb_thread_concurrency = 0 • Not optimized for faster storage • Still have bad configuration assumptions • Query cache • MyISAM < 5.5.5 • Still haunted by mutexes
- 21. What to Optimize - MySQL • Disable Query Cache
- 22. What to Optimize - MySQL • Disable Query Cache • query_cache_size = 0
- 23. What to Optimize - MySQL • Disable Query Cache • query_cache_size = 0 • query_cache_type = 0
- 24. What to Optimize - MySQL • Disable Query Cache • skip_name_resolve
- 25. What to Optimize - MySQL • Disable Query Cache • skip_name_resolve • Use >= 5.5
- 26. What to Optimize - MySQL • Disable Query Cache • skip_name_resolve • Use >= 5.5 • Use Indexes, EXPLAIN should be your friend!
- 27. What to Optimize - MySQL • Disable Query Cache • skip_name_resolve • Use >= 5.5 • Use Indexes, EXPLAIN should be your friend! • 5.6 does Subquery Optimizations
- 28. What to Optimize - MySQL • Slow queries - search and destroy • long_query_time = 0 • log_slow_verbosity - Percona Server • pt-query-digest/Percona Cloud Tools
- 29. What to Optimize - MySQL • Use InnoDB innodb_buffer_pool_size # keep hot data in memory innodb_log_file_size # allow more IO buffer innodb_flush_method = O_DIRECT # skip OS cache innodb_[read|write]_io_threads > 4 innodb_io_capacity innodb_adaptive_flushing_method
- 30. What to Optimize - Rails • counter_cache • Good for InnoDB if you often SELECT COUNT(*) • Indexes for find_by, where and family @posts = Post.where(title: params[:keyword]) ! mysql> EXPLAIN SELECT `posts`.* FROM `posts` WHERE `posts`.`title` = 'LongTitles' G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: posts type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 2 Extra: Using where 1 row in set (0.00 sec)
- 31. What to Optimize - Rails • dependent: :destroy 24 Query BEGIN 24 Query SELECT `comments`.* FROM `comments` WHERE `comments`.`post_id` = 1 24 Query DELETE FROM `comments` WHERE `comments`.`id` = 2 24 Query DELETE FROM `posts` WHERE `posts`.`id` = 1 24 Query COMMIT 24 Query BEGIN 24 Query DELETE FROM `comments` WHERE `comments`.`post_id` = 4 24 Query DELETE FROM `posts` WHERE `posts`.`id` = 4 24 Query COMMIT BAD Use dependent: :delete_all
- 32. What to Optimize - Rails • Cache SHOW FIELDS output - patches for now • Disable innodb_stats_on_metadata
- 33. What to Optimize - Rails • Cache SHOW FIELDS output - patches for now • Disable innodb_stats_on_metadata • Pull only the columns you need - especially excluding BLOBS @posts = Post.select(“title”)
- 34. What to Optimize - Rails • Avoid pessimistic locks when possible - SELECT … FOR UPDATE - UPDATE directly and return affected rows
- 35. What to Optimize - Rails • Avoid pessimistic locks when possible - SELECT … FOR UPDATE - UPDATE directly and return affected rows • Avoid N+1 queries - use JOIN or includes
- 36. What to Optimize - Rails @posts = Post.limit(2) ! @posts.each do |post| puts post.authors.name end 24 SELECT `posts`.* FROM `posts` LIMIT 2 24 Query SELECT `authors`.* FROM `authors` WHERE `authors`.`id` = 1 ORDER BY `authors`.`id` ASC LIMIT 1 24 Query SELECT `authors`.* FROM `authors` WHERE `authors`.`id` = 2 ORDER BY `authors`.`id` ASC LIMIT 1 With this: You get this:
- 37. What to Optimize - Rails @posts = Post.includes(:authors).limit(2) 24 Query SELECT `posts`.* FROM `posts` LIMIT 2 24 Query SELECT `authors`.* FROM `authors` WHERE `authors`.`id` IN (1, 2) With this: You get this:
- 38. What to Optimize - Rails @posts = Post.joins(:authors).limit(2); 24 Query SELECT `posts`.* FROM `posts` INNER JOIN `authors` ON `authors`.`id` = `posts`.`authors_id` LIMIT 2 With this: You get this:
- 39. What to Optimize - Rails • Avoid pessimistic locks when possible - SELECT … FOR UPDATE - UPDATE directly and return affected rows • Avoid N+1 queries - use JOIN or includes • Compress MySQL requests, especially transactions!
- 40. What to Optimize - Rails • Avoid pessimistic locks when possible - SELECT … FOR UPDATE - UPDATE directly and return affected rows • Avoid N+1 queries - use JOIN or includes • Compress MySQL requests, especially transactions! • Learn and use SQL - find_by_sql
- 41. What to Optimize - Rails • Avoid pessimistic locks when possible - SELECT … FOR UPDATE - UPDATE directly and return affected rows • Avoid N+1 queries - use JOIN or includes • Compress MySQL requests, especially transactions! • Learn and use SQL - find_by_sql • Don’t accept Model defaults
- 42. Further Thoughts • GDB, Strace, OProfile • Smart aggregation, use summary tables when possible • Rails > Caching > MySQL • Avoid: config.action_controller.session_store = :active_record_store
- 44. Thank you!
- 45. … and • We’re hiring! • http://www.percona.com/about-us/careers/open- positions