Date post: | 10-May-2015 |
Category: |
Software |
Upload: | jervin-real |
View: | 1,189 times |
Download: | 3 times |
High Performance Rails with MySQLJervin Real, March 2014
I am…
• Consultant, Percona
• @dotmanila
• http://dotmanila.com/blog/
• http://www.mysqlperformanceblog.com/
Rails Fu Mastah!
http://walksalong.files.wordpress.com/2007/05/bruce_on_rails.jpg
Beginner
http://www.devonring.ca/img/ouch.png
Customer Problems
Web Apps Performance
• Powerful servers
• CPUs, higher clock speeds
• Lots of memory
• Fast storage
• Scale in the cloud
• Agile development techniques
Why Not?
• Premature scaling is expensive
• Cost inefficient
• Agile means less effective measurement to compensate
for fast deployments
Squeeze the Software
• Exhaust application optimizations first
• You cannot optimize what you can’t measure
• Cacti, NewRelic, Scout
• NewRelic RPM Developer Mode, Rails Footnotes (2, 3,
4!), Google PerfTools for Ruby
• Dumb schemas and queries (somewhat)
Characteristics of Rails
:primary_key, :string, :text, :integer, :float, :decimal, :datetime, :timestamp, :time, :date, :binary, :boolean
• Dumb schemas
• Likes to use SHOW FIELDS
Characteristics of Rails
• Dumb schemas
• Likes to use SHOW FIELDS
• N+1 queries problem
• Well documented and discouraged for use
Characteristics of Rails
• Dumb schemas
• Likes to use SHOW FIELDS
• N+1 queries problem
• Well documented and discouraged for use
• Does SELECT FOR UPDATE
• NOOP transactions still wrapped in BEGIN/COMMIT
Characteristics of Rails
• FK relationships - logical - GOOD
Rails - Good
• FK relationships - logical - GOOD
• Knows how to use PRIMARY KEYs — VERY GOOD!
Rails - Good
• FK relationships - logical - GOOD
• Knows how to use PRIMARY KEYs — VERY GOOD!
• Knows NOOP changes - GOOD
Rails - Good
• FK relationships - logical - GOOD
• Knows how to use PRIMARY KEYs — VERY GOOD!
• Knows NOOP changes - GOOD
• Database agnostic - GOOD
Rails - Good
MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - innodb_thread_concurrency = 0
MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - innodb_thread_concurrency = 0
• Not optimized for faster storage
MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - innodb_thread_concurrency = 0
• Not optimized for faster storage
• Still have bad configuration assumptions
• Query cache
• MyISAM < 5.5.5
MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - innodb_thread_concurrency = 0
• Not optimized for faster storage
• Still have bad configuration assumptions
• Query cache
• MyISAM < 5.5.5
• Still haunted by mutexes
What to Optimize - MySQL
• Disable Query Cache
What to Optimize - MySQL
• Disable Query Cache
• query_cache_size = 0
What to Optimize - MySQL
• Disable Query Cache
• query_cache_size = 0
• query_cache_type = 0
What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
• Use >= 5.5
What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
• Use >= 5.5
• Use Indexes, EXPLAIN should be your friend!
What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
• Use >= 5.5
• Use Indexes, EXPLAIN should be your friend!
• 5.6 does Subquery Optimizations
What to Optimize - MySQL
• Slow queries - search and destroy
• long_query_time = 0
• log_slow_verbosity - Percona Server
• pt-query-digest/Percona Cloud Tools
What to Optimize - MySQL
• Use InnoDB
innodb_buffer_pool_size # keep hot data in memory innodb_log_file_size # allow more IO buffer innodb_flush_method = O_DIRECT # skip OS cache innodb_[read|write]_io_threads > 4 innodb_io_capacity innodb_adaptive_flushing_method
What to Optimize - Rails
• counter_cache
• Good for InnoDB if you often SELECT COUNT(*)
• Indexes for find_by, where and family@posts = Post.where(title: params[:keyword]) !mysql> EXPLAIN SELECT `posts`.* FROM `posts` WHERE `posts`.`title` = 'LongTitles' \G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: posts type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 2 Extra: Using where 1 row in set (0.00 sec)
What to Optimize - Rails
• dependent: :destroy
24 Query BEGIN 24 Query SELECT `comments`.* FROM `comments` WHERE `comments`.`post_id` = 1 24 Query DELETE FROM `comments` WHERE `comments`.`id` = 2 24 Query DELETE FROM `posts` WHERE `posts`.`id` = 1 24 Query COMMIT
24 Query BEGIN 24 Query DELETE FROM `comments` WHERE `comments`.`post_id` = 4 24 Query DELETE FROM `posts` WHERE `posts`.`id` = 4 24 Query COMMIT
BAD
Use dependent: :delete_all
What to Optimize - Rails
• Cache SHOW FIELDS output - patches for now
• Disable innodb_stats_on_metadata
What to Optimize - Rails
• Cache SHOW FIELDS output - patches for now
• Disable innodb_stats_on_metadata
• Pull only the columns you need - especially excluding
BLOBS
@posts = Post.select(“title”)
What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affected rows
What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affected rows
• Avoid N+1 queries - use JOIN or includes
What to Optimize - Rails
@posts = Post.limit(2) [email protected] do |post| puts post.authors.name end
24 SELECT `posts`.* FROM `posts` LIMIT 2 24 Query SELECT `authors`.* FROM `authors` WHERE `authors`.`id` = 1 ORDER BY `authors`.`id` ASC LIMIT 1 24 Query SELECT `authors`.* FROM `authors` WHERE `authors`.`id` = 2 ORDER BY `authors`.`id` ASC LIMIT 1
With this:
You get this:
What to Optimize - Rails
@posts = Post.includes(:authors).limit(2)
24 Query SELECT `posts`.* FROM `posts` LIMIT 2 24 Query SELECT `authors`.* FROM `authors` WHERE `authors`.`id` IN (1, 2)
With this:
You get this:
What to Optimize - Rails
@posts = Post.joins(:authors).limit(2);
24 Query SELECT `posts`.* FROM `posts` INNER JOIN `authors` ON `authors`.`id` = `posts`.`authors_id` LIMIT 2
With this:
You get this:
What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affected rows
• Avoid N+1 queries - use JOIN or includes
• Compress MySQL requests, especially transactions!
What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affected rows
• Avoid N+1 queries - use JOIN or includes
• Compress MySQL requests, especially transactions!
• Learn and use SQL - find_by_sql
What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affected rows
• Avoid N+1 queries - use JOIN or includes
• Compress MySQL requests, especially transactions!
• Learn and use SQL - find_by_sql
• Don’t accept Model defaults
Further Thoughts
• GDB, Strace, OProfile
• Smart aggregation, use summary tables when possible
• Rails > Caching > MySQL
• Avoid:config.action_controller.session_store = :active_record_store
http://www.amyvernon.net/wp-content/uploads/2014/01/upward-graph-striving.png
Thank you!
… and
• We’re hiring!
• http://www.percona.com/about-us/careers/open-
positions