Date post: | 08-May-2015 |
Category: |
Technology |
Upload: | john-barton |
View: | 3,105 times |
Download: | 1 times |
SCALING RAILS
A topic no longer as amusingly controversial as when I first started speaking on the topic, but Twitter does still have the odd lolz (ha ha, scala!)
@johnbarton
http://goodfil.ms http://whoisjohnbarton.com
Why me?
Because shut up, that’s why.
Because the Envato Marketplaces are really big in
traffic and workload...
... and I helped make the computer machines do that
At a point in time where people thought Rails couldn’t do that because Twitter crashed a lot and that’s all people thought of when they said “Big Rails”
50 Million Dynamic Requests a week when I left
or 80ish requests a second
Before that I did loads of performance work at
MyCareer.com.au
After that I’m building ROFL scale social network thingo.
You know, for movies.
Go sign up at http://goodfil.ms, so I don’t get rusty at web scale.
There is a risk of me veering into my grand theory of software
development and the universe and everything.
What does “Scaling Rails” mean?
Predictably serving more customers (with your rails app)
next week/month/year than you have right now
It’s a game of tradeoffs
Cost vs. Performance vs. Reliability vs. Flexibility
“Complex systems that work evolve from simple systems that
work”
Growing a stack for scale is a similar exercise in managing
complexity as a growing codebase and you can use the
same mental models
Coupling is the #1 enemy and beating it is an elaborate game
of divide and conquer
Anyway, so some concrete things about Rails
I can save the hand-wavy stuff for a devops meetup
Step #0Measure everything you can
afford to measure
I would hope that goes without saying.New Relic and Scout are my favourite tools for that.
Step #0.5Actually check up on what you
are measuring
And learn how to interpret that data correctly (read books).
Step #1YAGNI
You Aint Gonna Need it (with caveats - you need a wee safety buffer)
Know your end game
Based on what kind of app you are working on.
Social Network, Online News, Ecommerce all have different work loads and optimal stacks. There is no one size fits all stack at the pointy end.
Know your techniques
You need to know how to get yourself out of trouble when your metrics say that you are in it. I will give a reading list for that the end.
Then you fake it till you make it
Deploy to the cloud
Probably Rackspace or Amazon,Heroku if you are lazy and rich
Avoid vendor lock in for the full stack
But it can be OK for smaller independent components. You will want to move hosting companies in your future. Probably more than once.
JB’s Golden Mini Stack
one “magic vendor cloud load balancer” + 2 app servers + one datastore + frequent backups
Rent the high memory instances for app servers
it’s almost invariably the limiting factor
Stick with SQL
Rails tooling is heavily biased that way and you will get the most stuff for free.
IT DOES NOT MATTER IF YOU CHOOSE MYSQL OR
POSTGRESQL BOTH WORKFINE AND BOTH SCALE
Use DB backed queues...
delayed_job or roll your ownDBAs will roll their eyes at this
... for the short to mid term
it is a simpler “whole stack” at the expense of doing things right
Put job queue workers on your app slices...
...in direct proportion to the amount of jobs your web processes can make. If each “app” box can do all of the ruby work for N number of requests, you can
keep adding boxes until your database explodes.
Then build a brand spanking “job system”
With Redis and whatever else magic job shit you want, and dedicated machines to working off that queue, and probably split your codebase too.
Caching is your get out of jail free card
Be careful, it can hide genuine performance problems
and once caching runs out... you are well and truly cooked
YMMV
“I hate to advocate drugs, alcohol, violence, or insanity to anyone, but they’ve always worked for me”
BOOKS•Building Scalable Web Sites (only if totally new to this)•Release it!: Design and Deploy Production-Ready Software•Scalability Rules: 50 Principles for Scaling Web Sites•High Performance MySql•PostgreSQL 9.0 High Performance•PostgreSQL 9 Admin Cookbook
INTERWEBS•http://akfpartners.com/techblog/•http://highscalability.com/•http://codeascraft.etsy.com/•http://instagram-engineering.tumblr.com/•http://www.mysqlperformanceblog.com/•http://blog.newrelic.com/category/performance-tech-tips/
Thanks!
http://goodfil.ms ● http://whoisjohnbarton.com ● @johnbarton