Date post: | 04-Apr-2018 |
Category: |
Documents |
Upload: | simon-wistow |
View: | 219 times |
Download: | 0 times |
of 57
7/30/2019 Holistic Systems Understanding
1/57
Hi!
Tuesday, December 18, 2012
7/30/2019 Holistic Systems Understanding
2/57
Tuesday, December 18, 2012
7/30/2019 Holistic Systems Understanding
3/57
Simon Wistow
Tuesday, December 18, 2012
7/30/2019 Holistic Systems Understanding
4/57
Tuesday, December 18, 2012
This isme
7/30/2019 Holistic Systems Understanding
5/57
Tuesday, December 18, 2012
This is the company I helped found.
More on that later
7/30/2019 Holistic Systems Understanding
6/57
Tuesday, December 18, 2012
These are some of the places Ive worked.
Some of them were even cool at the time.
7/30/2019 Holistic Systems Understanding
7/57
Tuesday, December 18, 2012
Ive worked at a whole bunch of other startups.
Many of them are dead.
7/30/2019 Holistic Systems Understanding
8/57
Tuesday, December 18, 2012
I also used to work doing VFX for films including this cheeky chappy.
7/30/2019 Holistic Systems Understanding
9/57
Tuesday, December 18, 2012
This handsome devil (as me after for the story involving the chocolate river)
7/30/2019 Holistic Systems Understanding
10/57
Tuesday, December 18, 2012
This won anOscar
7/30/2019 Holistic Systems Understanding
11/57
7/30/2019 Holistic Systems Understanding
12/57
Tuesday, December 18, 2012
Im sorry about that lastone.
7/30/2019 Holistic Systems Understanding
13/57
Tuesday, December 18, 2012
I also used to work as aCowboy.
7/30/2019 Holistic Systems Understanding
14/57
Tuesday, December 18, 2012
But in Australia.
And yes, I did have to wear chaps.
Although I am at pains to point out that ALL chaps are assless. Otherwise theyre just leathertrousers.
I also had to shave my legs. True Story!
7/30/2019 Holistic Systems Understanding
15/57
Tuesday, December 18, 2012
How many of you read HackerNews?
7/30/2019 Holistic Systems Understanding
16/57
Tuesday, December 18, 2012
How about /r/programming?
7/30/2019 Holistic Systems Understanding
17/57
Tuesday, December 18, 2012
Out of curiosity -Slashdot?
7/30/2019 Holistic Systems Understanding
18/57
Tuesday, December 18, 2012
Anyone even rememberKuro5hin?
7/30/2019 Holistic Systems Understanding
19/57
Tuesday, December 18, 2012
The problem is that they all tend to slavishly follow the Gartner Hype Cycle
7/30/2019 Holistic Systems Understanding
20/57
\
Tuesday, December 18, 2012
7/30/2019 Holistic Systems Understanding
21/57
Tuesday, December 18, 2012
Node.js is magic - it means you scale without even thinking about it
7/30/2019 Holistic Systems Understanding
22/57
Tuesday, December 18, 2012
Then comes the inevitable backlash.
A quick straw poll - I want to store my users passwords - what should I use?
7/30/2019 Holistic Systems Understanding
23/57
Tuesday, December 18, 2012
Of course, bcrypt (or equivalent).
But lets go back to node.js ... what might be the problem?
7/30/2019 Holistic Systems Understanding
24/57
Tuesday, December 18, 2012
10 users trying to log in simultaneously when you have a work factor of 12 will mean thatperson 10 will take about 3-5 seconds to log in. Thats only 10 people.
7/30/2019 Holistic Systems Understanding
25/57
Tuesday, December 18, 2012
See, also, for example the wholeMongoDB is webscale
The problem is that people dont actually think about their problems.
7/30/2019 Holistic Systems Understanding
26/57
Tuesday, December 18, 2012
Who remembers thistalk?
7/30/2019 Holistic Systems Understanding
27/57
Tuesday, December 18, 2012
Or this one (or any of their similar ones).
The problem is that people didnt seem to understand them - they thought of them asrecipes not biographies and instead enshrined them as a set of rules Thou shalt shard,
Thou shalt use memcached
7/30/2019 Holistic Systems Understanding
28/57
Tuesday, December 18, 2012
So, what has this got to do with Fastly?
A little about us - were a next generation content delivery network. We can cache dynamicas well as static content. Were real time and we give you insane amounts of control over a
request life cycle.
7/30/2019 Holistic Systems Understanding
29/57
1. caching
2. naming3. Off BY ONE ERRORS
Tuesday, December 18, 2012
I am contractually obliged to include this joke. Its part of the Secret Guild of CDN providersrules.
7/30/2019 Holistic Systems Understanding
30/57
Tuesday, December 18, 2012
We currently do around 5-6 Billion requests a day.
7/30/2019 Holistic Systems Understanding
31/57
http://www.fastly.com/demo
Tuesday, December 18, 2012
This is a demo of our real time stats showing a subset of our customers.
We get around 90-95% hit rate depending on what sort of day some customers are having.
Out time to first byte is
7/30/2019 Holistic Systems Understanding
32/57
Tuesday, December 18, 2012
Were pathologically lazy.
Out of interest - anybody seen the Halloween Garfield cartoon? Its insane.
7/30/2019 Holistic Systems Understanding
33/57
Tuesday, December 18, 2012
This is Artur Bergman, our CEO.
[ INSERT origin story ]
7/30/2019 Holistic Systems Understanding
34/57
Tuesday, December 18, 2012
As an aside, his name literally means Mountain Man
7/30/2019 Holistic Systems Understanding
35/57
Tuesday, December 18, 2012
We famously use SSDs everywhere.
But SSDs are expensive? Not if you count IOps/$
7/30/2019 Holistic Systems Understanding
36/57
Tuesday, December 18, 2012
We try and do less with more and we really try and understand the full stack.
Were willing to put the efort in where necessary.
We wrote what amounts to our own Filesystem.
And because we dont trust Linux we have a custom Kernel build which makes fsync a no-op.
7/30/2019 Holistic Systems Understanding
37/57
gethostbyname(3)
Tuesday, December 18, 2012
Insert /etc/hostsanecdote.
7/30/2019 Holistic Systems Understanding
38/57
LEFT OUTER JOIN
Tuesday, December 18, 2012
As an aside any thoughts on how this and SSDs might be related?
7/30/2019 Holistic Systems Understanding
39/57
Tuesday, December 18, 2012
Another example of stufwere willing to do ...
How many of you have done any assembler?
[ INSERT assembler dispatch anecdote ]
7/30/2019 Holistic Systems Understanding
40/57
Tuesday, December 18, 2012
However theres some stufwe just dont care about.
The speed of our API just isnt an issue.
We happen to use Unicorn. When people start doing microbenchmarks of Unicorn vs$Whatever it makes me sigh ...
7/30/2019 Holistic Systems Understanding
41/57
Tuesday, December 18, 2012
Far better ofwith subjective wins, like making sure your site *feels* more responsive.
7/30/2019 Holistic Systems Understanding
42/57
Tuesday, December 18, 2012
[ INSERT anecdote aboutYahoo! ]
7/30/2019 Holistic Systems Understanding
43/57
Tuesday, December 18, 2012
We use MySQL cluster.
This is not a picture of MySQL cluster but its far more visually interesting.
[ INSERT Pros and Cons of MySQL Cluster ]
[ INSERT Why not Riak etc etc ]
7/30/2019 Holistic Systems Understanding
44/57
Tuesday, December 18, 2012
Weve tried various forms of Messaging.
[ INSERT anecdote about Node.js ]
7/30/2019 Holistic Systems Understanding
45/57
Tuesday, December 18, 2012
Which led to Artur tweeting this
[ INSERT anecdote about ZeroMQ ]
[ INSERT anecdote about switching to HTTP ]
7/30/2019 Holistic Systems Understanding
46/57
Tuesday, December 18, 2012
Were not special and uniquesnowflakes.
7/30/2019 Holistic Systems Understanding
47/57
Tuesday, December 18, 2012
The games industry generates stuf like this 60 times a second.
We concatenate a bunch of strings together twice a second and call it good.
7/30/2019 Holistic Systems Understanding
48/57
Tuesday, December 18, 2012
Another example: lets go back to messaging again.
These fine fellows are one of the top 5 websites in the world. They are primarily a messagingwebsite.
7/30/2019 Holistic Systems Understanding
49/57
Tuesday, December 18, 2012
By far their number 1 messaging moment came when Castle In The Sky, a 12 year old animeby Miyazaki was aired on Dec 9th, 2011.
They peaked 25,088 messages a second (by contrast the next biggest, the Euros final gotabout 15,358)
Lets assume that thats inbound tweets not total tweets and assume say a 10x fan out.
7/30/2019 Holistic Systems Understanding
50/57
Tuesday, December 18, 2012
In contrast take this story from Forbes magazine in 2009 about NASDAQ.
http://www.forbes.com/forbes/2009/0112/056.html
Obviously thats not exactly an apples to apples comparison but ...
7/30/2019 Holistic Systems Understanding
51/57
Tuesday, December 18, 2012
Lets talk about big data.
Every time people on Hacker News talk about Big Data an somewhat at CERN keels overwith sadness.
Its true. I read it on Digg.
Facebook has 40 billion photos.
Walmart does over 1 million transactions an hour. They have a database with over 2.5Petabytes.
The LHC collected 13 Pbs of data.
7/30/2019 Holistic Systems Understanding
52/57
Tuesday, December 18, 2012
I dont mean to sound relentlessly negative though.
As developers were dealing with ever increasing amounts of data - when I was building thesearch engine for LiveJournal in 2006 it was designed to deal with at least 6 billion objects.Thats only half of what Yahoo! and Google had in 2001.
7/30/2019 Holistic Systems Understanding
53/57
Tuesday, December 18, 2012
So playing around with new stuf is a good way to stop your brain atrophying and in thefuture these kind of techniques are going to be vital.
7/30/2019 Holistic Systems Understanding
54/57
Tuesday, December 18, 2012
But lets get some perspective, dont get mired in the X SUX, Y IS TEH WIN! mentality andremember that over focussing on one particular detail is not nearly as good as having anoverall understanding of how everything fits in together.
7/30/2019 Holistic Systems Understanding
55/57
Tuesday, December 18, 2012
Dont slavishly copy something just because someone else is doing it. The best technique isto analyze your problems and solve them instead.
7/30/2019 Holistic Systems Understanding
56/57
Be language agnosticBe technology agnostic
Join the cult of DevOpsLook to the past
Be open minded about the future
Tuesday, December 18, 2012
7/30/2019 Holistic Systems Understanding
57/57
QUESTIONS?REBUTTALS?
INSULTS?COMMENTARY?
Tuesday, December 18, 2012