Social networking architectures what can we

learn

An enthusiasts view2

How do sites with a social networking angle figure globally?

3

As ranked by AlexaSite Global rankingFacebook 2YouTube 3Yahoo 4Windows Live 5Blogger 7Wikipedia 8Twitter 10

3 Principles4

3 common principles

1.Fast feature delivery is key

2.Cache everything everywhere

3.Relational data is dead

Interesting stats5

Facebook - Serve 120 million queries per second without a single join

37 Signals - Developed a production application serving over 4 million items using only 579 lines of code

Flickr - 2 Billion photos served without using relational databases

How did they do it?6

Nobody thought this was possible

Unencumbered by history or restrictive rules

Had to be creative in solving problems that nobody had experienced using very little capital outlay

3 Principles7

1.Fast feature delivery is key

2. Cache everything everywhere

3. Relational data is dead

Fast feature delivery is key8

Choose an appropriate language

Speed of development more important than speed of execution

Languages like PHP and Ruby commonly used for rapid development and deployment

Language is not religion9

3 Principles10

1. Fast feature delivery is key

2.Cache everything everywhere

3. Relational data is dead

Cache everything everywhere11

You need a really good reason not to cache data for reading

Local caching a good start but more than one server means duplicating the cache no group invalidation memory limited to how much spare RAM on the server

Most social networks use a distributed cache like memcached

Cache everything everywhere12

1. Check if the information is in the cache. If so, use it

2. If not, query the database put the result in the cache

3. On update delete from the cache. The next user goes to the database

function get_foo(int userid) { result = memcached_fetch("userrow:" + userid);

if (!result) { result = db_select("SELECT * FROM users WHERE userid = ?", userid); memcached_add("userrow:" + userid, result);

} return result;

Responsivness is key13

3 Principles14

1. Fast feature delivery is key

2. Cache everything everywhere

3.Relational data is dead

Everybody wants to use a database15

Relational issue No 1 - Normalisation16

Relational databases do not scale well because of normalisation

Why normalise?- reduce storage space- reduce anomalies

Today - storage is cheap- as data gets larger, joins are

expensive

Relational issue No 2 - Transactions17

ACID principles govern transactions

Relational databases do not scale well because of transactions

After relational18

Use BASE (basically available, soft state, eventually consistent)

Shard Data

Favour Name value pair stores over relational databases

Lessons for enterprise19

Design of software should always be it depends.

Test your most basic assumptions

1. Dynamic languages and frameworks may be suitable to deliver a feature quickly

2. You don't need an RDBMS for everything, especially if you need huge scale

3. You should always cache data for read (unless you shouldn’t)

Fresh ideas always welcome20

Find me here 21

MarkGreville@itarc.ie

Or find me here22