Caching

Memcached & Redis

MMMMMM Databases

• We love them, otherwise we would not be here

Sometimes though, they suck

Need it (-5)

Need it (-4)

Need it (-3)

Need it (-2)

Need it (-1)

Need it N

ow!

Still need it

0

1000

2000

3000

4000

5000

6000

Database Performance

• Sometimes our databases under perform when we need them most!

• The mix of a heavy workload & lots of users can make performance unpredictable

Database & Application Harmony

• A well designed database schema may not store data in a way that is optimal for the application to consume it

• A well designed application does not always use objects that lend itself to optimal schema design• i.e. think ORM's like Hibernate or Active

Record

Example

• An Employee “Object” could be comprised of• Employee table• Department table• Position table • Office table• Office location table• Cost Center table• etc

Object Centric Schema

• Building a more object like schema can often lead to problems:• Bigger database sizes – i.e. table bloat,

duplicated data• Problems accessing data outside of

applications initial design • i.e. complex queries, weird access patterns

These can lead to performance issues!

Database Centric Schema

• Building a classic relational schema can also lead to issues:• Problems building and accessing the core

application object• i.e. having to access lots of tables, pulling back

data you may not need, etc

These can lead to performance issues too!

Enter External Caching

• Can speed up access substantially• Mask complex schema's

• Reduce workload on database servers

•

Why can external caching be faster?

It's really an unfair race

Reducing work

• Less Joins• Less Tables to access

• PK like access on “Objects”

Memory is faster then disk

VS

Component Access Times

• L1 cache reference

• Branch mispredict

• L2 cache reference

• Mutex lock/unlock

• Main memory reference

• Compress 1K bytes with Zippy

• Send 2K bytes over 1 Gbps network

• Read 1 MB sequentially from memory

• Round trip within same datacenter

• Disk seek

• Read 1 MB sequentially from disk

• Send packet CA->Netherlands->CA

• 0.5 ns

• 5 ns

• 7 ns

• 25 ns

• 100 ns

• 3,000 ns

• 20,000 ns

• 250,000 ns

• 500,000 ns

• 10,000,000 ns

• 20,000,000 ns

• 150,000,000 ns

See: http://www.linux-mag.com/cache/7589/1.html and Google http://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf

Network + Memory faster

+ >

External Caching Architecture

Web/App Servers

Cache Servers Database Server(s)

External Caching Architecture

DB

CacheIn

Cache?

No

Yes

ReturnData

Check Cache

Get Data ●Is the Data in Cache?

●If Yes Return it.●If no hit the database

● Store in cache● Return data

Note a Cache miss requires 3 ops instead of 1

Issues with external caching

• Thundering herd/stampede• Cache Miss is 3 hops

• Access Stale Data• Many layers here, consistency between db

and cache is only 1

• Cache warm-up• Over Cache

• Too many cache hits on a single page

Cache Stampede

• The cache key called CARS_SOLD_COUNT is set to expire at 9:07am.

• At 9:07am, the first apache request to discover no cache starts regenerating the cache key.

• This key is used on the homepage - which serves 60 apache requests/second.

• .. it takes 3 seconds to do this.• .. what happens to all other requests in this

time?

Cache Stampede (cont.)

• 60 requests / second * 3 seconds = 180.• In most naive caching implementations, this

is how many threads will be regenerating the cache key simultaneously!

• This problem is real, and can often be seen as ‘blips’ in cacti graphs.

Memcached

• The well established - battle tested cache solution• Solid• Well Known• Huge deploy base• Dead simple• Fast

How it works

• In simple terms its key-value (Value can be serialized object, text, etc)

• get(key) returns value• set(key,value)

• In Memory, no Persistence

• Supports distributed servers

• Consistent Hashing• No authentication

• By default objects < 1MB in size

• Objects can be pushed from cache or expire

Commands

• Limited commands help keep things simple

• Supported commands:• set,add,replace,append,prepend,cas• get,gets• delete• incr/decr• stat

Example

• Employee Object• $employee_id = 1234• $object ={name => 'Matt Yonkovit', Title =>

'Director of Consulting', Location => 'St Johns, MI USA', Age => 35, Department=> Consulting, Americas, Reports to=>'Peter Zaitsev', Reports to title=>'CEO' }

• set($employee_id,$object)• get($employee_id)

• Returns $object

More details

• Getting my employee object is a minimum of 194 bytes.

• Even if I need only my name, the full 194 bytes is returned.

• I then must parse this in code before I move on

Features

• Non-blocking sets• Multi-threaded

• Multi-operations

• Consistent hashing

Add Ons/Commercial

• There are add-ons for memcached that help do things like• Add replication• Add tags to your obects• Etc.

Where Memcached works

• When you have no control over the number of objects or growth

• When you want to keep things simple• Established applications with hooks

already built in• Consistent reliable speed

Redis

• Feature packed cache solution, designed for more then just caching• Advanced feature set

• Replication• Multiple data types• Persistence

• Via Journal or snapshot

• Gaining popularity•

Redis Datatypes

• Strings• Similar to memcached access• Support for

• Incr/decr• Get bit• Get/Set Range (i.e. substr)• Expiration time• Multi get/set

Redis Datatypes

• Lists • Stored in insert order• Can push at the end or beginning• Fast head/tail access• Access by index• Complex built in functions , i.e. move from 1

list to another

List Example

LPUSH employee_list "Matt"

:1

LPUSH employee_list "Peter"

:2

LRANGE employee_list 0 ­1

*2

$7

"Peter"

$6

"Matt"

RPUSH employee_list "Baron":3LRANGE employee_list 0 ­1*3$7"Peter"$6"Matt"$7"Baron"

Redis Datatypes

• Sets• Unordered collections of strings• There are no repeated members of a set• Support for

• diffing sets• Intersecting/merging• Moving form 1 set to another

Set Example

sadd matts.friends yves peter 

baron kenny

:4

smembers matts.friends

*4

$4

yves

$5

peter

$5

kenny

$5

baron

sadd kenny.friends yves baron matt:3sdiff matts.friends kenny.friends

*2$5peter$5kenny

Sorted Sets

• Sorted sets• Non-repeated• Ordered by a score• Supports

• most set functions• Ranges of scores•

Redis Datatypes

• Hashes • Object like access

• HGET <id> <key> returns Value• HGETALL <id> returns a list of key->value pairs

Revisit our Example

• Employee Object• $employee_id = 1234• $object ={name => 'Matt Yonkovit', Title =>

'Director of Consulting', Location => 'St Johns, MI USA', Age => 35, Department=> Consulting, Americas, Reports to=>'Peter Zaitsev', Reports to title=>'CEO' }

Using a hash

• HMSET 1234 name “Matt Yonkovit” Title “Director of Consulting” Location “St Johns, MI USA” Age “35” Department “Consulting, Americas” Reports_to “Peter Zaitsev” Reports_to_title “CEO”

• HGET 1234 name • returns “Matt Yonkovit”

More details

• In Memcached • Getting my employee object is a minimum of

194 bytes. • Even if I need only my name, the full 194

bytes is returned. I then must parse this in code before I move on

• Using a Redis Hash• I can return ~13 Bytes containing my name

Shortcomings

• Redis is single threaded

• Can work around it, like we used to with memcached starting multiple daemons

• Going to be fixed in upcoming Redis cluster release

• Not every client has consistent hashing

• Some operations require extra memory, i.e. snapshots

• Bound to memory on server

TBD: Where Redis works

• When storing rankings• sorted sets are great

• When you only need small pieces of an object• i.e. hash

• When you need ranges, lists, or queue• Need persistence

• Replication is helpful

Benchmarks

• Single node benchmarks are easy, but the because the server is single threaded the benchmark tools are not setup to benchmark multiple redis servers.

• Used redis-benchmark and mc-benchmark• Tests were run in a loop• Datasets where small• 2K objects, 500K requests/distribution• These will randomly return set/gets

Single Threaded Benchmarks

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 430

20000

40000

60000

80000

100000

120000

140000

160000

180000

200000

# of Writes per second

Single Threaded Server

Redis

Memcached

Single Threaded Benchmarks

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 430

20000

40000

60000

80000

100000

120000

140000

160000

180000

200000

# of Writes per second

Single Threaded Server

Redis

Memcached

Whats Up with This?

Benchmarks over time

2 22 42 62 82 102

122

142

162

182

202

222

242

262

282

302

322

342

362

382

402

422

442

462

482

502

522

542

562

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

200000

Long Run Benchmarks

redis -vs- memc

redis set

redis get

memc set

memc get

Change Versions

2 42 82 12212

2232 52

6272 92

102112 132

142152

162172

182192

202212

222232

242252

262272

282292

302312

322332

342352

362372

382392

402412

422432

442452

462472

482492

502512

522532

542552

562572

582592

602612

622632

642652

662672

682

80000

100000

120000

140000

160000

180000

200000

Long Running New Version

Redis 2.9.7 – Set Redis 2.4 – Set Memc – Set

Different Hardware – Much Slower CPU, 3.2 Kernel

1 21 41 61 81 1016

1116 26

3136 46

5156 66

7176 86

9196 106

111116

121126

131136

141146

151156

161166

171176

181186

191196

201206

211216

221226

231236

241246

251256

261266

271276

281286

291296

301306

311316

321326

331336

341346

351356

361366

371376

381

0

20000

40000

60000

80000

100000

120000

140000

Redis V Memcached

Ubuntu 12.04

Redis GET Redis SET Memc Get Memc Set

Ec2 Test

13

57

911

1315

1719

2123

2527

2931

3335

3739

4143

4547

4951

5355

5759

6163

6567

6971

7375

7779

8183

8587

8991

9395

9799

101103

105107

109111

113115

117119

121123

125127

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

ec2 tests Against Redis 2.4, Ubuntu 11.10

Get Set

There is more here that needs to be looked into

Persistence

• Redis config Options• Snapshots:

• save <seconds> <changes>

• Append Only (Journal)• appendonly <no/yes>• appendfsync <always/everysec/no>• no-appendfsync-on-rewrite <no/yes>•

•

Cost of Redis Persistance

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 430

20000

40000

60000

80000

100000

120000

140000

160000

180000

200000

Redis Benchmarks with Persistance

No Persistence

Snapshots

Append only

Need more Benchmarking

• Datatype specific benchmarks

• So I will need to build a test using the ruby client libraries that do support multiple servers via consistent hashing

• Need big dataset tests! Need to test with 64GB or 128GB of memory

• Impact of replication

Redis Issues

• Had several crashes under heavy load • Looked to be related to memory usage with

the append file

• Weird Performance Fluctuations repeated on multiple servers

• Missing consistent hashing in some clients• Rewrite underway to support cluster