Date post: | 05-Dec-2014 |
Category: |
Technology |
Upload: | naver-d2 |
View: | 4,403 times |
Download: | 0 times |
real-time large data
@ra!deview - 17 september 2012
there are over400 million tweetsa day
a second4600 tweets
0.2 msa tweet≈
Pull Push
Targeted twitter.comhome_timeline API
User / Site StreamsMobile Push (SMS, etc.)
Queried Search API Track / Follow Streams
Write API
Fanout
Tim
elin
e C
ache
Redis
Ingester
Sear
ch C
ache
Redis
Push
Com
pute
HTTP Push
Bat
ch C
ompu
te
Hadoop
Timeline Service
Ingester
Sear
ch C
ache
RedisRedisEarlybird
Blender
Push
Com
pute
HTTP Push
Mobile Push B
atch
Com
pute
Hadoop
Write API
Fanout
RedisRedis
Tim
elin
e C
ache
Redis
Social Graph Service
Timeline Service
Ingester
Sear
ch C
ache
RedisRedisEarlybird
Blender
Push
Com
pute
HTTP Push
Mobile Push B
atch
Com
pute
Hadoop
Write API
Fanout
RedisRedis
Tim
elin
e C
ache
Redis
Social Graph Service
Ingester
Sear
ch C
ache
RedisRedisEarlybird
Blender
Push
Com
pute
HTTP Push
Mobile Push B
atch
Com
pute
Hadoop
Write API
Fanout
RedisRedis
Tim
elin
e C
ache
Timeline Service
Redis
Bat
ch C
ompu
te
Hadoop
Push
Com
pute
HTTP Push
Sear
ch In
dex
Blender
Redis
Timeline Service
Ingester
Earlybird
Write API
Fanout
RedisRedis
Tim
elin
e C
ache
RedisEarlybirdMobile Push
STORAGE & RETRIEVAL
LOGICPRESENTATIONROUTING
Monorail
T-Bird
T-Flock + Haplo
Flock(s)
Darkwing
Macaw+Swift
Story Service
Macaw+Disco
STORAGE & RETRIEVAL
LOGICPRESENTATIONROUTING
T-Bird
T-Flock + Haplo
Flock(s)
Darkwing
Tweetypie
Gizmoduck
TLS
Social Graph Service
Monorail
Woodstar
TFE
timeline delivery statistics⇢30b deliveries / day (~20m / min)
⇢3.5 seconds @ p50 to deliver to 1m
⇢~350k deliveries / sec
#JoinTheFlock