Date post: | 29-Nov-2014 |
Category: |
Technology |
Upload: | ivayr-farah-netto |
View: | 196 times |
Download: | 1 times |
redis love at 8tracks
@nettofarah
8tracks
8tracks
stats
• users: 13.3mi!
• public playlists: 1.2mi (4,2 mi total)!
• uploaded tracks: 31mi
not a sysadmin talk
getting sudo
http://devopsreactions.tumblr.com/post/83704083404/getting-sudo
a melhor ferramenta para o problema
http://peopletakingpictureswithipads.tumblr.com/
melhor ferramenta
http://peopletakingpictureswithipads.tumblr.com/
para o problema!
http://peopletakingpictureswithipads.tumblr.com/
esforço humano esforço da máquina
F.C.P.P. = dev <3 + machine <3
key-value store
• open source
• in memory (most of the times)
• data-structure server
data types• strings
• lists
• sets
• sorted sets
• hashes
just like that data structures class
except it is COOL!
you can turn this
into this
next next next next
head
tail
or this
next next next
things to keep in mind
• all in memory (but not restricted to)
• no joins or iterating over data (but not restricted to)
• you should be able to rebuild data
why?
• sometimes basic data structures is all you need
• some problems are hard to fit in a relational way
redis @ 8tracks = <3sidekiq
feature toggles
caching
player (moved to couchdb, then to mysql)
autocomplete (moved to SOLR, then to ElasticSearch)
tag browsing!
news feed
listening sessions
rate limiter
awesome documentation
sidekiq
what’s indie?tag
artistplaylist
1 SELECT name !! ! FROM tags !! ! WHERE name !! ! LIKE 'ind%' LIMIT 10;!!2 SELECT name !! ! FROM mixes !! ! WHERE name !! ! LIKE 'ind%' LIMIT 10;!!3 SELECT login !! ! FROM users !! ! WHERE login !! ! LIKE 'ind%' LIMIT 10;!!4 SELECT name !! FROM artists !! WHERE name !! LIKE 'ind%' LIMIT 10;!
you could do
1 (SELECT name FROM tags WHERE name LIKE 'ind%' LIMIT 10)!2 UNION ALL!3 (SELECT name FROM mixes WHERE name LIKE 'ind%' LIMIT 10);!
the problems
• cross-type search
• I don’t want 10 of each, I want the 10 most relevant ones
• can’t select different column counts
not a fair comparison
people don’t use sql databases for search. they use SOLR (or elasticsearch)
sometimes basic data structures!
is all you need
sorted sets
• score
• add, remove, or update log(n)
• elements in order, fast existence test, fast access to elements in the middle
breaking it down
indie =>
probably irrelevant
relevant
i !! ! ! ! ! ! ! in !!
! ! ! ! ! ! ! ind!! ! ! ! ! ! ! indi!! ! ! ! ! ! ! indie
our collection
The indie summer Indiana Industrial
one zset per n-gram
ind [ ][ ][ ]
indi
indie
which is the same asZADD ind, <score>,
ZADD indi, <score>,
ZADD indie, <score>,
searching
1 ind = $r.zevrange('ind', 0, 10)!!
2 indie = $r.zevrange('indie', 0, 10)!
tag browsing
how?
tags mixes
taggings
a single tag1 SELECT m.* FROM mixes m!2 INNER JOIN taggings tg!3 ON tg.mix_id = m.id!4 WHERE tg.tag_id = 28!5 LIMIT 10;!
chill
two tags1 SELECT m.* FROM mixes m!2 INNER JOIN taggings tg!3 ON tg.mix_id = m.id!4 INNER JOIN taggings tg2!5 ON tg2.mix_id = m.id!6 WHERE tg.tag_id = 28!7 AND tg2.tag_id = 12!8 LIMIT 10;! chill
acoustic
adding sort1 SELECT m.* FROM mixes m!2 INNER JOIN taggings tg!3 ON tg.mix_id = m.id!4 INNER JOIN taggings tg2!5 ON tg2.mix_id = m.id!6 WHERE tg.tag_id = 28!7 AND tg2.tag_id = 12!8 ORDER BY m.first_published_at!9 LIMIT 10;!
ZINTERSTORE
http://redis.io/commands/zinterstore
zinterstore
Time complexity: O(N*K)+O(M*log(M))
intersects two sets, then store them in a new different set
we’ll get to this
chill
acoustic
chill acoustic
+
chill acoustic
1 sets = ['chill', 'acoustic']!2 cache_key = 'chill:acoustic:cache'!3 $r.zinterstore(cache_key, sets, :weights => [1, 1])!
chill
chill acoustic
popular
1 sets = ['popular', 'chill', 'acoustic']!2 cache_key = 'popular:chill:acoustic:cache'!3 $r.zinterstore(cache_key, sets, :weights => [1, 0, 0])!
back to the BIG O thing
O(N*K)+O(M*log(M))
- N: smallest input - K: number of sorted sets - M: number of elements of the resulting set
cool stuff worth checking out
• pipelining
• lua scripting
• replication + persistence
lessons
• it consumes a LOT of memory (duuuh)
• sentinel for failure recovery
• DO NOT USE “keys” in production
• duplication is tricky
gracias!
thank you!
obrigado!