Date post: | 15-Jan-2015 |
Category: |
Technology |
Upload: | swiss-big-data-user-group |
View: | 341 times |
Download: | 2 times |
Urturn on AWS -‐ scaling infrastructure, cost and 6me to market
confiden6al 3
Limited ways to express
confiden6al 4
Write something…
Write something…
Expressions ready-‐made templates
confiden6al 5
Music go-‐to-‐market
Urturn turns out to be a great response to audience engagement, one of the main challenge
of music industry
-‐> Hired talent from that industry
-‐> Got bands on urturn
-‐> Brought their fan base, centered on UK & US
confiden6al 6
confiden6al 7
#DrawUnionJ
confiden6al 8
#ImatchwithUnionJ
confiden6al 9
Traffic Peaks
Max
Time to half: 5 min for Twitter, one hour for FB
Share on FB or Twitter
Time to max: 10 sec on Twitter, 1 min on FB
Urturn Web Back - AWS Urturn Front
Js client
Urturn reactive Serving web pages
Rails
Storage AWS S3
Master Database MySQL
Global architecture
iPhone app
Expressions Apps
Urturn tasks Computing
activities, emails
Rails
Load
bal
ance
rs
Am
azon
Web
Ser
vice
s
Medias
Logs, analytics, datastores MongoDB
Mobile site
CDN CloudFront
Web server
Ngnix
Web server
Ngnix
Infrastructure @ Amazon Web Services
1 Replica
1 Slave
Urturn Web Back Urturn Front
Js client
Urturn reactive Serving web pages
Rails + Unicorn
Database MySQL
Global architecture
Google Analytics GoSquared Mixpannel
New Relic
iPhone app
Yahoo image, Youtube,
SoundCloud, Facebook, Twitter,
Instagram, ...
Mixpanel Google
Analytics
Expressions Apps
Urturn tasks Computing
activities, emails
Rails + Unicorn
Load
bal
ance
rs
Am
azon
Web
Ser
vice
s
Thum
bs
Fire
fox-
base
d
Cac
hing
M
emca
ched
Green= Scales
horizontally
Internal External
Fron
t Ba
ck
Medias
Logs, counts, analytics… MongoDB
Mobile site
CDN CloudFront
Web server
Ngnix
Scout
Web server
Ngnix
1 Slave
Indexation Solr
Mes
sage
s R
esqu
e
Mai
ls
Pos
tFix
Storage AWS S3
MySQL DB
Urturn clients
Js c
lient
Zoom on reacEve requests
iPho
ne
App
s
Load
ba
lanc
ers
MongoDB
Mob
ile s
ite
CloudFront
Memcached
Ngn
ix Queue
Rails
/public
Rails
Rails Rails
Caching
Browser
Manifest
Local Storage
Urturn JS cache
FAST SLOW -> $$$$ -> MAX USERS LIMIT LOCAL
MySQL DB
Urturn clients
Js c
lient
Keep a good experience during Overload
iPho
ne
App
s
Load
ba
lanc
ers
MongoDB
Mob
ile s
ite
CloudFront
Memcached
Ngn
ix Queue
Rails
/public
Rails
Rails Rails
Caching
Browser
Manifest
Local Storage
Urturn JS cache
Monitor overload here • Serve those already on the site • Overload page to others
FAST SLOW -> $$$$ -> MAX USERS LIMIT LOCAL
MySQL
Urturn clients
Js c
lient
A good request is a non-‐existant request
iPho
ne
App
s
Load
ba
lanc
ers
MongoDB
Mob
ile s
ite
CloudFront
Memcached
Ngn
ix Queue
Rails
/public
Rails
Rails Rails
Caching
Browser
Manifest
Local Storage
Urturn JS cache
FAST SLOW -> $$$$ -> MAX USERS LIMIT LOCAL
1 2 3 4
MySQL
Load testing
Load tesEng
Load
ba
lanc
ers
MongoDB
Grinder in the cloud
Load test scripts
Running on multiple
instances
Memcached
Ngn
ix Queue
Rails
/public
Rails
Rails Rails
FAST SLOW -> $$$$ -> MAX USERS LIMIT LOCAL
Grinder in the cloud • Automa6cally loads the
system to capacity • Make sure all pages are
in the scripts
Leveraging AWS
Reserve or not? Instance pricing
0
2000
4000
6000
8000
10000
12000
14000
16000
0 3 6 9 12 15 18 21 24 27 30 33 36
Cos
t (U
SD)
Months
On demand 1y heavy (bill anyway) 3y heavy (bill anyway) Spot
Spot instances
• AWS offers unused capacity @’spot’ price • Choose a maximum hourly price, get it at the market price
• Loose your instance without warning
• Pinterest massively uses spot intances
On-‐demand price
1 Replica
1 Slave
Urturn Web Back Urturn Front
Js client
Urturn reactive Serving web pages
Rails + Unicorn
Database MySQL
Google Analytics GoSquared Mixpannel
New Relic
iPhone app
Yahoo image, Youtube,
SoundCloud, Facebook, Twitter,
Instagram, ...
Mixpanel Google
Analytics
Expressions Apps
Urturn tasks Computing
activities, emails
Rails + Unicorn
Load
bal
ance
rs
Am
azon
Web
Ser
vice
s
Thum
bs
Fire
fox-
base
d
Cac
hing
M
emca
ched
Green= Scales
horizontally
Internal External
Fron
t Ba
ck
Medias
Logs, counts, analytics… MongoDB
Mobile site
CDN CloudFront
Web server
Ngnix
Scout
Web server
Ngnix
1 Slave
Indexation Solr
Mes
sage
s R
esqu
e
Mai
ls
Pos
tFix
Storage AWS S3
If we had to do it again…
1 Replica
1 Slave
Urturn Web Back Urturn client
Js client
Urturn reactive Serving web pages
Rails + Unicorn
Database MySQL
Google Analytics GoSquared Mixpannel
New Relic
iPhone app
Yahoo image, Youtube,
SoundCloud, Facebook, Twitter,
Instagram, ...
Mixpanel Google
Analytics
Expressions Apps
Urturn tasks Computing
activities, emails
Rails + Unicorn
Load
bal
ance
rs
Am
azon
Web
Ser
vice
s
Thum
bs
Fire
fox-
base
d
Cac
hing
M
emca
ched
Green= Scales
horizontally
Internal External
Fron
t Ba
ck
Medias
Logs, counts, analytics… MongoDB
Mobile site
Amazon CloudFront
Web server
Ngnix
Scout
Web server
Ngnix
1 Slave
Indexation Solr
Mes
sage
s R
esqu
e
Mai
ls
Pos
tFix
Amazon S3
DynamoDB
Amazon RDS
Amazon CloudSearch
Am
azon
S
QS
Am
azon
S
ES
Am
azon
E
last
iCac
he
… we’d consider this