+ All Categories
Home > Technology > Mongo db parse-v17

Mongo db parse-v17

Date post: 08-Sep-2014
Category:
Upload: mongodb
View: 422 times
Download: 4 times
Share this document with a friend
Description:
Parse powers over 250k mobile apps on MongoDB. Since we can have no prior knowledge of the schemas or usage patterns or scaling requirements for each app, we rely on advanced operational techniques and best practices for managing MongoDB and maintaining peak performance. In this talk we will talk about the unique challenges of this use case, such as dealing with cotenancy issues and how to automatically generate good indexes for each app. I will talk about some of the interesting operational techniques involved in managing hundreds of databases and millions of collections on MongoDB, as well as running large and growing database ecosystems on AWS. I will also talk about some of our favorite black belt tips for tuning and scaling MongoDB. And finally we will talk about how to recover from performance-impacting incidents gracefully, without making everything worse for yourself.
Popular Tags:
44
Charity Majors @mipsytipsy
Transcript
Page 1: Mongo db parse-v17

Charity Majors @mipsytipsy

Page 2: Mongo db parse-v17

</>

Mobile Backend as a Service

Data Storage

Push Notifications

Analytics

Server-Side Code

Page 3: Mongo db parse-v17

300K

250K

200K

150K

100K

50K

02011 2012 2013 2014 2015

Apps Built On Parse

270,000+

Page 4: Mongo db parse-v17

250%developers

500%api requests

Page 5: Mongo db parse-v17

a love^Wless hateful storyMongoDB + Ops:

Page 6: Mongo db parse-v17
Page 7: Mongo db parse-v17

?MongoDB + Ops:

?

Page 8: Mongo db parse-v17

Reliability

Flexibility

Automation

1

2

3

Page 9: Mongo db parse-v17

Reliability

Flexibility

Automation

1

2

3

Page 10: Mongo db parse-v17

re · li · abil · i · ty (noun) \ri-ˌlī-ə-ˈbi-lə-tē\1. Through resiliency 2. Magic of the replica set

Page 11: Mongo db parse-v17

Replica Set

Page 12: Mongo db parse-v17

Pets?

Page 13: Mongo db parse-v17

No Pets!

Page 14: Mongo db parse-v17
Page 15: Mongo db parse-v17

Arbiters

Extra Votes

Spanning AZs

Spanning DCs

Non-voting

Hidden

Client Logic

More!

Page 16: Mongo db parse-v17

Planning for failure.

Page 17: Mongo db parse-v17
Page 18: Mongo db parse-v17

Reliability

Flexibility

Automation

1

2

3

Page 19: Mongo db parse-v17

When you change a schema, everythi ng breaks !

Page 20: Mongo db parse-v17

DATA MODEL FLEXIBILITY

Page 21: Mongo db parse-v17

DATA MODELWORKLOAD FLEXIBILITY

Page 22: Mongo db parse-v17
Page 23: Mongo db parse-v17

270kapps hosted on Parse

Page 24: Mongo db parse-v17

270kapps hosted on Parse

270kdifferent workloads

=

Page 25: Mongo db parse-v17

270kapps hosted on Parse

270kdifferent workloads

=Every app must be individually performant!

Page 26: Mongo db parse-v17

DEPLOYMENT

BACKUPS

RECOVERY

DISASTER RECOVERY

DRIVERS

PARTIAL FAILURES

PERFORMANCE TUNING

INDEXING PLAN

DESIGN FAILOVER PLAN

OPERATIONAL EXPERTISE

Page 27: Mongo db parse-v17

ONEre-usable solution

Page 28: Mongo db parse-v17
Page 29: Mongo db parse-v17

Reliability

Flexibility

Automation

1

2

3

Page 30: Mongo db parse-v17

Replica Set

Page 31: Mongo db parse-v17
Page 32: Mongo db parse-v17
Page 33: Mongo db parse-v17

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

Page 34: Mongo db parse-v17

+

Page 35: Mongo db parse-v17
Page 36: Mongo db parse-v17

PLEASE. SHARE YOUR TOOLS.

Page 37: Mongo db parse-v17

MongoDB Chef Cookbook github.com/ParsePlatform/Ops/tree/master/chef/cookbooks/mongodb

AWS Chef Cookbook (ec2 raid recipe) github.com/opscode-cookbooks/aws

Continuous Compaction blog.parse.com/2013/03/26/always-be-compacting/

Warmup Scripts blog.parse.com/2013/03/07/techniques-for-warming-up-mongodb/

Page 39: Mongo db parse-v17

Mongo Proxy url url url url -- naitik shah (& blog post url)

NEW

Page 40: Mongo db parse-v17

Flashback url url url url -- kai liu (& blog post url)

NEW

Page 41: Mongo db parse-v17

a love story[*]. The End

* mutual loathing, bitterness, mistrust. Love the one you’re with.

MongoDB + Ops:

Page 42: Mongo db parse-v17

“Which database do I hate the most?”

Page 43: Mongo db parse-v17

“Which database do I hate the most?”

“The one I’m using right now”

Page 44: Mongo db parse-v17

Charity Majors @mipsytipsy


Recommended