Mongo db parse-v17

Post on 08-Sep-2014

423 views 4 download

Tags:

description

Parse powers over 250k mobile apps on MongoDB. Since we can have no prior knowledge of the schemas or usage patterns or scaling requirements for each app, we rely on advanced operational techniques and best practices for managing MongoDB and maintaining peak performance. In this talk we will talk about the unique challenges of this use case, such as dealing with cotenancy issues and how to automatically generate good indexes for each app. I will talk about some of the interesting operational techniques involved in managing hundreds of databases and millions of collections on MongoDB, as well as running large and growing database ecosystems on AWS. I will also talk about some of our favorite black belt tips for tuning and scaling MongoDB. And finally we will talk about how to recover from performance-impacting incidents gracefully, without making everything worse for yourself.

transcript

Charity Majors @mipsytipsy

</>

Mobile Backend as a Service

Data Storage

Push Notifications

Analytics

Server-Side Code

300K

250K

200K

150K

100K

50K

02011 2012 2013 2014 2015

Apps Built On Parse

270,000+

250%developers

500%api requests

a love^Wless hateful storyMongoDB + Ops:

?MongoDB + Ops:

?

Reliability

Flexibility

Automation

1

2

3

Reliability

Flexibility

Automation

1

2

3

re · li · abil · i · ty (noun) \ri-ˌlī-ə-ˈbi-lə-tē\1. Through resiliency 2. Magic of the replica set

Replica Set

Pets?

No Pets!

Arbiters

Extra Votes

Spanning AZs

Spanning DCs

Non-voting

Hidden

Client Logic

More!

Planning for failure.

Reliability

Flexibility

Automation

1

2

3

When you change a schema, everythi ng breaks !

DATA MODEL FLEXIBILITY

DATA MODELWORKLOAD FLEXIBILITY

270kapps hosted on Parse

270kapps hosted on Parse

270kdifferent workloads

=

270kapps hosted on Parse

270kdifferent workloads

=Every app must be individually performant!

DEPLOYMENT

BACKUPS

RECOVERY

DISASTER RECOVERY

DRIVERS

PARTIAL FAILURES

PERFORMANCE TUNING

INDEXING PLAN

DESIGN FAILOVER PLAN

OPERATIONAL EXPERTISE

ONEre-usable solution

Reliability

Flexibility

Automation

1

2

3

Replica Set

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

VPC

+

PLEASE. SHARE YOUR TOOLS.

MongoDB Chef Cookbook github.com/ParsePlatform/Ops/tree/master/chef/cookbooks/mongodb

AWS Chef Cookbook (ec2 raid recipe) github.com/opscode-cookbooks/aws

Continuous Compaction blog.parse.com/2013/03/26/always-be-compacting/

Warmup Scripts blog.parse.com/2013/03/07/techniques-for-warming-up-mongodb/

Mongo Proxy url url url url -- naitik shah (& blog post url)

NEW

Flashback url url url url -- kai liu (& blog post url)

NEW

a love story[*]. The End

* mutual loathing, bitterness, mistrust. Love the one you’re with.

MongoDB + Ops:

“Which database do I hate the most?”

“Which database do I hate the most?”

“The one I’m using right now”

Charity Majors @mipsytipsy