Date post: | 25-May-2015 |
Category: |
Technology |
Upload: | crcsmnky |
View: | 996 times |
Download: | 4 times |
MongoDBBackups & Disaster Recovery
Sandeep Parikh, Solutions Architect
2
Why Should You Care?
Curious about
backups and disaster recovery
Proactively planning for
your own deployment
3
• Recovery vs. Availability
• Review backup and restore tools– What is the relative complexity of each option?
• Understand high availability– What are you trying to protect against?
Goals
Availability vs. Recovery
Disasters do happen
Sometimes they are our fault
7
• Don’t confuse the two
• Distinctly different business requirements
• Technical solutions may converge
Availability vs. Recovery
8
• Availability– Data is readable/writable in the face of infrastructure
failures– Tunable resiliency depending upon failure scenarios
• Recovery– Data is safe, nothing is lost– Returning from failure is a straightforward task
Definitions
9
• How much data can you afford to lose?
• How long can you afford to be off-line?
• Cost is impacted by these decisions.
Considerations
Recovery
What’s the most important thing about creating backups?
Restoring
If you can’t restore, you aren’t backed up
13
• mongodump/mongorestore
• Storage-level options
• MongoDB Backup Service (new!)
Recovery Approaches
14
mongodump / mongorestore
MongoDB BSON files
mongodump
mongorestore
15
• Can be run in live or offline mode
• Oplog-aware for point-in-time operations
• Filter can be applied in both directions
• http://docs.mongodb.org/manual/reference/program/mongodump/
• http://docs.mongodb.org/manual/reference/program/mongorestore/
mongodump / mongorestore
16
• Copy files in your data directory (e.g. /data/db)
• Filesystem or block storage snapshot
• Considerations– Journaling (on by default*)– Ensuring consistency
Storage-level Backups
17
• Entire database is backed up– Backup files will be large
• Fastest way to create a backup
• Fastest way to restore a backup
• Ongoing management requires devops expertise
Storage-level Considerations
18
MongoDB Management Service
MongoDB
19
• Recovery at scale is challenging– Deployments are growing all the time
• MongoDB needed a simple, scalable approach to secure recovery
• Ongoing management of manual solutions was difficult
• Overhead needed to be minimal
MMS Backup: Background
20
MMS Backup: Features
Available• Cloud-based service• Archived across DCs
Secure• Data is encrypted in-
transit• 2-factor auth for
restores
Managed• Developed and
monitored by MongoDB• Point-in-time backups
Overhead• Lightweight agent,
processes oplog
Restores• Free, unlimited• Seed new environments
Unlimited restoresSeed dev, test, or new environments
22
Recovery Approaches
Complexity
Scalability
mongodump
storage-level
mms backup
Availability
Avoid a single point of failure
25
Replication
26
Failure Tolerance
Node
Network
RackData Center
Global
27
Node Failure
Primary
Secondary
Secondary
28
Rack Failure
Rack 2
Rack 1Primary
Secondary
Secondary
29
Data Center Failure
DC 2
DC 1Primary
Secondary
Secondary
30
• Increased data redundancy
• Replica Set “majority” drives availability
• Deploy across multiple levels– Racks, Regions, Data Centers
• Can support recovery and availability requirements– Recovery: geographically dispersed copies of data– Availability: multi-level failover protection
Resilient Topology
Recovery Examples
32
Stop the balancer (wait)
(or use scheduled window)
Stop one config server (data is still
r/w)
Backup config server
Execute backup across
all shards
Restart config server
Resume balancer
Backup: Sharded Cluster
33
• Goals:– Return to normal operation/configuration, or– Amend configuration parameters?
• Normal operation:– Stop mongod and mongos processes– Restore each shard as a replica set restore– Restore config server data– Restart mongos and mongod
• Amended deployment (advanced!):– Shard key, shard topology, config hostnames
Restore: Sharded Cluster
34
• mongodump/mongorestore– --oplog[Replay]– --objcheck/--repair– --dbpath– --query/--filter
• bsondump– inspect data at console
• lvm snapshot time/space trade-off– Multi EBS backup– clean up snapshots
Other Recovery Tools
Summary
36
• Pick the easiest solution and backup immediately– Then test the restore process
• Interim solutions are ok – data safety is paramount
• Iterate into a long-term scalable approach
• Happieness is at the intersection of availability and recovery
• Manage complexity and scalability
Work Towards a Balanced Solution
Questions?
38
• MMS Backup– https://mms.mongodb.com/backup/
• Replication:– http://www.mongodb.com/presentations/webinar-replicati
on-and-replica-sets-0
• Advanced Replication:– http://www.mongodb.com/presentations/advanced-replicat
ion-features
• Backup Strategies:– http://docs.mongodb.org/manual/core/backups/
Resources