Date post: | 20-Aug-2015 |
Category: |
Technology |
Upload: | couchbase |
View: | 988 times |
Download: | 2 times |
Couchbase Server and Cross Datacenter Replication
Anil KumarTechnical Product Manager
XDCR: Cross Datacenter Replication
US DATA CENTER
EUROPE DATA CENTER
ASIA DATA CENTER
http://blog.groosy.com/wp-content/uploads/2011/10/internet-map.jpg
Cross Datacenter Replication – The basics
• Replicate your Couchbase data across clusters
• Clusters may be spread across geos
• Configured on a per-bucket basis
• Supports unidirectional and bidirectional operation
• Application can read and write from both clusters (active – active replication)
• Replication throughput scales out linearly
• Different from intra-cluster replication
Intra-cluster Replication
Cross Datacenter Replication (XDCR)
XDCR Architecture
33 2
Single node - Couchbase Write Operation with XDCR
Managed Cache
Dis
k Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc 1Doc 1
Doc 1
To other node
XDCR Engine
Doc 1
To other cluster
Internal Data Flow
1. Document written to managed cache
2. Document added to intra-cluster replication queue
3. Document added to disk queue
4. XDCR push replicates to other clusters
XDCR in actionCOUCHBASE SERVER CLUSTERNYC DATA CENTERACTIVE
Doc
Doc 2
SERVER 1
Doc 9
SERVER 2 SERVER 3
RAM
Doc Doc Doc
ACTIVE
Doc
Doc
Doc RAM
ACTIVE
Doc
Doc
DocRAM
DISK
Doc Doc Doc
DISK
Doc Doc Doc
DISK
COUCHBASE SERVER CLUSTERSF DATA CENTER
ACTIVE
Doc
Doc 2
SERVER 1
Doc 9
SERVER 2 SERVER 3
RAM
Doc Doc Doc
ACTIVE
Doc
Doc
Doc RAM
ACTIVE
Doc
Doc
DocRAM
DISK
Doc Doc Doc
DISK
Doc Doc Doc
DISK
Bucket-level XDCR
Bucket A
Bucket B
Bucket C
Cluster 1
Bucket A
Bucket B
Bucket C
Cluster 2
Continuous Reliable Replication
• All data mutations replicated to destination cluster
• Multiple streams round-robin across vBuckets in parallel (32 default)
• Automatic resume after network disruption
Cluster Topology Aware
• Automatically handles node addition and removal in source and destination clusters
Efficient• Couchbase Server de-duplicates writes to disk
With multiple updates to the same document only the last version is written to disk
Only this last change written to disk is passed to XDCR
• Document revisions are compared between clusters prior to transfer
Active-Active Conflict Resolution
• Couchbase Server provides strong consistency at the document level within a cluster
• XDCR provides eventual consistency across clusters
• If a document is mutated on both clusters, both clusters will pick the same “winner”
• In case of conflict, document with the most updates will be considered the “winner”
{ … } 33{ … }
Doc 1 on DC1Doc 1 on DC2
Winner
Configuration and Monitoring
STEP 1: Define Remote Cluster
STEP 2: Start Replication
Monitor Ongoing Replications
Detailed Replication Progress• Source Cluster
• Destination Cluster
XDCR Topologies and Use Case
Unidirectional
• Hot spare / Disaster Recovery
• Development/Testing copies
Bidirectional
• Multiple Active Masters
• Disaster Recovery
• Datacenter Locality
Chain
Data aggregation
Data propagation
XDCR in the Cloud
• Server Naming Optimal configuration using DNS name that resolves to internal address
for intra-cluster communication and public address for inter-cluster communication
• Security XDCR traffic is not encrypted, plan topology accordingly Consider 3rd party Amazon VPN solutions
Impact of XDCR on the cluster
Your clusters need to be sized for XDCR
• XDCR is CPU intensive Configure the number of parallel streams based on your CPU capacity
• You are doubling your I/O usage I/O capacity needs to be sized correctly
• You will need more memory particularly for bidirectional XDCR Memory capacity needs to be sized correctly
Q & A
Coming September 13
Featured speakers include:
http://www.couchbase.com/couchbase-sf-2013
Thank you
Get Couchbase Server from http://www.couchbase.com/download