Date post: | 16-Apr-2017 |
Category: |
Engineering |
Upload: | cihan-biyikoglu |
View: | 357 times |
Download: | 4 times |
UNDER THE HOODCOUCHBASE SERVER 4.0 ARCHITECTURECihan BiyikogluDirector of Product Management
©2015 Couchbase Inc. 2
AgendaGoals Give you a full tour of the mansion! Zoom into major components and
services in Couchbase Server 4.0 Impress your date…
©2015 Couchbase Inc. 3
Agenda Overview
Cluster Architecture – 10k ft view Deep Dive
Connectivity Architecture with Client SDKs Replication Architecture with DCP Cluster Management Services
Data Service, Index Service, Query Service Recap Q&A
©2015 Couchbase Inc. 4
DisclaimerCouchbase Server 4.0 and ForestDB are still in development and the final version of the products may not be identical in details discussed on this session.
Overview
©2015 Couchbase Inc. 6
Couchbase Server 4.0 - Cluster Architecture
STORAGE
Couchbase Server 1
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerCluster Manager
Managed CacheStorage
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 2
Managed Cache
Cluster ManagerCluster Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 3
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerCluster Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 4
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerCluster Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 5
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerCluster Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 6
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerCluster Manager
Data Service
Index Service
Query Service
Managed CacheStorage
Managed CacheStorage
Managed CacheStorage
Managed CacheStorage
Managed CacheStorage
©2014 Couchbase Inc.
Couchbase Server 4.0 - Cluster Architecture
STORAGE
Couchbase Server 1
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerCluster Manager
Managed CacheStorage
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 2
Managed Cache
Cluster ManagerCluster Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 3
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerCluster Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 4
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerCluster Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 5
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerCluster Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 6
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerCluster Manager
Data Service
Index Service
Query Service
Managed CacheStorage
Managed CacheStorage
Managed CacheStorage
Managed CacheStorage
Managed CacheStorage
Deep DiveConnectivityReplication
Cluster ManagerData Service
Indexing & Index ServiceQuery Service
©2015 Couchbase Inc. 9
Connectivity and Client LibrariesType Port EndpointREST 8091, 18091 Admin Connections
Pointed at any node in the cluster
REST 8091, 18092 Query with View Load balanced across node of the cluster that runs data service
REST 8093, 18093 Query with N1QL Load balanced across node of the cluster that runs query service
ONLINE 11210, 11207 Core Data OperationsState-full connections from client app to nodes of the cluster that runs data service
©2015 Couchbase Inc. 10
Connectivity and Client LibrariesConnectivity Phases1. Auth2. Discovery
Get cluster map
3. Service ConnectionAuth to
ServiceRun
operationIf
(topology_change) Rerun
#2
10
1 2
…
©2015 Couchbase Inc. 11
Discovery and Cluster Map
©2015 Couchbase Inc. 12
Discovery and Cluster Map
©2015 Couchbase Inc. 13
Discovery and Cluster Map – 2 New Nodes
Deep DiveConnectivityReplication
Cluster ManagerData Service
Indexing & Index ServiceQuery Service
©2015 Couchbase Inc. 15
Database Change Protocol (DCP)Fast Streaming Replication DCP - An open streaming protocol that conveys the consistent database state to all
consumers Ordering (vbucket based seq.number) Re-startable, Resumable (version histories and rollbacks) Consistent (snapshots) High Performance (memory based with dedup)
Master
Local Replic
a
Index
Map/Reduc
eRemot
eReplic
a
IndexMap/Reduc
e
Source Cluster
Cross Data Center Cluster
Hadoop
Client/Applicati
on
NotificationIn future
Integration
Backup/Export
Tooling
Deep DiveConnectivityReplication
Cluster ManagerData Service
Indexing & Index ServiceQuery Service
©2015 Couchbase Inc. 17
Cluster ManagerCluster Manager = Governor of the ClusterManages cluster level operations and coordination among nodes
Cluster Membership & Service Layout Node Status & Failover Data Placement & Rebalance Auth
17
©2015 Couchbase Inc. 18
Cluster ManagerInside Cluster Manager
per-node-&-bucket services
generic distributed facilities
generic local facilities
Logging and Other Services
distributed node discovery
Master Services- cluster level
operations - data placement - rebalancer- auto-failover
Admin Portal – REST API
Global Config (gossip replication)
Local Config Store
Per-node Services - Heartbeats, - Babysitter
Bucket services - dcp init and teardown- stats collectors,
Auth
19©2014 Couchbase Inc.
Adding Nodes to Cluster Online
ACTIVE ACTIVE ACTIVE
REPLICA REPLICA REPLICA
Couchbase Server 1 Couchbase Server 2 Couchbase Server 3
ACTIVE ACTIVE
REPLICA REPLICA
Couchbase Server 4 Couchbase Server 5
SHARD5
SHARD2
SHARD SHARD
SHARD4
SHARD SHARD
SHARD1
SHARD3
SHARD SHARD
SHARD4
SHARD1
SHARD8
SHARD SHARD SHARD
SHARD6
SHARD3
SHARD2
SHARD SHARD SHARD
SHARD7
SHARD9
SHARD5
SHARD SHARD SHARD
SHARD7
SHARD
SHARD6
SHARD
SHARD8
SHARD9
SHARD
READ/WRITE/UPDATE
Cluster Manager receives the new nodes - Node inherit cluster
settings- Move active and replica
vbuckets using DCP- As vbuckets catch up,
Initiate online handoff from “existing node” to “new node”
Clients Receive Topology Change Notification- Trap not_my_vbucket
errors- Refresh cluster map and
retry operation
©2015 Couchbase Inc. 20
Deeper Dive into ArchitectureTHUR @1.00 - Architecture Track
Deep Dive into Cluster Manager in Couchbase Server 4.0
Dave Finlay, Senior Director of Development, Couchbase
Deep DiveConnectivityReplication
Cluster ManagerData Service
Indexing & Index ServiceQuery Service
©2015 Couchbase Inc. 22
Data Service Data Service = GET/SET + Map-Reduce Views*Tackles fast core data operations with efficient caching and disk persistence
Core Database Operations Core GET/SET operations Couchstore Based Storage
Terms:Bucket = database reside within a clustervBucket = hash partition of the database that reside within a node 22
©2015 Couchbase Inc. 23
Data Manager Architecture
…
Database Engine (ep-engine)
Listener
vBucket Manager
Item Pager
Expiry PagerCheckpoint Manager
CachePartition Hash
Tables (Active and
Replica)
Partition Hash Tables
(Active and Replica)
Partition Hash Tables
(Active and Replica)
AuthNetwork IO
Flusher
Scheduler
Reader IO
Writer IO
Non IO
Batch Reader
©2014 Couchbase Inc.
APPLICATION SERVER
MANAGED CACHE
DISK
DISK
Data Manager - Update Operation
24
DOC 1
DOC 1
DOC 1
DOC 1DOC 1
Caching based on Memcached: App gets an ACK when update is successfully in RAM Or RAM+Replicated Or RAM+Persisted Or RAM+Replicated+Persisted Or …
DCP base Replication: updates queued to other nodes
Couchstore based Storage: updates queued for storage
DCP
25©2014 Couchbase Inc.
Data Manager - Cache EjectionAPPLICATION SERVER
MANAGED CACHE
DISK
DISKDOC 1
DOC 2DOC 3DOC 4DOC 5
DOC 1
DOC 2 DOC 3 DOC 4 DOC 5
DCP
26©2014 Couchbase Inc.
APPLICATION SERVER
MANAGED CACHE
DISK
DISK
DCP
DOC 1
Data Manager - Cache Miss
DOC 2 DOC 3 DOC 4 DOC 5
DOC 2 DOC 3 DOC 4 DOC 5
GETDOC 1
DOC 1
DOC 1
©2015 Couchbase Inc. 27
Deeper Dive into ArchitectureWED @4.30 - Architecture Track
Next Generation Storage Engine: ForestDB
Chiyoung Seo, Software Engineer, Couchbase
©2015 Couchbase Inc. 28
Deeper Dive into ArchitectureTHUR @2.30 - Architecture Track
Under the Hood ForestDB: Performance on SSDs and File
Systems
Sundar Sridharan, Senior Developer, Couchbase Prof. Sang-Won Lee, Sungkyunkwan University,
Suwon, Korea
Deep DiveConnectivityReplication
Cluster ManagerData Service
Indexing & Index ServiceQuery Service
©2015 Couchbase Inc. 30
Indexing Indexers
Views: Incremental Map/Reduce with customer JavaScript for complex indexing logic for online reporting and analytics
Spatial Views: Incremental/Reduce with R-tree Indexes for Bounding-Box queries
GSI (Global Secondary Indexes): Efficient indexes for secondary lookups and ad-hoc query processing
©2015 Couchbase Inc. 31
Indexing with ViewsIncremental Map/Reduce ViewsQueries with incremental map/reduce processing that can execute custom JS
Flexible Indexer Indexer & Query Processor Couchstore Based Storage
31
©2015 Couchbase Inc. 32
Views
32
APPLICATION SERVER
VIEWINDEXER
Query Set
1. SET operation• In Memory SET: queued for
DCP Replication• View Indexer: Executes
incremental map/reduce on a batch of updates
• Couchstore based Storage: updates queued for storage
2. View Query Engine: REST Based queries with filters, limit and more executed with scatter-gather
©2015 Couchbase Inc. 33
Index Service Global Secondary Indexes (NEW in 4.0)Tackles indexer for fast query execution with efficient index maintenance for N1QL Queries
High Performance Indexing Projector and Router : Coordinate and communicate efficient
index change notifications between data service and index service.
Supervisor – Indexer and scannerIndexer : Maintain large number of indexes as change
notifications arriveScanner: Respond to Query Service index-scan requests with
rich set of consistency dials Index Storage &Caching
ForestDB: Brand new storage engine for high performance index caching and storage
33
©2015 Couchbase Inc. 34
Data Service
Projector & Router
Indexing Service
Query ServiceIndex Service
SupervisorIndex maintenance &
Scan coordinator
Index#2
Index#1
Query Processorcbq-engine
Bucket#1
Bucket#2
DCP Stream Index#4Index#3
...Bucket#2
Bucket#1
Projector and Router: 1 Projector and Router per node1 stream of changes per buckets per supervisor
ForestDBStorage Engine Supervisor
1 Supervisor per nodeMany indexes per Supervisor
©2015 Couchbase Inc. 35
Deeper Dive into ArchitectureWED @1.45 - Architecture Track
Global Secondary Indexing in Couchbase Server 4.0
Cihan Biyikoglu, Director of Product Management, Couchbase
©2015 Couchbase Inc. 36
Deeper Dive into ArchitectureWED @3.45 - Architecture Track
Deep Dive into Global Secondary Indexing Architecture in Couchbase
Server 4.0
John Liang, Architect, Couchbase
Deep DiveConnectivityReplication
Cluster ManagerData Service
Indexing & Index ServiceQuery Service
©2015 Couchbase Inc. 38
Query Service Query Service = N1QLTackles N1QL Query execution
Query Execution N1QL Parser & Optimizer: tokenize N1QL statement, and
generate an execution plan based utilizing indexes Query Execution Engine: Assigns resources to query and
coordinates query execution. Data Sources: Pluggable “data source driver” layer for
accessing data sources in Couchbase Server (data and index service) and other external data provides
38
©2015 Couchbase Inc. 39
Query Service N1QL Query Processing
Query Engine
Query Processor
Listeners
Parser Optimizer
Data Stores
Execution Engine
Couchbase Server
Auth DataIndexersGSI View
s
Others…
8093/18903
File systemData Service
Index Service
......
Cluster Manager
Bucket#2
Bucket#2
Index#2
Index#1
©2015 Couchbase Inc. 40
Deeper Dive into ArchitectureWED @2.30 - Architecture Track
Deep Dive into N1QL Internals in Couchbase Server 4.0
Keshav Murthy, Director of Development, Couchbase
©2015 Couchbase Inc. 41
RecapCouchbase Server decouples competing workloads into Services
Each Service tunes itself with caching and storage services
Each Service can be independently scale
Get Started Today Couchbase Server 4.0 & N1QL
Couchbase.com/beta
Thank you.