Manuel Hurtado. Couchbase paradigma4oct

Introducción a Couchbase

Manuel HurtadoSolutions Engineer

What is Couchbase?

©2015 Couchbase Inc. 3

Couchbase Server Defined

§ Couchbase Server 4.X is the first NoSQL Database to enable you to develop with agility and operate at any scale.

Managed Cache Key-Value Store Document Database Embedded Database Sync Management


The Power of the Flexible JSON Schema

Ability to store data in multiple ways• Denormalized single document, as opposed to normalizing data across multiple table• Dynamic Schema to add new values when needed


Document Fundamentals -‐ Composition

§ Similar to primary keys in relational databases§ Documents are partitioned based on the

document ID§ ID based document lookup is extremely fast § Must be unique

§ JSON§ Binary -‐ integers, strings, booleans§ Common binary values include serialized objects,

compressed XML, compressed text, encrypted values

Document ID or Key

Value

§ CAS Value (unique identifier for concurrency)§ TTL§ Flags (optional client library metadata)§ Revision #

Metadata

Document

Document ID / Key

MetadataValue


Storing and Retrieving Documents


Couchbase Data Access

• Everything is layered on top of Key Value

• A Document store is a special case of Key-Value

• Views provide aggregation and real-time analytics through incremental map-reduce

• Global Secondary Indexes provide low latency/high throughput indexes

• N1QL is a language that provides a powerful and expressive way of accessing documents


Choose how to build your apps, and where to run themCouchbase supports a wide range of frameworks, languages, platforms, and infrastructure choices


Docker Support

§ Deployment Scenarios§ Single host, single container§ Single host multiple containers§ Multiple hosts, single container on each host§ Multiple hosts, multiple containers per host

§ https://hub.docker.com/_/couchbase/


Top Use Cases

Profile Management Personalization 360o Customer View Internet of Things

Content Management Catalog Real Time Big Data Digital Communication

Mobile Applications

High Availability Caching


World of Operational Databases

11

Graph Stores

Neo4j

RDBMS/Transactional Stores

Oracle, DB2, SQLServer, MySql, Postgres

Caching Stores

Coherence, Redis, Gemfire

Full-‐text Stores

Lucene, SOLR, ElasticSearch

Key/Value Stores

Hbase,Cassandra,

Riak, Dynamo

Document Stores

Marklogic, mongoDB, Cloudant, DocumentDB

Couchbase

Couchbase ArchitectureSingle Node

12


Couchbase Architecture

• Data Service – builds and maintains Distributed secondary indexes (MapReduce Views)

• Indexing Engine – builds and maintains Global Secondary Indexes

• Query Engine – plans, coordinates, and executes queries against either Global or Distributed indexes

• Cluster Manager – configuration, heartbeat, statistics, RESTful Management interface


Couchbase Read Operation

APPLICATION SERVER

MANAGED CACHE

DISK

DISKQUEUE

REPLICATIONQUEUE

DOC 1

GETDOC 1

DOC 1


APPLICATION SERVER

MANAGED CACHE

DISK

DISKQUEUE

REPLICATIONQUEUE

Write Operation

DOC 1

DOC 1DOC 1


Auto sharding – Bucket and vBuckets

vB

Data buckets

vB

1 ….. 1024

Virtual buckets

§ A bucket is a logical, unique key space§ Multiple buckets can exist within a single cluster of nodes

§ Each bucket has active and replica data sets (1, 2 or 3 extra copies)§ Each data set has 1024 Virtual Buckets (vBuckets)§ Each vBucket contains 1/1024th portion of the data set§ vBuckets do not have a fixed physical server location

§ Mapping between the vBuckets and physical servers is called the cluster map

§ Document IDs (keys) always get hashed to the same vbucket§ Couchbase SDK’s lookup the vbucket -‐> server mapping


Cluster Map

Couchbase Cluster

CLUSTER MAP

Hashing Algorithm

CRC32

vBucket2

vBucket3

vBucket4

vBucket5

vBucket6

vBucket7

vBucket1024

vBucket1

...

Couchbase SDK


Cluster Map

Couchbase Cluster

CLUSTER MAP

Hashing Algorithm

CRC32

vBucket2

vBucket3

vBucket4

vBucket5

vBucket6

vBucket7

vBucket1024

vBucket1

...

Couchbase SDK


Cluster Map – 2 nodes added

Couchbase Cluster

CLUSTER MAP

Hashing Algorithm

CRC32

vBucket2

vBucket3

vBucket4

vBucket5

vBucket6

vBucket7

vBucket1024

vBucket1

...

Couchbase SDK


ACTIVE ACTIVE ACTIVE

REPLICA REPLICA REPLICA

Couchbase Server 1 Couchbase Server 2 Couchbase Server 3

Basic Operation

SHARD5

SHARD2

SHARD9

SHARD SHARD SHARD

SHARD4

SHARD7

SHARD8

SHARD SHARD SHARD

SHARD1

SHARD3

SHARD6

SHARD SHARD SHARD

SHARD4

SHARD1

SHARD8

SHARD SHARD SHARD

SHARD6

SHARD3

SHARD2

SHARD SHARD SHARD

SHARD7

SHARD9

SHARD5

SHARD SHARD SHARD


Add Nodes to Cluster




ACTIVE ACTIVE

REPLICA REPLICA

Couchbase Server 4 Couchbase Server 5

SHARD5

SHARD2

SHARD SHARD

SHARD4

SHARD SHARD

SHARD1

SHARD3

SHARD SHARD

SHARD4

SHARD1

SHARD8

SHARD SHARD SHARD

SHARD6

SHARD3

SHARD2

SHARD SHARD SHARD

SHARD7

SHARD9

SHARD5

SHARD SHARD SHARD

SHARD7

SHARD

SHARD6

SHARD

SHARD8

SHARD9

SHARD

READ/WRITE/UPDATE


Fail Over Node




ACTIVE ACTIVE

REPLICA REPLICA

Couchbase Server 4 Couchbase Server 5

SHARD5

SHARD2

SHARD SHARD

SHARD4

SHARD SHARD

SHARD1

SHARD3

SHARD SHARD

SHARD4

SHARD1

SHARD8

SHARD SHARD

SHARDSHARD6

SHARD2

SHARD SHARD SHARD

SHARD7

SHARD9

SHARD5

SHARD SHARD

SHARD

SHARD7

SHARD

SHARD6

SHARDSHARD8

SHARD9

SHARD

SHARD3

SHARD1

SHARD3

SHARD


Modern Architecture – Multi-Dimensional Scaling

MDS is the architecture that enables independent scaling of data, query, and indexing workloads while being managed as

one cluster.






Demo: Install Couchbase

§ Install node 1§ Add node 2§ Rebalance§ Admin UI

beer-‐sample travel-‐sample

node 1192.168.56.101

node 2192.168.56.102

Cluster Madrid prod

Cross Datacenter Replication (XDCR)

27


XDCR: Cross Data Center Replication

§ Application can access both clusters (master – master)§ Scales out linearly§ Within a cluster operations are strongly consistent and you can read-‐your-‐own-‐write. Between clusters mutations are eventually consistent.


Memory-‐to-‐memory, topology aware replication

New York

San Francisco

SF Server Cluster

Couchbase Server 1

MEMORY DISK

Couchbase Server 2

MEMORY DISK

Couchbase Server 3

MEMORY DISK

NYC Server Cluster

Couchbase Server 1

MEMORY DISK

Couchbase Server 2

MEMORY DISK

Couchbase Server 3

MEMORY DISK

Couchbase Server 4

MEMORY DISK

USE CASES

§ Access by Localization§ Active-‐Active§ Active-‐Passive§ Disaster Recovery§ BI environment§ Test environment


Demo: XDCR: Replicate data between clusters

§ Install node 3 / Cluster backup§ Enable XDCR between clusters§ Replicate airlines

Developing an Application

31


Languages and Interfaces for Couchbase

§ Official SDKs– Java– .NET– Node.js– Python

§ For each of these we have– Full Document support– Interoperability– Common Programming Model

Others: Erlang, Perl, TCL, Clojure, Scala

– PHP– C– Go– Ruby

JDBC and ODBC


Interacting with Data

View

s

N1Q

LQuery

Key Value Store

JSON Documents

Document Modeling


Data Modeling

§ An ecommerce order in relational world


Data Modeling

§ Embed vs Refer


Data Modeling

§ Embed vs Refer


Reference Documents for Lookups

38

§ Lookup documents which reference the primary one§ PROS§ High Performance§ Extremely scalable§ Immediately consistent

§ CONS§ Needs to be maintained by the application

§ GSI is a viable alternative§ PROS§ No need for Referential integrity§ Good performance

§ CONS§ Not immediately consistent -‐ but can optional wait

for index updates (READ YOUR OWN WRITES)


Compound keys: example

u::1001

{

"name": "Matthew Revell",

"email": "[email protected]",

"address": "11-21 Paul Street",

"city": "London",

"postCode": "EC2A 4JU",

"telephone": "44-20-3837-9130",

"orders": [ 1, 9, 698, 32 ],

“productsViewed”: [8, 33, 99, 100]

}



u::1001

{




"city": "London",


"telephone": "44-20-3837-9130",

"orders": [ 1, 9, 698, 32 ]

}

u::1001::productsviewed

{"productsList": [

8, 33, 99, 100]

}



u::1001

{




"city": "London",


"telephone": "44-20-3837-9130",

"orders": [ 1, 9, 698, 32 ]

}


{"productsList": [

8, 33, 99, 100]

}

p::8

{

id": 1,"name": "T-shirt","description": "Red Couchbase shirt","quantityInStock": 99,"image": "tshirt.jpg”

}



u::1001

{




"city": "London",


"telephone": "44-20-3837-9130",

"orders": [ 1, 9, 698, 32 ]

}


{"productsList": [

8, 33, 99, 100]

}

p::8

{

id": 1,"name": "T-shirt","description": "Red Couchbase shirt","quantityInStock": 99

}

p::8::img

“http://someurl.com/tshirt.jpg”


Demo: Code examples: CRUD

Sample code for:

§ Create § Read§ Update§ Delete

Couchbase Query (N1QL)N1QL Introduction


Why SQL for JSON?

JSON Documents Provide

Rich Structure(No assembly)

Structure Evolution(Flexible schema, easy to change)

SQL Provides

General query functionality

Query across relationships

Why N1QL?

Most developers already know SQL

No need for complex query frameworks


SELECT

§ Standard SELECT pipeline

– SELECT, FROM, WHERE, GROUP BY, ORDER BY, LIMIT, OFFSET

§ Queries across relationships

– JOINs

– Subqueries

– NEST — a JOIN that embeds child objects within their parent

– UNNEST — a JOIN that surfaces nested objects as top-‐level data

§ Aggregation

§ Set operators

– UNION, INTERSECT, EXCEPT


Indexing

§ CREATE / DROP INDEX

§ Two types of indexes– View indexes– GSI indexes (global secondary indexes)

§ Can index any data expression– Nested / complex expressions– Computed expressions

§ EXPLAIN


Data writes

§ UPDATE … WHERE …– Partial updates; deep updates

§ DELETE … WHERE …– Deeply nested conditions

§ INSERT … VALUES …; INSERT … SELECT …– Bulk insert; transfer and transformation

§ MERGE– INSERT or UPDATE; ETL support


Document-‐oriented Extensions

§ Nested

– Paths—user.profile.email, children[0], children[0:2]

– NEST, UNNEST

– Ranging—EVERY child.age > 5 FOR child IN children END

– Transformations—SELECT {“name”: first_name || last_name}

§ Non-‐uniform

– IS MISSING

– Type checking and conversion

§ Distributed– Direct lookup—USE KEYS

– Efficient joins—ON KEYS


Expressions

©2014 Couchbase, Inc. 50

Expressions

Literalsq Primitives [ 0, ‘hello’, TRUE ]

q NULL

Operators

q Arithmetic [ +, -‐, *, /, % ]

q Logical [ AND, OR, NOT ]

q Comparison [ <, <=, =, !=, >=, >, BETWEEN, IS NULL ]

q Pattern matching [ LIKE ]

q Conditional [ CASE ]

Scalar functions

q Numeric [ trigonometric, ROUND, TRUNC, … ]

q String [ UPPER, LOWER, TRIM, SUBSTR, … ]

q Date [ string and numeric dates, NOW, date arithmetic, … ]

Aggregate functions q MIN, MAX, SUM, AVG, COUNT [ DISTINCT ]

Subqueries q Subqueries are full expressions


Document-‐Oriented Expression Features

©2014 Couchbase, Inc. 51

Features

Ranging over collectionsWHERE ANY c IN children SATISFIES c.age > 10 END

WHERE EVERY r IN ratings SATISFIES r > 3 END

Mapping with filtering ARRAY c.name FOR c IN children WHEN c.age > 10 END

Deep traversal and updateWHEREANY node WITHIN request SATISFIES node.type = “xyz” END

UPDATE doc SET c.name = Title(c.name) FOR c WITHIN doc END

Dynamic ConstructionSELECT { “a”: expr1, “b”: expr2 } AS obj1, name FROM … // Dynamic object

SELECT [ a, b ] FROM … // Dynamic array

Nested sourcing SELECT * FROM a.b.c … // Select from nested objects


Demo: N1QL samples

N1QL Queries from

§ Web UI§ Command line cbq§ REST

Summary

53


Develop with Agility

With Couchbase Server, you can Develop Applications with Agility. Here’s why:

Easier, Faster Development Flexible Data Modeling Powerful Querying

SQL Integration & Migration Big Data Integration Mobile / IoT


Operate at Any Scale

With Couchbase Server, you can deploy, manage, and scale apps without limits. Here’s why:

Elastic Scalability Consistent High Performance

Always-on Availability

Multi-Data Center Deployment Simple, Powerful Administration

Enterprise Grade Security

Q&A

56


Additional Resources

57

§ Couchbase Labs: https://github.com/couchbaselabs

§ Sample Applications:§ https://github.com/couchbaselabs?utf8=%E2%9C%93&query=try§ https://github.com/couchbaselabs?utf8=%E2%9C%93&query=beer

§ General Docs: http://docs.couchbase.com

§ Query Portal: http://query.couchbase.com

§ Blog: http://blog.couchbase.com

§ Forum: http://forums.couchbase.com


beer-‐sample travel-‐sample

node 1192.168.56.101

node 2192.168.56.102

Cluster Madrid prod

airlines

node 3192.168.56.1013

Cluster Backup

XDCR

airlines


airlines

node 3192.168.56.1013

Cluster Backup

Date post:	21-Jan-2017
Category:	Technology
Upload:	paradigma-digital
View:	42 times
Download:	0 times

Manuel Hurtado. Couchbase paradigma4oct

Technology