Relational to NoSQL: Getting started from SQL Server

transcript

Relational to NoSQL:Getting started from SQL Server

Shane JohnsonSr. Product Marketing Manager

Couchbase

Today’s agenda

§ Why NoSQL?§ Identifying the right application§ Modeling your data§ Accessing your data§ Installing and scaling your database§ Monitoring and managing your deployment§ Q & A

About the speakers – Shane Johnson

Shane JohnsonSenior Product Marketing ManagerCouchbase (since Dec 2013)

Experience:- Proud Red Hatter

- Consulting (Architect)- Transitioned to Marketing (Technical Manager)- Passionate about Open Source

- Expertise- Java and Distributed Systems

What’s Couchbase?

Couchbase is the company behind Couchbase Server & Couchbase Mobile

• Open source JSON database• Founded 2010• 500+ enterprise customers globally

Some of our customers:

Couchbase Server can be deployed as:

Document database Key-value store Distributed cache

What is NoSQL?

§ No SQL? No.§ Not only SQL? Not really.§ Non relational? Yes.

§ Distributed (most)– Scaled out, not up• Elasticity and commodity hardware

– Partitioned and replicated• Scalability, performance, availability

§ Schema-less (most)– Flexible model– JSON (some)

Who is using NoSQL?

Enterprises are adopting NoSQL for mission critical applications

Media & Publishing eCommerce Hospitality

Who is using NoSQL?

§ Gannett, publisher of USA Today and 90+ media properties, replaced relational database technology with NoSQL to power its digital publishing platform.

§ Marriott deployed NoSQL to modernize its hotel reservation system that supports $38 billion in annual bookings.

§ FHLBank Topeka leverages NoSQL on top of SQL Server to speed up access to customer financial data for its 770 member banks

§ Cars.com, with over 30 million visits per month, replaced SQL Server with NoSQL to store customer and vehicle data

§ Vente-privee.com improved the customer experience of its 18 million members by replacing SQL Server with NoSQL for better performance

Why are they using NoSQL?

Technology Drivers

§ Customers are going online§ The Internet is connecting everything§ Big Data is getting bigger§ Applications are moving to the cloud§ The world has gone mobile

Technical Needs

§ Develop with agility– Flexibility + Simplicity– Easier + Faster

§ Operate at any scale– Elasticity + Availability– Performance at scale– Always-on, global deployment

Business Needs

§ Innovate and compete– Faster time to market– Reduced costs (operational + hardware)– Increased revenue

Why migrate from SQL Server?

§ Easier to scale3 nodes to 100s, 1 data center to many, commodity hardware

§ Better performanceintegrated caching, memory-optimized indexes, memory-based replication

§ Up to 40x lower costopen source, subscription-based, per instance (not per core)

§ Cross-platformruns on Windows or Linux (Red Hat, Ubuntu, Debian, etc)

§ Greater agilityJSON-based data model, SQL-based query language

How do you get started?

§ Identify the right application§ Model your data§ Access your data§ Install and scale your database§ Monitor and manage your deployment

Identifying the right application

Have one or more of the following characteristics or requirements:

Iterate fasterSend and receive JSONProvide low latency at any throughputSupport many concurrent users

Supports users anywhere and everywhereBe available 24x7Store terabytes of dataRead and write to multiple data centers

Service

Service Service

ApplicationExamples:

§ High performance, high availability caching service§ Small, independent application with a narrow scope§ Logical or physical service within a large application§ Global service that powers multiple applications

Concepts: Terminology

Relational (SQL Server) NoSQL (Couchbase)

Failover Cluster Cluster

Availability Group Cluster

Database Bucket

Table Bucket

Row (Tuple) Document (JSON)

Primary Key Object ID

IDENTITY Counter

Indexed View View

SQL N1QL

Model your data

Modeling your data: Fixed vs. self-describing schema

Modeling your data: The flexibility of JSON

Same document type,Different fields

• Different types• Optional• On-demand

Tip: Add a version field to track changes.

{“docType”: “user”, “docVersion”: “1”, …}{“docType”: “user”, “docVersion”: “2”, …}

Modeling your data: Changing the data model

Relational database

• Modify the database schema• Modify the application code (e.g. Java)• Modify the interface (e.g. HTML5/JS)

Document database

• Modify the interface (e.g. HTML5/JS)

Modeling your data: Object IDs

Best Practices

• Natural Keys• Human Readable• Deterministic• Semantic

Examples

• author::shane• author::shane::blogs• blog::nosql_fueled_hadoop• blog::nosql_fueled_hadoop::comments

What about identity columns?

1. Document<Long> nextAuthorIdDoc= bucket.counter(“authorIdCounter”, 1);2. Long nextAuthorId = nextAuthorIdDoc.content();3. String authDocId = “author::” + nextAuthorId; // author::101

Tip: Increment the counter by 10, 20, etc. instead of doing it for every insert.

Modeling your data: Relationships

Author

Blog (FK)Blog (FK)

Comment (FK) Comment (FK)

Author (FK x2)

BlogBlog (FK x2)

Comment Comment

Bottom up Top down

Modeling your data: Relationships

Modeling your data: Strategies and best practices

If… Then…

Relationship is one-to-one or one-to-many Store related data as nested objects

Relationship is many-to-one or many-to-many Store related data as separate documents

Data reads are mostly parent fields Store children as separate documents

Data reads are mostly parent + child fields Store children as nested objects

Data writes are mostly parent or child (not both) Store children as separate documents

Data writes are mostly parent and child (both) Store children as nested objects

Modeling your data: Strategies and best practices

§ Are they a lot of concurrent writes, continuous updates?§ Store children as separate documents

Blog§ Thread

§ Comment§ Comment

§ Thread§ Comment§ Comment

{“docType”: “blog”,

“author”: “author::shane”,“title”: “Couchbase Wins”,“threads”: [

“blog::couchbase_wins::threads::001”,“blog::couchbase_wins::threads::002”

Thread

{“docType”: “thread”,

“comments”: [{

“visitor”: “Brendan Bond”,

“text”: “This blog is amazing!”“replies”: [

“user”: “Dustin Johnson”,“text”: “No, it is not.”

Access your data

Accessing your data: Options

Key-Value(CRUD)

N1QL(Query)

Views(Query)

Documents

Indexes MapReduce

Full Text(Search)

Geospatial(Search)

We’ll focus on these three for now.

Indexes MapReduce

Accessing your data: Connecting to the database

§ Access data via topology-aware smart clients§ Maintains an up-to-date cluster map§ Communicates directly with database nodes – no proxies, no routers, etc.

§ Available for Java, Node.js, PHP, .NET, Python, C, Go, and more§ With standard, certified JDBC/ODBC drivers (if you want to)

Accessing your data: Domain objects vs. document objects

* JSON serialization via Boon.

Working with document objects requires less code, provides more flexibility.

Accessing your data: Key-value operations – referenced data

Accessing your data: Key-value operations – nested data

Accessing your data: Subdocument operations

Accessing your data – N1QL queries: Capabilities

Feature SQL N1QL

JOIN ✔ ✔

TRANSFORM ✔ ✔

FILTER ✔ ✔

AGGREGATE ✔ ✔

SORT ✔ ✔

SUBQUERIES ✔ ✔

PAGINATION ✔ ✔

OPERATORS ✔ ✔

FUNCTIONS ✔ ✔

Accessing your data: N1QL queries – referenced data

Accessing your data: N1QL queries – nested data

Accessing your data: N1QL queries – CRUD

Accessing your data: LINQ

_context.Query<Users>().Where(u => u.status == “Platinum”)

from user in _context.Query<Users>()join account in _context.Query<Account>()on user.accountId equals N1QlFunctions.Key(account) into userGroupfrom accounts in userGroup.DefaultIfEmpty()where (account.type == “Visa” || account.type== “MasterCard”)select new { firstName = user.firstName, lastName = user.LastName };

from user in _context.Query<Users>()join address in _context.Query<Address>()on user.addresses.shipping.addressId equals N1QlFunctions.Key(address) into addressGroupfrom address in addressGroup.DefaultIfEmpty()where address.state == “CA”select new { firstName = user.firstName, lastName = user.LastName };

Accessing your data: N1QL queries – indexes

Simple

Compound

Functional

Partial

Accessing your data: Views

What if indexed views worked great with write intensive workloads?

And you could use COUNT, ORDER BY, and everything else…

Accessing your data: Views

COUNT ?!?

Accessing your data: Views – Incremental MapReduce

Accessing your data: Views – queries

Accessing your data: Strategies and best practices

Concept Strategies & Best Practices

Key Value Operations provide the best possible performance

• Create an effective key naming strategy• Create an optimized data model

Incremental MapReduce (Views) are well suited to aggregation

• Ideal for large data sets, the entire data set• Can be used to create complex secondary

indexes

N1QL queries provide the most flexibility –everything else

• Query data regardless of how it is modeled• Remember to create indexes, leverage

covering indexes where possible

Install and scaling your database

Installing and scaling your database

Admin screenshot – add node

1. Download2. Install3. Configure4. Cluster / Scale5. Rebalance

Installing and scaling your Couchbase database: XDCR

§ Multiple Data Centers§ Cluster per data center§ Replicate between clusters§ Unidirectional / bidirectional§ Master / Master§ Local reads and writes§ Ring§ Hub-and-spoke§ Mesh§ Combination§ Built-in

Managing and monitoring your deployment

§ Configuration– Authentication and authorization– Auditing

§ Monitoring– View and collect log information

§ Tooling– cbbackup & cbrestore– cbcollect_info & cbdstats– cbq– cbtransfer– and more…

§ Management, monitoring, and configuration – Cluster / node / bucket / views– Cross data center replication (XDCR)– Database performance, network

utilization, resource utilization

§ Tasks– Add and remove nodes– Failover nodes– Rebalance cluster

* Web, REST API, and CLI

Where to go next?

Conduct a Successful Proof of Concept

1. Select a use case and application2. Define the success criteria3. Understand the data4. Identify the access patterns5. Review the architecture

Measure your Return on Investment

§ Greater agility?§ Faster time to market?§ Easier scalability?§ Better performance?§ Better availability?§ Lower costs?

You’re just in time

Couchbase Server 4.5 Beta

Easier, more efficient queryingIntegrated query editorIndexed joinsIntegrated full-text searchReady your own writes

Simpler, more advanced data accessPartial updates, reads, etc.

www.couchbase.com/next

Faster, more powerful indexingMemory-optimized indexesArray indexingCircular reuse

Better, more comprehensive adminQuery monitoringFaster backup and restoreX.509 certificatesRole-based access control

Query Workbench: ad-hoc queries

Query Workbench: schema browser

Questions?

WANT TO LEARN MORE?

Getting Started guide:http://www.couchbase.com/get-started-developing-nosql

Download Couchbase software:http://www.couchbase.com/nosql-databases/downloads

Free Online Traininghttp://training.couchbase.com/online

“How to Get Started” White Paperhttp://www.couchbase.com/binaries/content/assets/us/product/couchbase-server-

4.0/sql_servertonosql.pdf

Thank you.