Relational to NoSQL:Getting started from SQL Server
Shane JohnsonSr. Product Marketing Manager
Couchbase
©2015 Couchbase Inc. 2
Today’s agenda
§ Why NoSQL?§ Identifying the right application§ Modeling your data§ Accessing your data§ Installing and scaling your database§ Monitoring and managing your deployment§ Q & A
©2015 Couchbase Inc. 3
About the speakers – Shane Johnson
Shane JohnsonSenior Product Marketing ManagerCouchbase (since Dec 2013)
Experience:- Proud Red Hatter
- Consulting (Architect)- Transitioned to Marketing (Technical Manager)- Passionate about Open Source
- Expertise- Java and Distributed Systems
©2015 Couchbase Inc. 4
What’s Couchbase?
Couchbase is the company behind Couchbase Server & Couchbase Mobile
• Open source JSON database• Founded 2010• 500+ enterprise customers globally
Some of our customers:
Couchbase Server can be deployed as:
Document database Key-value store Distributed cache
©2015 Couchbase Inc. 5
What is NoSQL?
§ No SQL? No.§ Not only SQL? Not really.§ Non relational? Yes.
§ Distributed (most)– Scaled out, not up• Elasticity and commodity hardware
– Partitioned and replicated• Scalability, performance, availability
§ Schema-less (most)– Flexible model– JSON (some)
©2015 Couchbase Inc. 6
Who is using NoSQL?
Enterprises are adopting NoSQL for mission critical applications
Media & Publishing eCommerce Hospitality
©2015 Couchbase Inc. 7
Who is using NoSQL?
§ Gannett, publisher of USA Today and 90+ media properties, replaced relational database tech- nology with NoSQL to power its digital publishing platform.
§ Marriott deployed NoSQL to modernize its hotel reservation system that supports $38 billion in annual bookings.
§ FHLBank Topeka leverages NoSQL on top of SQL Server to speed up access to customer financial data for its 770 member banks
§ Cars.com, with over 30 million visits per month, replaced SQL Server with NoSQL to store customer and vehicle data
§ Vente-privee.com improved the customer experience of its 18 million members by replacing SQL Server with NoSQL for better performance
©2015 Couchbase Inc. 8
Why are they using NoSQL?
Technology Drivers
§ Customers are going online§ The Internet is connecting everything§ Big Data is getting bigger§ Applications are moving to the cloud§ The world has gone mobile
Technical Needs
§ Develop with agility– Flexibility + Simplicity– Easier + Faster
§ Operate at any scale– Elasticity + Availability– Performance at scale– Always-on, global deployment
Business Needs
§ Innovate and compete– Faster time to market– Reduced costs (operational + hardware)– Increased revenue
©2015 Couchbase Inc. 9
Why migrate from SQL Server?
§ Easier to scale3 nodes to 100s, 1 data center to many, commodity hardware
§ Better performanceintegrated caching, memory-optimized indexes, memory-based replication
§ Up to 40x lower costopen source, subscription-based, per instance (not per core)
§ Cross-platformruns on Windows or Linux (Red Hat, Ubuntu, Debian, etc)
§ Greater agilityJSON-based data model, SQL-based query language
©2015 Couchbase Inc. 10
How do you get started?
§ Identify the right application§ Model your data§ Access your data§ Install and scale your database§ Monitor and manage your deployment
©2015 Couchbase Inc. 11
Identifying the right application
Have one or more of the following characteristics or requirements:
Iterate fasterSend and receive JSONProvide low latency at any throughputSupport many concurrent users
Supports users anywhere and everywhereBe available 24x7Store terabytes of dataRead and write to multiple data centers
Service
RDMBS
Service Service
NoSQL
ApplicationExamples:
§ High performance, high availability caching service§ Small, independent application with a narrow scope§ Logical or physical service within a large application§ Global service that powers multiple applications
©2015 Couchbase Inc. 12
Concepts: Terminology
Relational (SQL Server) NoSQL (Couchbase)
Failover Cluster Cluster
Availability Group Cluster
Database Bucket
Table Bucket
Row (Tuple) Document (JSON)
Primary Key Object ID
IDENTITY Counter
Indexed View View
SQL N1QL
©2015 Couchbase Inc. 13
Model your data
©2015 Couchbase Inc. 14
Modeling your data: Fixed vs. self-describing schema
©2015 Couchbase Inc. 15
Modeling your data: The flexibility of JSON
Same document type,Different fields
• Different types• Optional• On-demand
Tip: Add a version field to track changes.
{“docType”: “user”, “docVersion”: “1”, …}{“docType”: “user”, “docVersion”: “2”, …}
©2015 Couchbase Inc. 16
Modeling your data: Changing the data model
Relational database
• Modify the database schema• Modify the application code (e.g. Java)• Modify the interface (e.g. HTML5/JS)
Document database
• Modify the interface (e.g. HTML5/JS)
©2015 Couchbase Inc. 17
Modeling your data: Object IDs
Best Practices
• Natural Keys• Human Readable• Deterministic• Semantic
Examples
• author::shane• author::shane::blogs• blog::nosql_fueled_hadoop• blog::nosql_fueled_hadoop::comments
What about identity columns?
1. Document<Long> nextAuthorIdDoc= bucket.counter(“authorIdCounter”, 1);2. Long nextAuthorId = nextAuthorIdDoc.content();3. String authDocId = “author::” + nextAuthorId; // author::101
Tip: Increment the counter by 10, 20, etc. instead of doing it for every insert.
©2015 Couchbase Inc. 18
Modeling your data: Relationships
Author
Blog (FK)Blog (FK)
Comment (FK) Comment (FK)
Author (FK x2)
BlogBlog (FK x2)
Comment Comment
Bottom up Top down
©2015 Couchbase Inc. 19
Modeling your data: Relationships
©2015 Couchbase Inc. 20
Modeling your data: Strategies and best practices
If… Then…
Relationship is one-to-one or one-to-many Store related data as nested objects
Relationship is many-to-one or many-to-many Store related data as separate documents
Data reads are mostly parent fields Store children as separate documents
Data reads are mostly parent + child fields Store children as nested objects
Data writes are mostly parent or child (not both) Store children as separate documents
Data writes are mostly parent and child (both) Store children as nested objects
©2015 Couchbase Inc. 21
Modeling your data: Strategies and best practices
§ Are they a lot of concurrent writes, continuous updates?§ Store children as separate documents
Blog§ Thread
§ Comment§ Comment
§ Thread§ Comment§ Comment
Blog
{“docType”: “blog”,
“author”: “author::shane”,“title”: “Couchbase Wins”,“threads”: [
“blog::couchbase_wins::threads::001”,“blog::couchbase_wins::threads::002”
}
Thread
{“docType”: “thread”,
“comments”: [{
“visitor”: “Brendan Bond”,
“text”: “This blog is amazing!”“replies”: [
{
“user”: “Dustin Johnson”,“text”: “No, it is not.”
}]
}}
©2015 Couchbase Inc. 22
Access your data
©2015 Couchbase Inc. 23
Accessing your data: Options
Key-Value(CRUD)
N1QL(Query)
Views(Query)
Documents
Indexes MapReduce
Full Text(Search)
Geospatial(Search)
We’ll focus on these three for now.
Indexes MapReduce
©2015 Couchbase Inc. 24
Accessing your data: Connecting to the database
§ Access data via topology-aware smart clients§ Maintains an up-to-date cluster map§ Communicates directly with database nodes – no proxies, no routers, etc.
§ Available for Java, Node.js, PHP, .NET, Python, C, Go, and more§ With standard, certified JDBC/ODBC drivers (if you want to)
©2015 Couchbase Inc. 25
Accessing your data: Domain objects vs. document objects
* JSON serialization via Boon.
Working with document objects requires less code, provides more flexibility.
©2015 Couchbase Inc. 26
Accessing your data: Key-value operations – referenced data
©2015 Couchbase Inc. 27
Accessing your data: Key-value operations – nested data
©2015 Couchbase Inc. 28
Accessing your data: Subdocument operations
©2015 Couchbase Inc. 29
Accessing your data – N1QL queries: Capabilities
Feature SQL N1QL
JOIN ✔ ✔
TRANSFORM ✔ ✔
FILTER ✔ ✔
AGGREGATE ✔ ✔
SORT ✔ ✔
SUBQUERIES ✔ ✔
PAGINATION ✔ ✔
OPERATORS ✔ ✔
FUNCTIONS ✔ ✔
©2015 Couchbase Inc. 30
Accessing your data: N1QL queries – referenced data
©2015 Couchbase Inc. 31
Accessing your data: N1QL queries – nested data
©2015 Couchbase Inc. 32
Accessing your data: N1QL queries – CRUD
©2015 Couchbase Inc. 33
Accessing your data: LINQ
_context.Query<Users>().Where(u => u.status == “Platinum”)
from user in _context.Query<Users>()join account in _context.Query<Account>()on user.accountId equals N1QlFunctions.Key(account) into userGroupfrom accounts in userGroup.DefaultIfEmpty()where (account.type == “Visa” || account.type== “MasterCard”)select new { firstName = user.firstName, lastName = user.LastName };
from user in _context.Query<Users>()join address in _context.Query<Address>()on user.addresses.shipping.addressId equals N1QlFunctions.Key(address) into addressGroupfrom address in addressGroup.DefaultIfEmpty()where address.state == “CA”select new { firstName = user.firstName, lastName = user.LastName };
©2015 Couchbase Inc. 34
Accessing your data: N1QL queries – indexes
Simple
Compound
Functional
Partial
©2015 Couchbase Inc. 35
Accessing your data: Views
What if indexed views worked great with write intensive workloads?
And you could use COUNT, ORDER BY, and everything else…
©2015 Couchbase Inc. 36
Accessing your data: Views
COUNT ?!?
©2015 Couchbase Inc. 37
Accessing your data: Views – Incremental MapReduce
©2015 Couchbase Inc. 38
Accessing your data: Views – queries
©2015 Couchbase Inc. 39
Accessing your data: Strategies and best practices
Concept Strategies & Best Practices
Key Value Operations provide the best possible performance
• Create an effective key naming strategy• Create an optimized data model
Incremental MapReduce (Views) are well suited to aggregation
• Ideal for large data sets, the entire data set• Can be used to create complex secondary
indexes
N1QL queries provide the most flexibility –everything else
• Query data regardless of how it is modeled• Remember to create indexes, leverage
covering indexes where possible
©2015 Couchbase Inc. 40
Install and scaling your database
©2015 Couchbase Inc. 41
Installing and scaling your database
Admin screenshot – add node
1. Download2. Install3. Configure4. Cluster / Scale5. Rebalance
©2015 Couchbase Inc. 42
Installing and scaling your Couchbase database: XDCR
§ Multiple Data Centers§ Cluster per data center§ Replicate between clusters§ Unidirectional / bidirectional§ Master / Master§ Local reads and writes§ Ring§ Hub-and-spoke§ Mesh§ Combination§ Built-in
©2015 Couchbase Inc. 43
Managing and monitoring your deployment
©2015 Couchbase Inc. 44
Managing and monitoring your deployment
§ Configuration– Authentication and authorization– Auditing
§ Monitoring– View and collect log information
§ Tooling– cbbackup & cbrestore– cbcollect_info & cbdstats– cbq– cbtransfer– and more…
§ Management, monitoring, and configuration – Cluster / node / bucket / views– Cross data center replication (XDCR)– Database performance, network
utilization, resource utilization
§ Tasks– Add and remove nodes– Failover nodes– Rebalance cluster
* Web, REST API, and CLI
©2015 Couchbase Inc. 45
Where to go next?
Conduct a Successful Proof of Concept
1. Select a use case and application2. Define the success criteria3. Understand the data4. Identify the access patterns5. Review the architecture
Measure your Return on Investment
§ Greater agility?§ Faster time to market?§ Easier scalability?§ Better performance?§ Better availability?§ Lower costs?
©2015 Couchbase Inc. 46
You’re just in time
Couchbase Server 4.5 Beta
Easier, more efficient queryingIntegrated query editorIndexed joinsIntegrated full-text searchReady your own writes
Simpler, more advanced data accessPartial updates, reads, etc.
www.couchbase.com/next
Faster, more powerful indexingMemory-optimized indexesArray indexingCircular reuse
Better, more comprehensive adminQuery monitoringFaster backup and restoreX.509 certificatesRole-based access control
©2015 Couchbase Inc. 47
Query Workbench: ad-hoc queries
©2015 Couchbase Inc. 48
Query Workbench: schema browser
©2015 Couchbase Inc. 49
Questions?
©2015 Couchbase Inc. 50
WANT TO LEARN MORE?
Getting Started guide:http://www.couchbase.com/get-started-developing-nosql
Download Couchbase software:http://www.couchbase.com/nosql-databases/downloads
Free Online Traininghttp://training.couchbase.com/online
“How to Get Started” White Paperhttp://www.couchbase.com/binaries/content/assets/us/product/couchbase-server-
4.0/sql_servertonosql.pdf
©2015 Couchbase Inc. 51
Thank you.