Date post: | 13-Jul-2015 |
Category: |
Data & Analytics |
Upload: | couchbase |
View: | 1,990 times |
Download: | 0 times |
A N1QL for Every QuerySQL and More for a Document Database
Gerald Sangudi | Chief Architect, Couchbase
@sangudi
Rich data
Language
Architecture
Ecosystem
Mission
Roadmap
Resources
Demo
Q & A
Agenda — Previewing N1QL
©2014 Couchbase, Inc. 2
A Basic Data Model
©2014 Couchbase, Inc.
4
Tables, schemas, constraints
Prevailing model for 30+ years
A Rich Data Model
©2014 Couchbase, Inc.
5
Person
Name
DOB
Billing
Connections
PurchasesProduct
Structure and richness of real-world data
Drivers: Big Data, agility
Enabler: JSON — convenient, standard
Comparing the Data Models
©2014 Couchbase, Inc. 6
Basic Data Model Rich Data Model
Technology Relational Document
Building blocks
Flat tables Rich objects
Developer impact
Real world must be translated
Objects must be assembled
Data must be uniform
Change must be an exception
Real world can be reflected
Objects can be represented
Data can be uniform or varied
Change can be a norm
Query language SQL N1QL — SQL and more
N1QL Defined
©2014 Couchbase, Inc.
8
JSON: Rich DataSQL: Expressive Power N1: Nested Algebra
N1QL: Query Language for Rich Data
Systematic language design
Established theory & model
SQL++ benchmark & presentation
Features from SQL — Reads
©2014 Couchbase, Inc. 9*Upcoming
Reading Data
SELECT Projection
DISTINCT De-duplication
FROM Sourcing
JOIN INNER, LEFT OUTER
WHERE Filtering
GROUP BY Aggregation — HAVING, MIN, MAX, SUM, AVG, COUNT [ DISTINCT ]
ORDER BY Sorting
LIMIT, OFFSET Paging
UNION*, INTERSECT*, EXCEPT* Set operators
EXPLAIN Analyzing and tuning query execution plans
Features from SQL — Writes
©2014 Couchbase, Inc. 10
*Upcoming
Writing Data
CREATE INDEX Create a Couchbase view index or secondary index*
DROP INDEX Drop an index
INSERT* INSERT using VALUES or SELECT
UPDATE* UPDATE…WHERE, with document-level atomicity
DELETE* DELETE FROM…WHERE, with document-level atomicity
MERGE* MERGE with INSERT or UPDATE, depending on match
Features from SQL — Expressions
©2014 Couchbase, Inc. 11
*Upcoming
Expressions
Literals Primitives [ 0, ‘hello’, TRUE ]
NULL
Operators
Arithmetic [ +, -, *, /, % ]
Logical [ AND, OR, NOT ]
Comparison [ <, <=, =, !=, >=, >, BETWEEN, IS NULL ]
Pattern matching [ LIKE ]
Conditional [ CASE ]
Scalar functions
Numeric [ trigonometric, ROUND, TRUNC, … ]
String [ UPPER, LOWER, TRIM, SUBSTR, … ]
Date [ string and numeric dates, NOW, date arithmetic*, … ]
Aggregate functions MIN, MAX, SUM, AVG, COUNT [ DISTINCT ]
Subqueries* Subqueries are full expressions
Features for Rich Data — Nested Model
©2014 Couchbase, Inc. 12
Nested Data
Multi-valued attributes Arrays as attributes
Nested objects Multi-level nesting of objects; path navigation
NEST Collecting second term into nested array [ INNER, LEFT OUTER ]
UNNEST Flattening nested array [ INNER, LEFT OUTER ]
Collection operators Ranging, filtering, predicate, lookup, and slicing operators
Collection functions Sort, Reverse, Distinct, Append, Concatenate, Contains…
Collection aggregation* MIN, MAX, SUM, AVG, COUNT [ DISTINCT ]
Deep traversal* Finding or collecting of matching elements WITHIN any depth
Array and deep update* UPDATE of matching elements IN arrays and WITHIN any depth
Construction Dynamic construction of objects, arrays, and their combinations
*Upcoming
Heterogeneous Data
Heterogeneous inputs Input data of varying structure and type
Heterogeneous results Results of varying structure and type
Raw projection Support for non-tuple, non-object results
Schema inference* Inference of result structure and types, based on result expressions
Missing attributes IS [ NOT ] MISSING and other operators and functions
Attribute removal* UPDATE…UNSET to remove attributes
Total ordering* Comparison and ordering for all values, across all types
Implicit types* Type-encoding functions + expression indexes*
For example, range scans on dates
Closure JSON in, JSON out
Features for Rich Data — Heterogeneity
©2014 Couchbase, Inc. 13*Upcoming
Features for Distributed Data
©2014 Couchbase, Inc. 14
*Upcoming
Distributed Data
Key-value access KEYS available in SELECT, UPDATE, DELETE
KEYS used in JOIN, NEST, subqueries*, MERGE*
Document metadata META function to access ID, TTL, CAS, FLAGS…
LIMIT on UPDATE & DELETE* LIMIT available in write statements
Topology
©2014 Couchbase, Inc. 16
Client SDK
App
Index
CB Node
xDBC
App
Query
Manager
Data
Index
CB Node
Query
Manager
Data
Index
CB Node
Query
Manager
Data
Index
CB Node
Query
Manager
Data
Index
CB Node
Query
Manager
Data
Index
CB Node
Query
Manager
Data
Homogeneous node image (same bits)
Node roles & deployment flexibility
Independent resourcing & scaling
Query throughput & availability
Query Execution
©2014 Couchbase, Inc. 17
Index
Client
Data
FetchScanParse Plan Join Filter
Pre-Aggregate
Offset Limit Project
Data-parallel — Query is N data streams over N cores*
Memory-based
Pluggable architecture — datastore, index…
Request Response
*Upcoming
SortAggregate
ProjectorIndex
KeysRouter
Index Topology
Query Interface
Indexer
Indexer
Indexer
Indexer
Write Interfaces
Distributed
Document Store
Write Workload
DCP Mutation Stream
Timer
Incremental
Update
Index
Scan
Timestamp
Timestamp
Re-balancerTopology
Changes
Coordinator
Query Statistics
Usage Statistics
Secondary Indexing
Ecosystem
©2014 Couchbase, Inc. 19
xDBC
App
CB Node
xDBC
ETL
xDBC
BI
xDBC
Visualization
CB Node CB Node
Standards-based drivers
Simba partnership & presentation
Mission
©2014 Couchbase, Inc. 21
Power for Big Data Applications
Ability to index and query rich data at scale
Power of Couchbase extended to querying
Joy for Developers
Freedom, flexibility, alignment, productivity
Best of both — SQL and rich data
Value for Customers
Query technology for a new era
Enabler for applications of the future
Roadmap
©2014 Couchbase, Inc. 22
DP1 - Start
DP2 - Fixes
DP3 - Joins
DP4 - Language
Alpha - Integration
Beta - PerformanceGA - Production
9/13
11/13
3/14
Q4 ‘14
H1 ‘15
CB Connect
10/14
Resources
©2014 Couchbase, Inc. 23
query.couchbase.com
Downloads
Tutorial
Documentation
Blogs
Webinars
Forum
@N1QL
Follow us
Engage