Polyglot Persistence & Multi-Model Databases

1

Michael Hackstein @mchacki

www.arangodb.org

Michael Hackstein @mchacki

‣ ArangoDB Core Team

‣ Web Frontend

‣ Graph visualisation

‣ Graph features

!!

‣ Organiser of cologne.js

‣ Master’s Degree(spec. Databases andInformation Systems)

2

Once upon a time…

‣ Use a SQL-based database

‣ Implement logic to transform your data into table format

‣ Create / Generate complex queries

‣ Implement logic to transform data from table format in required format

3

4

Shopping-Cart

RecommendationsSales-History Customer

Product-Catalog

An e-commerce system in Relational World

4

Main Categories of NoSQL DBs

5

Key/Value Store Document Store Graph Database

Source: Andrew Carol

Polyglot Persistence

Key-Value Store

‣ Map value data to unique string keys (identifiers)

‣ Treat data as opaque (data has no schema)

‣ Can implement scaling and partitioning easily due to simplistic data model

‣ Key-value can be seen as a special case of documents

6

Document Store

‣ Normally based on key-value stores (each document still has a unique key)

‣ Allow to save documents with logical similarity in “databases” or “collections”

‣ Treat data records as attribute-structured documents (data is no more opaque)

‣ Often allow querying and indexing document attributes

7

!

!

!

!

!

!

!

Graph Store

8

Polyglot Persistence

‣ „Use the right tool for the job“

‣ If you have structured data with some differences

‣ Use a document store

‣ If you have relations between entities and want to efficiently query them

‣ Use a graph database

‣ If you manage the data structure yourself and do not need complex queries

‣ Use a key-value store

‣ If you have structured data where all objects have equal attributes

‣ Use a relational database

9

10

Recommendations

Product-CatalogShopping-Cart

Sales-History Customer

KeyValueStore

Single-Model-Databases

10

{ “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, }

{ “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, }

DocumentStore GraphStore { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” }

{ “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, }

DocumentStore

423453453

4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95

=>

8743655635463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95

=>

{ “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" }

{ “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” }

{ “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }

{ “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }

DocumentStore

Benefits & Costs

11

‣ Native mapping of data into DB

‣ DB optimized

‣ Queries are tailored for your data format

‣ Focus on writing business logic

‣ Several technologies involved

‣ Experts required for each

‣ Administration effort is huge

‣ Application logic has to interface with several sources

‣ Data has to be stored redundantly and has to be kept in sync

Syncing Requirements

12

Customers Products

Recommendations

Sales-History

Extract E

dges

Suggest only Available Products

Filter suggestions

Find connected users

Buyers Purchases

‣ Problem: Different Data-Formats

‣ On small systems: One database could be sufficient

Multi Model Database

13

‣ Can store several kinds of data models:

‣ Acts as a key-value store

‣ Acts as a document store

‣ Stores graphs natively

‣ Delivers query mechanisms for all data models

14

‣ a multi-model database (document store & graph database)

‣ is open source and free (Apache 2 license)

‣ offers convenient queries (via HTTP/REST and AQL)

‣ offers high performance and is memory efficient

‣ uses JavaScript throughout (V8 built into server)

‣ doubles as a web and application server (Foxx)

‣ offers many drivers for a wide range of languages

‣ is easy to use with web frontend and good documentation

‣ enjoys good professional as well as community support

‣ and recently added sharding in Version 2.0.

Different query interfaces

Different scenarios require different access methods:

‣ Query a document by its unique id / key:

GET /_api/document/users/12345

‣ Query by providing an example document:

PUT /_api/simple/by-example { "name": "Michael", "age": 26 }

‣ Query via AQL: FOR user IN users FILTER user.active == true RETURN { name: user.name }

‣ Graph Traversals

15

Why another query language?

‣ Initially, we implemented a subset of SQL's SELECT

‣ It didn't fit well

‣ UNQL addressed some of the problems

‣ Looked dead

‣ No working implementations

‣ XQuery seemed quite powerful

‣ A bit too complex for simple queries

‣ JSONiq wasn't there when we started

16

Other Document Stores

‣ MongoDB uses JSON/BSON as its “query language”

‣ Limited

‣ Hard to read & write for more complex queries

‣ CouchDB uses Map/Reduces

‣ It‘s not a relational algebra, and therefore hard to generate

‣ Not easy to learn

17

ArangoDB Query Language (AQL)

‣ We came up with AQL mid-2012

‣ Declarative language, loosely based on the syntax of XQuery

‣ Other keywords than SQL so it's clear that the languages are different

‣ Implemented in C and JavaScript

18

Extendable through JS

‣ Dynamic Language that enriches ArangoDB

‣ Multi Collection Transactions

‣ Graph Traversals

‣ Cascading deletes/updates

‣ Aggregate data from multiple queries into a single response

‣ Data-intensive operations

‣ Actions, Foxx, Application Server

19

Application Server / Action Server

‣ ArangoDB can answer arbitrary HTTP requests directly

‣ You can write your own JavaScript functions (“actions”) that will be executed server-side

‣ Includes a permission system

‣ Build your own API rapidly using the Foxx framework

!

!

➡ You can use it as a database or as a combined database/application server

20

21

Product-CatalogShopping-Cart

Sales-History Recommendations Customer

DocumentStore

DocumentStore

DocumentStoreGraphStore

KeyValueStore

Use Case: Multi-Model-Databases

21

Syncing Requirements

22

Customers Products

Recommendations

Sales-History

Extract E

dges

Suggest only Available Products

Filter suggestions

Find connected users

Buyers Purchases

‣ Data-format is unified, no transformations

‣ On small systems: (Single Database)

‣ Separate into collections

‣ No redundancy

‣ Gain transaction possibilities

Join our growing community

23

.. working on the geo index, the full text search and many APIs: Ruby, Python, PHP, Java, D, ...

Summary

‣ Multi-Model databases simplify database setup

‣ Cross-out transformation of data-formats

!

‣ ArangoDB

‣ Based on web standards: HTTP, REST, JSON

‣ Flexible querying: Documents, Graphs in the same language

‣ Reduce Server <-> Database communication: ‣ Define your own API using Foxx

‣ Execute data-intensive code within the database

‣ You can even use ArangoDB as our application server

24

25

Thank you

!

‣ Further questions?

‣ Follow me on twitter/github: @mchacki

‣ Or write me a mail: mchacki@arangodb.org

25