The Dark Side of NoSQL

Post on 20-Nov-2014

602 views 2 download

Tags:

description

NoSQL databases like MongoDB are making it easier than ever for developers to quickly build complex, agile applications that scale well. Unfortunately, the rise of NoSQL has dark side: data stored in NoSQL databases is invisible to traditional business analytics and intelligence apps. As a result, operations often spends a massive amount of time and effort developing complex pipelines to transform NoSQL data into a format compatible with traditional RDBMS. In this talk, John A. De Goes introduces SlamData, a new open source project designed to build a NoSQL BI application by extending the operations and data model of relational algebra. The result is backward compatible with SQL, but allows freeform analytics on semi-structured, heterogeneous, deeply nested data. John discusses the design of SlamData and highlights some of the issues involved in bringing the project to life.

transcript

The Dark

Side of NoSQLJohn A. De Goes - @jdegoes

CTO SlamData Inc.

May 20, 2014 -- campDevOps

What

is N

oSQ

L? NoSQL = non-relational, operational,

(generally) distributed

databases.

Battle

of th

e

Para

dig

ms

SQL

• Flat

• Schema-ful

• Uniform

• SQL QL

• Atomic

• Ops Power

• Old/Boring

NoSQL

• Nested

• Schema-less

• Non-Uniform

• Ad Hoc QL

• Varies

• Dev Power

• New/

Exciting!

The R

ise o

f N

oSQ

L

Google Trends“MongoDB vs. Oracle DB”

So what’s the #1

problem with NoSQL?It’s not scaling, migrations, training, etc.

A S

ad, C

om

mon

Sto

ry

1.Dev chooses NoSQL DB

2.App generates data3.PHB wants answers4.Dev / Ops morph into

report monkeys, or5.Dev / Ops build ETL

equivalent of Death Star

What’s the solution?[Drumroll…]

“BI fo

r N

oSQ

L”

• 100% SQL

• 100% NoSQL

• 100% Open Source

• Doors open June 23!!!

“So simple my cat could use it!”*

*Simulated quote from hypothetical user

HTML5 GUI

SlamSQL

SlamEngineMRA

Ria

k

MongoD

B

Etc

.

SlamData

Snazzy BI interface for NoSQL

SlamData

Snazzy BI interface for NoSQL

• Run SlamSQL on different backends and visualize

results

• Create static or interactive reports

• Publish reports online or send them by email

• Very simple now but will evolve into NoSQL BI

SlamSQL

SQL for NoSQL

SELECT C

OUNT(*)

AS cnt,

v

iews[*].

searchTe

rms[*] A

S term,

v

iews[*].

date AS

date

FROM pro

ducts

GROUP BY

views[*

].date,

views[*

].search

Terms[*]

SlamSQL

SQL for NoSQL

• Nothing new to learn

• Backward compatible with ANSI SQL

• New operators to work with heterogeneous, nested

data

• Substantially more composable & expressive than

SQL

• Fully-inferred structural typing

• Intuitive JS syntax for accessing nested data

MRA

Multi-Dimensional Relational Algebra

1.Extends relational algebra

2.Extends relational data model

3.Strictly more powerful than relational

1. Unlike most NoSQL QLs designed for operations!

4.Natively designed for schema-less data

SlamEngine

The Brainz of SlamData

SlamSQL Parser

Structural Type-Inferencer

Logical Plan Compiler

Ria

k Pl

anner

MongoD

B

Planner

Etc

.

SlamEngine

The Brainz of SlamData

1.Written in 100% Scala

2.Clean, modular, robust architecture

3.Geared at supporting many back-ends

4.MongoDB first supported backend

5.Early in development (doesn’t fully support MRA)

6.Rapid progress with growing team

7.Easy admin: install, point & launch!

Quest

ions?

Q: Why is all the text

slanted?

A: Because you should

be paying attention to

the speaker and not trying to read the slides!

Thank you!*

*Special thanks to Laura Crow & the awesome JumpCloud team for

making this talk possible on shorter-than-short notice!

MRA

More

deta

il th

an y

ou e

ver

wante

d t

o k

now

• [*] – array flatten / hidden deconstructing expansion

• {*} – object flatten / hidden deconstructing

expansion• [<] – array dimensional expansion

• {<} – object dimensional expansion

• [e…] – array dimensional aggregation

• {e…} – object dimensional aggregation

• INDICES A – Array index replicating expansion

• FIELDS A – Field name replicating expansion

• A GROUP BY B – dimensional contractionPlus all the standard and extended relational operators!