IntravertExperimental

Transport for Apache Cassandra

Friday, April 5, 13

Hi Folks!I’m Nate @zznate

Friday, April 5, 13

API ManagementAPI AnalyticsAPI Tools

Friday, April 5, 13

Vert.x + Cassandra = ?

Friday, April 5, 13

But first: Architectural stuff

Friday, April 5, 13

Collapsed for brevity.see slides 6-31 from this presentation:http://www.slideshare.net/zznate/apachecon-cassandra-transport

Friday, April 5, 13

So there are options, why create something new?

Friday, April 5, 13

Common client patterns emerged

Friday, April 5, 13

union operationsaggregationsfilteringjoins (round-trip required!)redundant data on insertion

Friday, April 5, 13

These client patterns are wasteful.

Friday, April 5, 13

Lots of CPU and memory burned (server and client)

Lots of data on the wire won’t be used (ser/deSer cost)

Friday, April 5, 13

One more thing about CQL...

Friday, April 5, 13

“If [they] think that CQL is the answer to usability then I just won. We at least know where our problems are.”- 10gen exec.

Friday, April 5, 13

Transports in sum

Friday, April 5, 13

Thrift:I can has distributed computing degree

Friday, April 5, 13

CQL:MOAR TOOLz!!!

Friday, April 5, 13

IMO: In 2013, binary protocols with idiomatic clients misses the market.

Friday, April 5, 13

I love writing apps in Java (sometimes python), but...

Friday, April 5, 13

The most popular application framework these days:

Friday, April 5, 13

The most popular application framework these days:HTML5

Friday, April 5, 13

and thus...Intravert

Friday, April 5, 13

Any new transport had to be HTTP based

Friday, April 5, 13

Easy to useEasy to secureEasy to compressYou can test with a browser!

Friday, April 5, 13

The payload had to be JSON based

Friday, April 5, 13

Quick Vert.x summary

Friday, April 5, 13

- Event-driven IO (Netty)- Message bus- Runtime load/unload of modules- Polyglot support via invokeDynamic in JDK7

http://vertx.ioFriday, April 5, 13

Initial use case: Server side indexing

Friday, April 5, 13

Easy to construct most common operations

Friday, April 5, 13

Simple REST semanticsPOST /myks/mycf/key1{“uname”:”zznate”}

GET /myks/mycf/key1/uname{”zznate”}

Friday, April 5, 13

Command-driven JSON payload

Friday, April 5, 13

Extensible and transparentOrdered operations Not tied to any predefined notions

Friday, April 5, 13

Benefits of doing this with Vert.x

Friday, April 5, 13

Message bus flexibilityConfigure which modules to load (read-only nodes!)Hot deploy of fixes and new features

Friday, April 5, 13

Cassandra is best-of-breed for a lot of things. So we used it as a framework.

Friday, April 5, 13

Example Intravert Operations

Friday, April 5, 13

Slice:{"id":6,"type":"slice","op":{      "end":"9",      "rowkey":"5",      "size":4,      "start":"1"}}

Friday, April 5, 13

Set:{"id":2,"type":"SET","op":{            "rowkey":"key1",      "uname":"zznate",      "email":"zznate.m@gmail.com",      "age":37}}

Friday, April 5, 13

Easy to use Composites

Friday, April 5, 13

{"id":1,"type":"SET","op":{            "rowkey":"key1",      "COMP_NAME":[          "37",  "zznate"      ],      "value":"key1",      }}

Friday, April 5, 13

One or more commands per payload

Friday, April 5, 13

Friday, April 5, 13

Flexible batching - batch reads and writes- different keyspaces in same op- change ks/cf mid-op- streaming (soon)

Friday, April 5, 13

Server-side Filtering

Friday, April 5, 13

{ "type": "CREATEFILTER", "op": { "name": "stouts", "spec": "javascript", "value": "function(row) { if (row['value'] == 'Breakfast Stout') return row; else return null; }" } }, { "type": "FILTERMODE", "op": { "name": "stouts", "on": true } }, { "type": "SLICE", "op": { "rowkey": "beers", "start": "Allagash", "end": "Sierra Nevada", "size": 9 } }

Friday, April 5, 13

{ "exception":null, "exceptionId":null, "opsRes": { "0":"OK", "1":"OK", "2":"OK", "3":"OK", "4":"OK", "5":"OK", "6":[{ "name":"Founders", "value":"Breakfast Stout" }]}}

Friday, April 5, 13

GetRef:Results of one operation as input from another

Friday, April 5, 13

{ "type": "GET", "op": { "rowkey": "getref_test", "name": "col1" } }, { "type": "SET", "op": { "rowkey": "getref_test", "name": "col3", "value": { "type": "GETREF", "op": { "resultref": 4, "wanted": "value" } } } }, { "type": "GET", "op": { "rowkey": "getref_test", "name": "col3" } }

Friday, April 5, 13

“Triggers”- Perform complex multi-step operations server side- Examine and change request objects- Interface directly with IntraVert- Interface directly with Cassandra (StorageProxy, CFMetaData, etc)

Friday, April 5, 13

Multi-Processors:Shape results with a union of operations

Friday, April 5, 13

Prepared Request Statement:Requests saved server side - send only the bind variables

Friday, April 5, 13

CQL support built-in.

Friday, April 5, 13

What’s missing?

Friday, April 5, 13

Security (Shiro integration coming soon)Distribution semanticsCassandra-RESTHBASE-like scanners

Friday, April 5, 13

github.com/zznate/intravert-ug

Friday, April 5, 13