NoSQL in der Cloud - Why?

transcript

NoSQL in der Cloud – Why?

Andreas Hartmann

19.11.2012

19.11.2012 NoSQL in der Cloud – Why?2

Quelle: http://res.sys-con.com/story/mar12/2188748/ CloudBigData_0_0.jpg

Why Cloud???

What does Big Data mean???

Why NoSQL???

BigData Connectivity P2P Knowledge

Concurrencey Diversity Cloud

What is the Problem with Big Data

► Caching

► Master/Slave

► Master/Master

► Cluster

► Table Partitioning

► Federated Tables

► Sharding

Quelle: http://www.codefutures.com/database-sharding/

NoSQL Basics – CAP Theorem

Consistency

Each client always has the same view of the data

Availability

All clients can always read and write

Partition tolerance

The System continues to operate despite arbitrary message loss

Quelle: http://blog.nahurst.com/visual-guide-to-nosql-syste ms

ACID-BASE

Atomicity

► all or nothing rule

Consistency

► any transaction the database performs will take it from one consistent state to another

Isolation

► no transaction should be able to interfere with another transaction at all

Durability

► once a transaction has been committed, it will remain so

Basically Available

► the system does guarantee availability, in terms of the CAP theorem

Soft-State

► the state of the system may change over time, even without input

Eventual Consistency

► the system will become consistent over time, given that the system doesn't receive input during that time

Quelle: http://en.wikipedia.org/wiki/ACID

Quelle: http://stackoverflow.com/questions/3342497/explanat ion-of-base-terminology

Things to think about!!!

► Scalability

► High Availability

► Caching

► Architecture

► Load Balancing

► Queuing

► Data Handling

► Federation

► Serving Files

► Storing Files

Quelle: http://cloudscaling.com/blog/cloud-computing/up-out -centralized-and-decentralized

Open Your Mind;-)

Quelle: http://images.tribe.net/tribe/upload/photo/ deb/074/deb074db-81fc-4b8a-bfbd-b18b922885cb

NoSQL Tagcloud

Datastore Types

Key/Value Store

Document Store

Extensible Record

Stores (Wide-column Stores)

Graph Database

What ist means

Data is easily and quickly read/storedusing primary key

Denormalize data for commonly usedqueries

► Shema Design is optimized for the mostcommon Use-Cases

Developer

More technologies to have fun with

Broader choice of persistence stores

Probably Cross Store Persistence

► Store name, firstname etc in RDBMS

► Store followers in Graph database

► Store Content in RDBMS

► Store User Generated Content in Document database

Quelle: http://www.slideshare.net/adessoAG/no-sql-9355109

Wich is the right one

Quelle: http://www.slideshare.net/emileifrem/nosql-east-a-nosql-overview-and-the-benefits-of-graph-databases

Wich is the right one

Quelle: http://martinfowler.com/articles/nosql-intro.pdf

Architecture Case Study

Business Critical Data Read/Write

Data should be consistentRarley Changed Data /

Various Reads / Slight inconsistencies are OK

mongoDB as an Example

Document-oriented &

schemaless

JSON like documents

written in C++Open Source with a AGPL-Licence v.3.0

mongoDB in Detail

MongoDB Basics

Security and Authentication – Indexes

Replication – Scaling

Map/Reduce – Binary Data Sets

Monitoring – Backup

Schema Design – Connectivity – Ecosystem

mongoDB – Replication

Replica Sets

► A replica set consists of two or more nodes that are copies of each other

► The replica set automatically selects a primary (master).

► Drivers can automatically detect when a replica set primary changes and will begin sending writes to the new primary

Why Replica Sets

► Automated Failover

► Read Scaling (slaveOkay Method)

► Maintenance

► Disaster Recovery

Quelle: http://www.mongodb.org/display/DOCS/Replica+Sets

mongoDB – Scaling

Sharding

► Horizontal scaling across multiple nodes

Sharding Key

mongoDB – Scaling

Replica Sets comes together with Sharding

Quelle: http://www.mongodb.org/display/DOCS/Sharding+Introd uction

mongoDB – Map/Reduce

Parallel processing huge datasets on distributed sy stems

Data Data DataData

R E D U C E

► XXX

Map Reduce Execute

var map = function() {

emit( this.author, { pages: this.pages } );

Reduce

var reduce = function( key, values ) {var sum = 0;

values.forEach( function( doc ) {sum += doc.pages;

return { "pages": sum };};

Map Reduce Execute

Execute

db.bookstore.mapReduce( map, reduce,{ out: "myresultcollection" } );

{"result" : "myresultcollection","timeMillis" : 156,"counts" : {

"input" : 7,"emit" : 7,"reduce" : 3,"output" : 3

},"ok" : 1,

Map Reduce Execute

Quelle: http://res.sys-con.com/story/mar12/2188748/ CloudBigData_0_0.jpgQuelle: http://www.techweekeurope.co.uk/wp-content/ uploads/2011/03/Big-Data.jpg

Thank you for your attention!

www.adesso.deinfo@adesso.de

NoSQL in der Cloud - Why?

Technology