+ All Categories
Home > Data & Analytics > Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Date post: 11-Jul-2015
Category:
Upload: nosqlmatters
View: 456 times
Download: 4 times
Share this document with a friend
32
© 2014 IBM Corporation Building Applications for Success Glynn Bird, Developer Advocate
Transcript
Page 1: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

© 2014 IBM Corporation

Building Applications for SuccessGlynn Bird, Developer Advocate

Page 2: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Agenda

• Application Development

• Scaling

• Real-world examples & how can Cloudant help?

Page 3: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Introduction

@glynn_bird [email protected]

Glynn BirdDeveloper AdvocateCloudant

http://glynnbird.com

Page 4: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Development - Go-to tools

Basic

Pascal

C

C++

Forth

PHP4Java

PHP5

ObjC

Node.js

1980s 1990s 2000s 2010s

File System

NoSQL

SQL

Page 5: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Scalability?

• Pick your tools of choice• PHP + MySQL, Ruby + Mongo

• Build on your machine

• Deploy!

• Scalability is an ops problem?

• Hint: “no”

Page 6: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Scaling tip #1 - Keep things separate

• Separate • load balancing• app• database• cache

• Separation of layers allows independent scaling

http://bit.ly/10xGe2YGoogle’s first server rack

Page 7: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Scaling tip #2 - Keep app servers stateless

• Store all persistent data in a database

• Don’t be tempted to store sessions on local disk

• Stateless app servers can be created and destroyed at will

http://bit.ly/10xGe2Y

IBM 305 RAMAC 5Mb Disk - 1957

Page 8: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Scaling tip #3 - queue-driven workflow

• Store tasks in queues

• Workers servers consume the queue

• Queues allow workers to be scaled at will

http://bit.ly/1Aycjse

Page 9: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Scaling tip #4 - Automate everything

• No need to manually configure servers anymore

• Workers servers consume the queue

• It should be as easy to deploy N servers as 1 server

http://bit.ly/1Aycjse

Page 10: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

What does failure to scale look like?

http://bit.ly/1Aycjse

Page 11: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014
Page 12: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

What does failure to scale look like?

• increased latency

• downtime

• high maintenance costs

• data loss

• disgruntled customers

http://bit.ly/1Aycjse

Page 13: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Cloudant Highlights• Founded in 2009 by data scientists:

Hoffman, Miller & Kocoloski

• 20,000+ freemium and 225+ paying customers

• Key collaboration & investment agreement with Samsung Electronics

• HQ’d in Boston with offices/presence in Seattle, San Francisco, Bristol, UK & Seoul, SK

• 34 global locations on 4 hosting providers

• Acquired by IBM in February 2014 into the Information Management software group

Page 14: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Cloudant NoSQL DBaaS

Indexed JSON

Storage

Geo-Load Balancing

App Layer / Browser / Mobile App

Physical Storage @ IBM SoftLayer, Rackspace, Microsoft Azure, AWS

Cloudant JSON API

JSON CRUD

Multi- Master

Replication

Device Sync

Lucene Search

2D, 3D Geospatial

Incremental MapReduce Monitoring Security

IO Control

Managed for you 24x7 by Cloudant developer-operators

REST (HTTPS)

Page 15: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

How can Cloudant help you to scale?

• Scales massively & elastically• Handle millions of daily active users

• Guaranteed performance & up time• Removes risk of project & SLA delivery failure

• More agile development for web & mobile• No rigid schemas to slow development

• Managed for you 24x7• Stay focused on new development, no DB administration

Cloudant empowers clients to get their applications to market faster in a cost-effective, hassle-free delivery model with guaranteed performance and support.

Page 16: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Example 1 - IoT Data Logging

• Temperature sensor £1.50

• Raspberry Pi £25

• Network attachedthermometer

DS18B20

Page 17: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

DS18B20

Software

• Bash script

• Where should we put the data ?

sensorid=`find /sys/bus/w1/devices/* -type l | grep -o '[0-9]*-.*$' | head -n 1`filename="/sys/bus/w1/devices/$sensorid/w1_slave"T=`tail -n 1 "$filename" | grep -o "[0-9]*$"`t=${T:0:2}.${T:2:3}unix=`date "+%s"`d=`date "+%Y-%m-%d %H:%M:%S %Z"`json="{ \"ts\": $unix, \"date\": \"$d\", \"temperature\": $t, \"sensor_id\": \"$sensorid\" }"

Page 18: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

DS18B20

Storage

• Local CouchDB

• Installed with one line

sensorid=`find /sys/bus/w1/devices/* -type l | grep -o '[0-9]*-.*$' | head -n 1`filename="/sys/bus/w1/devices/$sensorid/w1_slave"T=`tail -n 1 "$filename" | grep -o "[0-9]*$"`t=${T:0:2}.${T:2:3}unix=`date "+%s"`d=`date "+%Y-%m-%d %H:%M:%S %Z"`json="{ \"ts\": $unix, \"date\": \"$d\", \"temperature\": $t, \"sensor_id\": \"$sensorid\" }"curl -X POST -H "Content-Type: application/json" -d "$json" "http://localhost:5984/logger/"

Page 19: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Where does Cloudant fit in?

• Replicate to Cloudant

• Scale by adding more sensors

• Data collection can be offline

• Sync when connected

Page 20: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

Demo

• http://celsius.mybluemix.net

Page 21: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

• Replication• collect data at the edge, replicate when online

• Cloudant can store data at massive scale with high availability

• Time-series data is a great fit for Cloudant

Why is Cloudant a good fit?

Page 22: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

• MapReduce indexes on date field

• Allows range querying with grouping e.g. find min/max/avg between two dates, grouped by year/month/day/hour etc.

Time-series data in Cloudant

{ "_id": "00339bc7", "_rev": "1-0ef01b53", "ts": 1411723490, "date": "2014-09-26 09:24:50 UTC", "temperature": 20.937, "sensor_id": "28-000006746cde" }

[2014,9,26,9,24,50] ---> 20.937

Page 23: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

• Mobile game

• Stores games state, usage stats, achievements etc. locally

• Needs to be cross-platform

Example #2 - Mobile applications

Page 24: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

• In-browser, cross-platform datastore

• CouchDB compatible

• Syncs with Cloudant

Option 1 - PouchDB + HTML5

http://pouchdb.com/

Page 25: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

• iOS and Android native libraries

• Local storage + query

• Syncs to and from Cloudant

Option 2 - CloudantSync

https://cloudant.com/product/cloudant-features/sync/

Page 26: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

One database per user

Page 27: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

• Each user’s data has it’s own access control

• Each database is relatively small; small enough to fit on phone

• Scales massively; hundreds of thousands of concurrent users

One database per user

https://cloudant.com/product/cloudant-features/sync/

Page 28: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

• Business Card collection app

• Native Android App

• Companion Web App

Demo

https://cloudant.com/product/cloudant-features/sync/

Page 29: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

How did that work?

pouchdb

Android Phone App

IBM BlueMix

Web App

Page 30: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

var db = new PouchDB("myfirstdatabase");

db.post( { a:1, b: "2", c: true } );

db.sync(URL, { live: true });

What about the code?

https://cloudant.com/product/cloudant-features/sync/

Page 31: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

• Replication is a hard problem. PouchDB + Cloudant solve it

• One database per-user is a great way to scale mobile apps

Why is Cloudant a good fit?

Page 32: Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014

• Building with Cloudant as your data layer has become a “go to” development solution

• Building with Cloudant builds in scale from day one• large data sets• lots of smaller data sets

• Cloudant syncs!

Conclusion

https://cloudant.com/product/cloudant-features/sync/


Recommended