Post on 29-Apr-2015
description
transcript
Pivotal #CassandraSummit
Cassandra on Pivotal CF
Pivotal #CassandraSummit
Tammer SalehDirector of Product
Pivotal CF Services, London !
http://tammersaleh.com
Overview of Pivotal CF!Services API!
Why Cassandra!Building Cassandra!
Automating Cassandra Operations
Pivotal #CassandraSummit
Overview of Pivotal CF
Pivotal #CassandraSummit
$ cf push is golden.
$ cf create-service p-cassandra development-plan eventsdb $ cf bind-service eventsdb dashboard $ cf push dashboard $ open https://dashboard.mypcf.com
Pivotal #CassandraSummit
Pivotal #CassandraSummit
Pivotal #CassandraSummit
vSphere
BOSH provisioning / configuration / orchestrationOperations
Manager
AWS OpenStack
Console
Service Foo
Broker DB
Service Bar
Broker DB
Service Baz
Broker DB
Runtime
DEA Cloud Controller
Router
DEA
DEA DEA
Health Manager
SSO
Services APIPivotal #CassandraSummit
vSphere / AWS / OpenStack
BOSHOperations Manager
CF Runtime
Application Instances
Cloud Controller
Router
vSphere / AWS / OpenStack
BOSHOperations Manager
CF Runtime
Cloud Controller
Router
Service
Service Instances
Service Broker
Application Instances
vSphere / AWS / OpenStack
BOSHOperations Manager
CF Runtime
Cloud Controller
Router
Service
Service Instances
Service Broker
Application Instances
Catalog
$ cf marketplace
vSphere / AWS / OpenStack
BOSHOperations Manager
CF Runtime
Cloud Controller
Router
Service
Service Instances
Service Broker
Application Instances
Provision instance
Done
$ cf create service foo
CF Runtime
Application Instances
Cloud Controller
Router
Service
Service Instances
Service Broker
Provision binding
Binding info
$ cf bind service my_app foo
$VCAP_SERVICES={ "pgsql": [ { "name": "UUID", "plan": "small", "credentials": { "uri": “postgres://u:p@service_ip” } } ], … }
Binding info
CF Runtime
Application Instances
Cloud Controller
Router
Service
Service Instances
Service Broker
$ cf bind service my_app foo
App instance reads and writes directly to
service instance.
GET /v2/catalog What’s available?
PUT /v2/service_instances/:id Create service
DELETE /v2/service_instances/:id Delete service
PUT /v2/service_instances/:id/service_bindings/:id Create binding
DELETE /v2/service_instances/:id/service_bindings/:id Delete binding
Pivotal CF Services
Insanely fast KV storage - The AK-47 of Databases
The World's Leading Graph DB
Distributed, real-time search and analytics engine
Simple and Fast NoSQL DB
High-performance memory object cache.
Next generation MySQL replacement.
Enterprise Hadoop Distribution.
S3 Compatible Blobstore
Highly distributed, write-heavy KV/column store.
Our first production-grade service.
"In terms of scalability, there is a clear winner throughout our experiments.
Cassandra achieves the highest throughput for the maximum number of
nodes in all experiments."
- University of Toronto, 2012
Why ?
Why ?
Cassandra is truly cloud-scale.
Why ?
Why ?
Multi-datacenter deployments are commonplace in
the cloud.
Building Cassandra
Pivotal #CassandraSummit
Pivotal #CassandraSummit
Plan InstanceService Binding
Services API Redux
Pivotal #CassandraSummit
InstanceWhat is an ?
InstanceWhat is an ?
Scales to the size of a single cluster.!
Single Keyspace.!
Good for development and testing.
Pivotal #CassandraSummit
Part of a Shared Cluster
VM VM VM
Cassandra Node Cassandra Node Cassandra Node
Instance 1 (Keyspace)
Instance 2 (Keyspace)
Instance 3 (Keyspace)
Keyspace visibilityPivotal #CassandraSummit
Noisy Neighbours
Pivotal #CassandraSummit
Quotas
Pivotal #CassandraSummit
InstanceWhat is an ?
True production grade.!
No noisy neighbours.!
Expensive.
Pivotal #CassandraSummit
Cluster of VMs
Instance 1 (Cluster of VMs)
Cassandra VM Cassandra VMCassandra VM
Instance 2 (Cluster of VMs)
Instance 3 (Cluster of VMs)
Cassandra VM Cassandra VMCassandra VM
Cassandra VM Cassandra VMCassandra VM
InstanceWhat is an ?
Looks identical to production.!
Fairly good noisy neighbour isolation.!
Constrained resources per instance.
Pivotal #CassandraSummit
Cluster of Containers
VM VM VM
Instance 1 (Cluster)
Cassandra Node
Instance 2 (Cluster)
Instance 3 (Cluster)
Cassandra Node
Cassandra Node
Cassandra Node
Cassandra Node
Cassandra Node
Cassandra Node
Cassandra Node
Cassandra Node
Automating Cassandra Operations
Pivotal #CassandraSummit
BOSH
Pivotal #CassandraSummit
BOSH is…• Predictable!• Repeatable!• IaaS agnostic!• Built for large-scale deployments
Pivotal #CassandraSummit
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
VM
Agent
hello!
hey !there!
ahoy!
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
VM
Agent
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
VM
Agent IaaS
STONITH!
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
VM
Agent IaaS
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
IaaS
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
IaaS
Provision!
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
IaaS
VM
Agenthowdy!
NTPd
Pivotal #CassandraSummit
Pivotal #CassandraSummit
NTPd!Accuracy vs Precision
Pivotal #CassandraSummit
NTPd!!
https://blog.logentries.com/2014/03/synchronizing-clocks-in-a-cassandra-cluster-pt-2-solutions/
Cassandra Node
Cassandra Node
Cassandra Node
Pivotal #CassandraSummit
NTPd!!
https://blog.logentries.com/2014/03/synchronizing-clocks-in-a-cassandra-cluster-pt-2-solutions/
Cassandra Node
Cassandra Node
Cassandra Node
NTPd!Server
Pivotal #CassandraSummit
NTPd!!
https://blog.logentries.com/2014/03/synchronizing-clocks-in-a-cassandra-cluster-pt-2-solutions/
Cassandra Node
Cassandra Node
Cassandra Node
NTPd!Server
Internet
NTPd!Server
NTPd!Server
NTPd!Server
Running repairs
Pivotal #CassandraSummit
Running repairs
Pivotal #CassandraSummit
1. When decommissioning the Node.
2. When the node has been down for N minutes.
3. Once per node per week within GC_GRACE_PERIOD
Services Are the Key to the Success of
Pivotal CF
Pivotal #CassandraSummit
Tammer Salehtsaleh@pivotal.io
http://tammersaleh.com
Thank you.
Pivotal #CassandraSummit
If this interests you, we’re hiring CF engineers in SF and London.