Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA

transcript

Confidential

Polyglot Persistence

December 22nd, 2015

Art van Scheppingen

Severalnines

art@severalnines.com

Confidential

Copyright Severalnines AB

Logistics

Webinar is recorded Replay available soon Feel free to ask questions at any time Use your control panel to contact us Or email us as well: info@severalnines.com

Confidential 3

Agenda What is Polyglot Persistence? Example project Is Polyglot Persistence good or bad? Leverage ClusterControl for Polyglot Persistence

(demo) Q&A

Confidential

What is Polyglot Persistence?

Confidential

Using multiple specialized persistent stores rather than

one single general-purpose database

Confidential

Where does the term come from?

The way we work is changing Enterprise applications are becoming more complex Separate (devops/agile) teams Ownership of applications (Micro)services

Everyone has their preference Various programming languages Various storage systems

Confidential

Where does the term come from?

Monoglot Programming Only one programming language allowed Readability

All code is in the same language Support

One platform to support Knowledge

Everybody is an expert Is there a jack-of-all-trades language?

Confidential

Monoglot Programming

Does a carpenter only use just one tool?

Confidential

Monoglot Programming

Carpenters use a broad variety of tools

Confidential

Polyglot Programming

Polyglot Programming Use programming languages for what they are good at Flexibility

Use Java for a secure API Use Scala for real time stream processing Use Python for text analysis Tie everything together using AngularJS

Knowledge Everybody is expert at one or more languages

Confidential

Polyglot Programming

Confidential

Monoglot Persistence

Confidential

Data storage landscape changes

Relational data stores (RDBMS) Key-Value data stores Columnar data stores Document data stores Graph data stores

Confidential

Data stores overview

SoftwareRDMBS MySQL,

PostgreSQL, Oracle

Key-Value Redis, RiakColumnar Cassandra,

Hbase, InfiniDB

Document MongoDB, Couchbase

Graph Neo4J, Titan

Confidential

Data stores overview

Software AWS GoogleRDMBS MySQL,

PostgreSQL, Oracle

RDS, Aurora CloudSQL

Key-Value Redis, Riak DynamoDB DatastoreColumnar Cassandra,

Hbase, InfiniDB

Redshift Big Query

Document MongoDB, Couchbase

SimpleDB Bigtable

Graph Neo4J, Titan Titan + DynamoDB

Confidential

Even Hadoop becomes a polyglot

Confidential

Complex problems require different storage systems Use the right tool for the job, for example

Use PostgreSQL for financial data Use MySQL for website contents Use MongoDB for user profiles Use Cassandra for real time streams Use Neo4J for recommendation analysis

Confidential

Use the right tool for the right job Document storage: MongoDB

Confidential

Use the right tool for the right job Columnar storage: Cassandra

Confidential

Use the right tool for the right job Graph storage: Neo4J

Confidential

Example project

Confidential

Project management lifecycle

Confidential

Development

Requirements are gathered Create a recommendation system Storage must be (blazingly) fast Document must contain user details Data must be kept until the next visit

Choose the right tool for the right job Store data as a document Use MongoDB

Development

Confidential

But what happens after development?

Delivery Ops sets up the application on the infrastructure Monitoring is added to see if the application works If everything is fine the product will be handed over

What’s next? Systems monitoring Backups Scaling Software updates BI

Confidential

The ops perspective

Confidential

Systems monitoring

Supported data stores so far: MySQL Memcached

How do we monitor MongoDB? Integration with Nagios / Zabbix / Shinken Integration with Graphite / OpenTSDB / InfluxDB

What should we monitor? Key performance metrics SQL vs Mongo queries

Confidential

Backups

Supported backups so far Xtrabackup streams

How do we backup MongoDB? What should we backup from MongoDB? How do we restore from a backup? How do we provide a backup for development?

Confidential

Scaling out

We are data hoarders and like to keep everything Scaling options in MySQL

Read-bound workloads: add more slaves Write-bound workloads: sharding or NDB Cluster

Scaling in MongoDB? When should we scale out? How do we add a new shard? How do we redistribute between shards? How do we add a new read slave?

Confidential

Software updates

When do we need to update with MySQL? Once we hit a bug? Every major release? Every minor release?

When do we need to update with MongoDB? How do we know the impact?

Confidential

Business Intelligence

BI with MySQL? Export full database from a slave Allow schemas/tables to be queried for specific tables Most BI tools can connect to MySQL

BI with MongoDB? Native MongoDB analytics tools Write your own exports BI connector (since version 3.2)

Confidential

A new project emerges

Confidential

With new requirements

Requirements are gathered Allow users to view their earnings Data must always be 100% reliable Storage must be ACID compliant

Choose the right tool for the right job Store data relational Use PostgreSQL

Confidential

Is Polyglot Persistence good or bad?

Confidential

The drawbacks

More specific (domain) knowledge is needed Additional tooling may be necessary Need more hands to maintain the systems

Confidential

The benefits

More flexibility to your company Driven and dedicated devops who take ownership Use the right tool for the right job

Confidential

The challenges

Deploy various data stores Handle HA and failovers Scaling and sharding Interpret graphs and monitoring Schedule backups and handle recovery Handle updates/upgrades

Confidential

Polyglot Persistence is here to stay

Variety of data storage needs keeps growing Polyglots in the cloud Better be prepared for it

Confidential

Leverage ClusterControl for Polyglot Persistence (demo)

Confidential

Confidential 43

Thank You! ClusterControl

www.severalnines.com/product/clustercontrol

ClusterControl – Getting Started www.severalnines.com/getting-started

Polyglot Persistence meetups http://goo.gl/64Ga5z

Severalnines Blog www.severalnines.com/blog

Contact: info@severalnines.com

New ClusterControl Tips & Tricks

Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA

Technology