Post on 21-Jan-2018
transcript
Agenda
Why OpenEBS?
OpenEBS Community Updates
Demo - OpenEBS on AWS
Demo - Benchmarking DB on Containers
Q/A
● Back in the day of IPX/SPX
○ We had many servers with all a
little storage
● Performance was not at the level
of current 15K SAS drives
● Back-ups jobs were hard to
manage
● Networking was really fast back
then
The idea was to ‘consolidate’ all storage
islands into a large one such that all
things become easier and faster
Why do we have centralised storage?
● Network of storage systems
○ FC loops as ethernet was not
fast enough
● Special purpose file systems to
implement advanced storage
features
○ Deduplication
○ Snapshots
○ Clone
○ Replication
○ Etc
● Scale as far as the controller
would let you
Storage (attached) Networks
● Container change the way we
typically consume storage from
an application perspective
○ Number of mounts points
high
● This alone creates scalability
problems
● Solution for OpenEBS would
be to explore a distributed
solution
● But…
○ Applications are already
distributed!
Apps changed; storage is still the same
● Distributed filesystems are inherently difficult to develop and more importantly hard and
nasty to maintain in production
● Require more often than not, specialised client drivers to really leverage distributed
nature. NFS and SMB are not distributed client protocols
● We could have lipsticked something already there
○ Linux comes with distributed filesystems within kernel-tree
Not Yet Another Distributed Filesystem
● Application have native scalability features, load balancers, database sharding, Paxos and RAFT
(applications go down stack)
● Does not mean that we don't need storage HA by definition but it does show
○ Data availability is not exclusively controlled at the storage layer
○ App performance is now loosely coupled from storage
○ Does the developer know best what the micro services need?
● How we manage and deploy application has also changed
● Applications have changed and somebody has forgotten to tell storage
What app changes are we talking about?
● Bringing enterprise class storage features to individual storage containers○ No compromise on features
● Cloud native design build using the same containers technology it stores for○ Don't monkey patch existing storage with plugins
● Implemented in user space to fully leverage high speed fabrics and devices○ 24 cores, 40gbe networking
● Declarative provisioning and protection policies○ You decide what the storage should look like
● Multi-cloud○ Data mobility included
● Not a cluster storage instance but a cluster of storage instances
Why OpenEBS ?
Hacktoberfest
Find out the following tags in Github
first-timers-only
hacktoberfest
for-new-contributors
Pre-requisites
● AWS account.● User with Identity and Access Management permission.
● uday with IAMFullAccess permission.● uday user’s SecretAccessKey and AccessKeyID as Environment Variables.
export AWS_ACCESS_KEY_ID=<access key>export AWS_SECRET_ACCESS_KEY=<secret key>
Deployment - Is It Easy?
OpenEBS GitHub Repo:● Clone OpenEBS GitHub repo.
https://github.com/openebs/openebs● Deployment Instructions and Automation Script.
○ https://github.com/openebs/openebs/tree/master/e2e/terraform○ OpenEBS cloud deployment script oebs-cloud.sh.
Steps to run the percona TPC-C benchmark
● Deploy the percona pod on kubernetes (percona.yaml)
● Setup the percona monitoring by deploying the PMM-server & registering the percona host into it (pmm-server.yaml)
● Create the TPC-C benchmark specifications via configmap (tpcc.conf)
● Prepare database, load data per spec & start benchmark (tpcc-bench.yaml)
❖ YAMLs : https://github.com/openebs/performance-benchmark
❖ Custom images : https://github.com/openebs/test-storage
Running Percona on OpenEBS
● Percona server is an instrumented & enhanced open-source drop in replacement for MySQL DB. Better performing InnoDB, called XtraDB
● Percona official image on dockerhub has 5M+ pulls
Monitoring Percona using PMM
● Percona Monitoring and Management (PMM) is an open-source platform from Percona for managing and monitoring MySQL performance
● PMM-Client sits on the DB host & collects server & general system metrics
● PMM-Server aggregates collected data & presents it on a dashboard.
Benchmarking Percona : TPC-C
● 1 - Company● N - Warehouses ● 100K Items / WH● 10 - Districts / WH● 3000 Customers / District● Stocks, History, Orders● User think times !
Next steps ?
● Resulting metric tpmC (New orders per minute) accounts for ~45% of the Transactions only
● Analyze results, repeat with new set of tpcc specs OR mysql/innodb startup variables !
● Doesn’t reveal everything about the underlying storage’s performance, but aids Apple to Apple comparison under standard operating environment
● The steps we have used are generic & can be applied with any storage volume !
● Let us collaborate !!
What do you think?
https://github.com/openebs
Join the community
#slackSlack.openebs.io
blog.openebs.io
@openebs