Ceph and Storage

Management with openATTICCeph Tech Talks

2016-06-23

Lenz Grimmer <lenz@openattic.org>

openATTIC – Our Vision

─ Develop an open source alternative to proprietary storage

management systems

─ „Traditional” unified storage (NAS/SAN)

─ Support Ceph for scale-out scenarios

─ Backed with commercial support and services

openATTIC – Notable Recent Changes

─ Removed Enterprise/Community Edition split

─ Now fully under the GPLv2

─ Removed requirement for CLA

─ DCA (Signed-off-by) is all that's needed

─ Public Jira Bug Tracker

─ Public Pull requests / code reviews on BitBucket

─ Development/Release Branch

─ Entire code base (Backend/WebUI/Test/Docs) in one branch

openATTIC – What Sets us Apart?

─ Focus on data center storage management

─ Support both SAN and NAS functionality without limitations

─ Fully Open Source (GPLv2)

─ No arbitrary functional restrictions

─ Low entrance barrier for adoption

─ Based on Linux / OSS tools

─ Multiple Linux distributions (Debian/Ubuntu/Red Hat/SUSE)

─ Well-established technology stack (e.g. drivers, hardware support)

─ Broad user base

─ Modern Web UI

─ RESTful API (Software-

Defined Storage)

─ Unified Storage─ NAS (NFS, CIFS, HTTP)

─ SAN (iSCSI, Fibre Channel)

─ LVM, XFS, ZFS, Btrfs,

ext3/4

─ Volume mirroring (DRBD®)

─ Multi-node support

─ Monitoring (Nagios/Icinga)

built-in

─ Ceph management and

monitoring (WIP)

─ Development sponsored by

it-novum

openATTIC – Open Source Storage Management

─ Backend─ Python (Django)

─ Django REST Framework

(RESTful API)

─ Linux tools for storage

management, e.g. LVM, LIO,

filesystem utilities, DRBD,

etc.

─ Nagios/Icinga & PNP4Nagios

(Monitoring and Graphing)

─ Web Frontend─ AngularJS (JS framework)

─ Bootstrap (HTML, CSS, and

JS framework)

─ Uses REST API exclusively

─ Automated Test Suites

─ Python Unit Tests

─ REST Backend Tests

(Gatling)

─ WebUI Tests

(Protractor/Jasmine)

openATTIC – Components

openATTIC – High Level Architecture

openATTIC – Ceph Management Challenges

─ Managing and monitoring Ceph is complex

─ Many tools exist (e.g. Calamari, Intel VSM, ceph-dash)

─ Limited functionality, unclear roadmaps

─ Finding the best approach for managing Ceph

openATTIC – Ceph Management Goals

─ Create a management & monitoring GUI tool

─ A tool that administrators actually want to use

─ That scales without becoming overwhelming

─ Still should allow changes to be made elsewhere, without

becoming inconsistent

openATTIC – Ceph Management Implementation

─ Which Ceph Management API?

─ How to manage a distributed system?

─ How to monitor the cluster‘s health/performance?

─ How to perform remote management tasks?

─ How to monitor cluster nodes in a scalable way?

Django REST Framework

Django Models

Client (GUI)

REST API

DB

(PostgreSQL)

nodb

Python librados / librbddjango.db.models

openATTIC – “NoDB” Django models

openATTIC – Current Ceph Development Status

─ “NoDB” backend architecture / framework in place

─ Create and map RBDs as block devices (volumes)

─ Pool Management Web UI (table view)

─ OSD Management Web UI (table view)

─ RBD Management Web UI (table view)

─ Monitor a cluster health and performance

─ CRUSH Map Editor

─ Support for managing multiple Ceph clusters

openATTIC – Ceph REST API Overview

openATTIC – Ceph Development Roadmap

─ Ceph Cluster Status Dashboard incl. Performance Graphs

─ Extend Pool Management

─ OSD Monitoring/Management

─ RBD Management/Monitoring

─ CephFS Management

─ RGW Management (users, buckets keys)

─ Deployment, remote configuration of Ceph nodes (via Salt)

─ Public Roadmap on the openATTIC Wiki to solicit

community feedback: http://bit.ly/28PCTWf

openATTIC – Storage Management Dashboard

openATTIC – Ceph Pool List

openATTIC – Ceph OSD List

openATTIC – Ceph CRUSH map editing

openATTIC – Volume Management

openATTIC – API Recorder

openATTIC – Attending SUSE Hackweek

─ June 24th

– July 1st

─ https://hackweek.suse.com/

─ Pick open Jira issues

─ Join #openattic on Freenode

─ Submit your project

─ Have fun, learn new stuff

─ openATTIC Team will be in the SUSE HQ in Nuremberg

─ www.openattic.org─ demo.openattic.org─ blog.openattic.org─ docs.openattic.org

─ bitbucket.org/openattic─ tracker.openattic.org─ @openATTIC─ openATTIC

openATTIC – Resources

Questions / Discussion

Thank you!