openATTIC Technology Overview - Ceph ManagementLenz Grimmer <lenz.grimmer@it-novum.com>Vault Conference 2016, Raleigh (NC), USA

2

openATTIC – History / Motivation / Background

─ Development started ~5 years ago

─ Develop a free alternative to commercial / traditional

storage management systems

─ Based on Linux / OSS tools

─ Development sponsored by it-novum

─ „Traditional Storage“ and Scale-out

3

─ Modern Web UI─ Software-Defined Storage:

RESTful API─ Unified Storage

─ NAS (NFS, CIFS)─ SAN (iSCSI, FC)

─ LVM, XFS, ZFS, Btrfs, ext3/4

─ Ceph Support: CRUSH map, Pool/RBD management (WIP)

─ Mirroring (DRBD®)─ Clustering

(Pacemaker/Corosync)─ Monitoring (Nagios) built-in─ Open Source & commercial

Support

openATTIC – Open Source Storage Management

4

openATTIC Components – Backend

─ Django (Python)─ Django REST Framework (RESTful API)─ Various Linux tools for storage management, e.g. LVM, LIO,

filesystem utilities, DRBD, etc.─ Nagios/Icinga & PNP4Nagios (Monitoring and Graphing)

5

openATTIC Components – Frontend

─ AngularJS (JS framework)─ Bootstrap (HTML, CSS, and JS framework)─ Uses REST API exclusively─ Bower (JS package manager, NodeJS)─ Grunt (Automation, NodeJS)─ Protractor / Jasmine (UI Test framework)

6

Django REST Framework

Backend/Models(Django)

Client (GUI)

oA-systemd

REST API

D-Bus

Node 1

OS Commands

Django REST Framework

Backend/Models(Django)

Custom App

oA-systemd

REST API

D-Bus

Node 2

OS Commands

MultinodeInterconnect

openATTIC – Architecture

DB(PostgreSQL)

7

openATTIC – Ceph Management Challenges

─ How to find the best approach for managing Ceph─ An ISV‘s view

─ Distributed systems vs. storing configuration data in a local

database─ Allow external modifications / administration of the Cluster─ How to maintain consistency of a local configuration DB?

8

openATTIC – Choosing the right Ceph API

─ Option 1: Calling CLI tools and parsing output (JSON-

format)

─ Option 2: Use existing APIs and libs, e.g.─ librados (Python bindings)─ Ceph-REST-API─ Calamari Server

─ API documentation

─ Redundancy / Scalability / Dependencies

─ Which is the best choice?

9

openATTIC – Other Ceph Management Tools

─ How are others doing this?

─ Calamari Server / Romana

─ Intel VSM

─ Inkscope / Kraken / Ceph-Dash

10

openATTIC - Ceph Management Implementation

─ Based on librados Python bindings─ Low overhead

─ Implemented “NoDB” Django Models─ All benefits of Django models (e.g. relations, methods, pagination)─ REST API viewsets─ Query Ceph for the model state instead of local DB

─ Use Salt for remote execution on nodes

11

Django REST Framework

Django Models

Client (GUI)

REST API

openATTIC – “nodb” Django models

DB(PostgreSQL)

nodb

Python librados django.db.models

12

openATTIC - Current Development Status

─ “Nodb” Architecture / Framework in place─ Pool Management REST API

─ List Pools, view Pool details─ Create/Delete/Update (WIP)

─ Pool Management Web UI (table view)

13

openATTIC – Future/ongoing Development Work

─ Ceph Cluster Status Dashboard─ Extend Pool Management (Wizard)─ OSD Monitoring/Management─ RBD Management─ CephFS Management─ RGW Management (users, buckets keys)

14

openATTIC – Dashboard

15

openATTIC – Ceph Pool List

16

openATTIC – Ceph CRUSH map editing

17

openATTIC – Wizards

18

openATTIC - Volume View

19

openATTIC - Volume View (Performance)

20

openATTIC – API Recorder

21

www.openattic.org demo.openattic.org blog.openattic.org docs.openattic.org

bitbucket.org/openattic tracker.openattic.org @openATTIC openATTIC

openATTIC – Resources

22

Questions / Discussion

23

Thank you!