Ceph and Storage Management with openATTICSUSE Monthly Open Source Talks

2016-07-06

Lenz Grimmer <lenz@openattic.org>

2

openATTIC – Our Vision

─Develop an open source alternative to proprietary

storage management systems

─„Traditional” unified storage (NAS/SAN)

─Support Ceph for scale-out scenarios

─Backed with commercial support and services

3

openATTIC – Notable Recent Changes

─ Removed Enterprise/Community Edition split

─ Now fully under the GPLv2

─ Removed requirement for CLA

─ DCA (Signed-off-by) is all that's needed

─ Public Jira Bug Tracker

─ Public Pull requests / code reviews on BitBucket

─ Development/Release Branch

─ Entire code base (Backend/WebUI/Test/Docs) in one branch

4

openATTIC – What Sets us Apart?

─ Focus on data center storage management

─ Support both SAN and NAS functionality without limitations

─ Fully Open Source (GPLv2)

─ No arbitrary functional restrictions

─ Low entrance barrier for adoption

─ Based on Linux / OSS tools

─ Multiple Linux distributions (Debian/Ubuntu/Red Hat/SUSE)

─ Well-established technology stack (e.g. drivers, hardware support)

─ Broad user base

5

─Modern Web UI

─RESTful API (Software-Defined Storage)

─Unified Storage─NAS (NFS, CIFS, HTTP)─SAN (iSCSI, Fibre Channel)

─LVM, XFS, ZFS, Btrfs, ext3/4

─Volume mirroring (DRBD®)

─Multi-node support

─Monitoring (Nagios/Icinga) built-in

─Ceph management and monitoring (WIP)

─Development sponsored by it-novum

openATTIC – Open Source Storage Management

6

─Backend─Python (Django)─Django REST Framework

(RESTful API)─Linux tools for storage

management, e.g. LVM, LIO, filesystem utilities, DRBD, etc.

─Nagios/Icinga & PNP4Nagios (Monitoring and Graphing)

─Web Frontend─AngularJS (JS framework)─Bootstrap (HTML, CSS, and

JS framework)─Uses REST API exclusively

─Automated Test Suites─Python Unit Tests─REST Backend Tests

(Gatling)─WebUI Tests

(Protractor/Jasmine)

openATTIC – Components

7

openATTIC – High Level Architecture

8

openATTIC – Installation on SUSE Linux

─OBS Project filesystems:openATTIC

─Packages available for openSUSE Leap 42.1 & SLES12

─Thanks to Eric Jackson (swiftgist) for the support!

─Feedback is welcome!

9

openATTIC – Storage Development Roadmap

─ Add Disk and Storage Pool Management to the API & WebUI

─ Creating/Modifying LVM Volume Groups / MD RAID setups

─ Creating/Modifying Btrfs/ZFS Pools (incl. RAID setups)

─ Automatic discovery of disks/pools (via udev)

─ Monitoring Disk health (SMART)

─ Manage HW RAID controllers

─ Add volume mirroring support to the WebUI

─ Extend SAN functionality (more iSCSI/FC features)

─ Public Roadmap on the openATTIC Jira/Wiki to solicit community feedback

10

openATTIC – Ceph Management Challenges

─Managing and monitoring Ceph is complex

─Many tools exist (e.g. Calamari, Intel VSM, ceph-dash)

─Limited functionality, unclear roadmaps

─Finding the best approach for managing Ceph

11

openATTIC – Ceph Management Goals

─Create a management & monitoring GUI tool

─A tool that administrators actually want to use

─That scales without becoming overwhelming

─Still should allow changes to be made elsewhere, without

becoming inconsistent

12

openATTIC – Ceph Management Implementation

─Which Ceph Management API?

─How to manage a distributed system?

─How to monitor the cluster‘s health/performance?

─How to perform remote management tasks?

─How to monitor cluster nodes in a scalable way?

13

Django REST Framework

Django Models

Client (GUI)

REST API

DB

(PostgreSQL)

nodb

Python librados / librbddjango.db.models

openATTIC – “NoDB” Django models

14

openATTIC – Current Ceph Development Status

─ “NoDB” backend architecture / framework in place

─ Create and map RBDs as block devices (volumes)

─ Pool Management Web UI (table view)

─ OSD Management Web UI (table view)

─ RBD Management Web UI (table view)

─ Monitor a cluster health and performance

─ CRUSH Map Editor

─ Support for managing multiple Ceph clusters

15

openATTIC – Attending SUSE Hackweek

─ June 24th – July 1st

─ https://hackweek.suse.com/

─ openATTIC Team visited SUSE HQ in Nuremberg

─ Implementation design decisions

─ Salt integration

─ New features implemented

─ Ceph Monitoring Dashboard

─ Ceph RBD Management

16

openATTIC – Ceph Management REST API

https://wiki.openattic.org/display/OP/openATTIC+Ceph+REST+API+overview

17

openATTIC – Ceph Development Roadmap

─ Ceph Cluster Status Dashboard incl. Performance Graphs

─ Extend Pool Management

─ OSD Monitoring/Management

─ RBD Management/Monitoring

─ CephFS Management

─ RGW Management (users, buckets keys)

─ Deployment, remote configuration of Ceph nodes (via Salt)

─ Public Roadmap on the openATTIC Wiki to solicit community

feedback: http://bit.ly/28PCTWf

18

openATTIC – Storage Management Dashboard

19

openATTIC – Volume Management

20

openATTIC – API Recorder

21

openATTIC – Ceph Pool List

22

openATTIC – Ceph OSD List

23

openATTIC – Ceph RBD list

24

openATTIC – Create Ceph RBD

25

openATTIC – Ceph CRUSH map editing

26

─ www.openattic.org

─ demo.openattic.org

─ blog.openattic.org

─ docs.openattic.org

─bitbucket.org/openattic

─tracker.openattic.org

─@openATTIC

─openATTIC

openATTIC – Resources

SUSE CERTIFIED LINUX PROFESSIONAL ACADEMYTechnical Training for SUSE Linux Enterprise Server 12

No Cost Flexible 12-Week Online Training This popular training model is for technical sales/engineering employees of

SUSE partners interested in learning to install and manage SUSE Linux

Enterprise Server 12 (SLES 12). Participation in our SUSE Academy also

offers no-cost exam vouchers for participants seeking SUSE certification.

Learn through self-study of the SLES 12 curriculum and weekly webinar

reviews with our field-experienced instructor.

Each Academy participant will need 5-7 hours per week to self-study the

SLES 12 course assignments, plus 1 hour each week to attend a webinar

review of the key concepts studied. Certification is not required to participate.

Contact your company’s Partner Executive to ask to be sponsored in our

next Academy.

28

Questions / Discussion

29

Thank you!