compass: yet another deployment system or an networking gear company’s embrace to openstack...

17
20131113日星期三 Shuo Yang Principal Architect of Cloud Computing, US R&D Center Compass A Streamlined OpenStack Deployment System

Upload: ow2-consortium

Post on 28-Nov-2014

1.002 views

Category:

Technology


2 download

DESCRIPTION

There is no lack of deployment software in the OpenStack market, and each one of them provides great value from different perspective and scenario. These deployment solutions did a great job on software deployment layer based upon OS provisioning tools like Cobbler and configuration management tools like Puppet/Chef, but they generally do not go deep down to networking and server hardware level configuration, which is a key step for large scale OpenStack solution. As part of a networking company and a server vendor, Huawei Cloud team are developing Compass, a system not only for OpenStack software deployment, but also for fully automated hardware level server and networking gear configuration. Our system automates hardware resource discovery, hardware configuration (e.g., hardware RAID configuration, switch configuration), topology-aware OpenStack service deployment and etc. Therefore, end users have a streamlined OpenStack deployment experience with Compass. We are using Compass for deploying OpenStack cloud in several deployments and got very constructive feedback for us to make more robust and open deployment solution. We plan to open source our project. One major goal we have in designing the system is to achieve true openness at hardware level, so that other hardware vendors can write plug-in toward their special server designs. Just like OpenStack’s openness makes it a valuable game-changing cloud solution, we hope we can work with other hardware vendors to make OpenStack universally available on various hardware platforms in a streamlined fashion.

TRANSCRIPT

Page 1: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

2013年11月13日星期三

Shuo Yang Principal Architect of Cloud Computing, US R&D Center

Compass – A Streamlined OpenStack Deployment System

Page 2: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

1

Outline of This Talk

DRY, Truly Open Deployment

Scope of Problem for Compass

Compass Explained

1

2

3

Page 3: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

2

A General System to Deploy Distributed Systems, Extensibility as a Primary Design Goal

Not Limited to OpenStack, but Streamlined Our OpenStack Deployment Like a Charm

100% Python, 5000 Line of Python Code

Successfully Deployed Several Dogfood Clusters

Compass Wiki Page: https://wiki.openstack.org/wiki/Compass

Compass at a Glimpse

Think Big, Start Small

To Be Open Sourced – Apache 2.0 Soon

Page 4: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

3

Open Cloud OS (OpenStack) Open Deployment (Compass)

OpenStack

OpenStack Control Channel

Cinder Quantum Nova

NIC CPU Disk CPU Server

(2285/1285)

Disk Storage (N8000/N900)

Switch Switch (5020)

LILO/GRUB/LiveCD for OpenStack

HW/SW Configuration Deployment Channel

Auto Deploy

Data Center as a Computer

CPU CPU

Disk Disk

Switch

Live CD /

GRUB

NIC

/dev eth0, lo /proc

Linux

Page 5: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

4

No.1 as Storage Revenue Growth

No. 2 as `x86 Server Revenue Growth

Needless to Say, Networking Gears..

Global Excellent Telecom Cloud Solution Provider of year 2012

Why We Are Doing This?

Full HW Portfolio in Data Center

OpenStack Makes the Above a Full Cloud Solution

Page 6: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

5

Outline of This Talk

DRY, Truly Open Deployment

Scope of Problem for Compass

Compass Explained

1

2

3

Page 7: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

6

Switch 1

Server 1

Host OS

Switch 2 Switch k

Server 2

Host OS

Server n

Host OS

IPMI

SNMP

Cobbler / Razor / …

Chef / Puppet / …

A Life of Deployment – How to Streamline All Controls

Plus a lot of scripts: Boilerplate Target system

description

Page 8: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

7

A Restful API service instead of a web app

Functional modules with plug-in architecture (e.g., 200 Loc to support HP Switch)

Carefully designed boundary to mature tools like Cobbler, Chef and etc.

Work WITH the above tools, not to replace any of them.

Compass Philosophies – Programmability and Extensibility

Programmability – To Be a Programmable Framework

Extensibility – NOT To Be A Conglomerate

Page 9: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

8

Huawei UI

DB

Cobbler plugin Razor plugin Chef plugin Ansible plugin

Package Deployment Hardware Discover

Huawei Plugin microkernel

OS Provisioning

Message bus (RabbitMQ)

Restful API Server

3rd Party ISV UI Rest Client

Compass -- Open Source

Compass Internals

Page 10: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

9

Currently we use Chef as deployment management Engine

Puppet or Ansible devopers, we want to work with you

Networking gear based bare metal discovery (topolology awareness)

Currently we are working on RAID Controller for Huawei

OCP community, what’s your thought

NOT to Be Conglomerate – Plug-in Architecture

Plug-in architecture for configuration management system

Plug-in architecture for Hardware vendors IPMI / SNMP

Page 11: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

10

Chef-Server

Compass

Cobbler-server

Policy Databag (e.g., exclusive; default) OpenStack Cookbook

snippets kickstart files

NOT to Be Conglomerate – Roles in the System

App writer

Lib writer

Restful Reqs

Page 12: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

11

Outline of This Talk

DRY, Truly Open Deployment

Scope of Problem for Compass

Compass Explained

1

2

3

Page 13: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

12

Distributed System

Hardware

HP

OS

Ubuntu

CentOS

Windows Server?

OpenStack Hadoop Ceph?

OCP?

ESXi ?

Huawei

Extensible in Use Cases

Page 14: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

13

OS Provisioning Tools

Cobbler

Razor

Chef Puppet Ansible Salt Configuration Management Tools

Extensible in Tool Chain

Page 15: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

14

To Summarize

General Purpose Deployment System (Ultimate Goal)

Deploys OpenStack Cloud Infrastructure Smoothly (Dogfoods Done)

Restful API: Allow Third-party UI Integration

Contact us: [email protected]

Santa Clara R&D Center, US

Open Sourced under Apache 2.0

Open to Collaboration

Page 16: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

15

Real Demo

Video Clips Recording Our First Deployment

Page 17: Compass: Yet Another Deployment System or An Networking Gear Company’s Embrace to OpenStack Community?, Shuo YANG, Huawei

Copyright©2013 Huawei Technologies Co., Ltd. All Rights Reserved.

The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product

portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive

statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time

without notice.