Storage is not virtualized enough

Zhipeng (Howard) HuangHuawei

Bio Standard Engineer

• ETSI NFV• SNIA Object Drive TWG• OCI/Spec

Open Source Community Operation Manager• OpenStack (Tricircle, Kingbird, Dragonflow, …)• OPNFV (Parser, Multisite, DPACC, …)• OpenVswitch

Like Heavy Metal Music and father of twin girls !

Why storage is not virtualized enough

Converged Data Plane（ Support File, Block, Object）

NoSQL1（ Graph）NoSQL2（ kv） OpenSwift Lustre …

app1

App admin

Interface block file object Kv db Graph db Mpp sql …

Performance Ops xxx Ops xxx Ops xxx Ops xxx …

Capacity 10PB 100PB 1000PB …

Reliability 5个 9 6个 9 7个 9 8个 9 …

Storage admin

Cost $1/GB $0.1/GB Free Service

Open Source …

Functionality Data Replica

Clone Backup Dedup …

Maintainability …

app2 app3 app4 app5 app…

What is Storage Function Virtualization (SFV)

SFVNFV

OS/Hypervisor

VM1

VNF1

VMn

VNFn…

OS/Hypervisor

VMx

VNFx

VMy

VNFy…

…

Cloud OS (NFV Enhanced)OS/Hypervisor

VM1

VSF1

VMn

VSFn…

OS/Hypervisor

VMx

VSFx

VMy

VSFy…

…

Cloud OS (SFV Enhanced)

FW DPI Router LB Blk/EBS MPP DB NAS Hadoop

OpenStack SFV

OpenStack Components

Third Party Storage Management Software

Third Party virtual storage apps

OpenStack SFV Enhancements

Cinder ManilaNova Glance

Heat

Murano

Virtualized Storage Funtions

OpenStack SFV

AMQP

REST

REST

REST

DriverDriver

Driver

REST

REST

REST

REST

App-Image

App-Template

DPS

REST

REST

Backup Driver

Storage Orchestration Tool

OpenStack SFV Sample Workflow

Pool’s volume、medium、 I

OPS

Nova Cinder-API

Cinder-scheduler

Cinder-volume（ LVM driver）Cinder-volume（ HSDP driver）

Pool/VG Pool/VG Pool/VG Pool Pool

HDD HDD HDD

LVM

SSD NVDIMM

HSDP

Create Instance（ bdm)

1

2 1. filter for localized compute and storage

2. filter for volume anti-affinity rule

3. filter for capability scheduling

3 4

Pool’s volume、medium、 IO

PS

NOVAscheduler

2

OpenStack SFV related BPs Improve nova attach volume api to support disk-bus : The purpose of this spec is to add disk-bus parameter to nova attach volume api. https://review.openstack.org/#/c/185973/

Add local-to-instance to create server api : The purpose of this spec is to add "local_to_instance" to block_device_mapping parameter in nova create instance API. https://review.openstack.org/#/c/190561/

Add volume type to create server API : Currently in create server API, using block_device_mapping parameter can create cinder volume and attach to instance, but there aren't volume type parameter to create a SLA based volume, so this bp will add the volume type to create api. https://review.openstack.org/#/c/187812/

Add NVME bus to KVM hypervisor https://blueprints.launchpad.net/nova/+spec/add-nvme-bus-to-kvm-hypervisor

vHost Supporthttps://blueprints.launchpad.net/cinder/+spec/vhost-support

Notes:Some of the BPs would be resubmitted for Release MCollaboration on SFV related works are highly welcomed.

SFV Goes Beyond OpenStack

OCI

CNCF OpenStack

OCP

KVM

AddMore

…

AddMore

…

ContainerFlockerApp 1

ContainerHyperApp 2

OSv-ScyllaDB

App N

AddMore

…

AddMore

…

Kinetic

• OpenStack-dev mailinglist with [SFV] in the subject • Join and Discuz at https://groups.google.com/forum/#!forum/sfv-dev • Also hack ideas for development in various upstream projects at https://github.com/OpenSFV• More trendy communication at https://sfvdev.slack.com/