Interoperability with ML2: LinuxBridge, OVS and SDN

Sungman Jang (LinuxBridge& OVS)

Guohua Lee (SDN)

ML2 with Linux Bridge & OVS

Part 1

Environment• Host: HP Elitebook 820

G1 (Intel i5-4300U, 2.50GHz / 12GB RAM) / Windows 7 Enterprise

• VirtualBox: 5.0.6 with Extension Pack installed

• Guest: Ubuntu x64 14.04

• OpenStack: Liberty (Provider Net-work)

• 설치목차• 1. 기본 프로그램 설치 (python-open-

stackclient chrony) 및 네트워크 인터페이스 설정2. 컨트롤러 전용 프로그램 설치(mariasql, rabbitmq)3-1. Keystone 설치3-2. 기본 계정 설정하기 (Admin/Demo)3-3. Keystone 설치 후 동작 확인4-1. Glance 설치4-2. Glance 설치 후 동작 확인

• -----------• 5-1. Nova 설치• 5-2. Nova 설치 후 동작 확인• 6-1. Neutron 설치• 6-2. Neutron 설치 후 동작 확인• 7. Horizon 설치

Provider Networks• Provider networks generally of -

fer simplicity, performance, and reliability at the cost of flexibility.

• Operators can build on this min-imal deployment to enable more cloud networking features.

공유기 =

<OpenStack Networking Guide> <VirtualBox 로 구현한 네트워크 구조 >

Why LinuxBridge and OVS?• LinuxBridge (since 1999/01/25):

– A Linux bridge is more powerful than a pure hard-ware bridge because it can also filter and shape traffic. The combination of bridging and firewalling is done with the companion project ebtables.

• Open vSwitch (since 2009/07/29)– It is designed to enable massive network automa-

tion through programmatic extension, while still supporting standard management interfaces and protocols (e.g. NetFlow, sFlow, IPFIX, RSPAN, CLI, LACP, 802.1ag). 

Why OVS so complicated?

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/5/html/Cloud_Administrator_Guide/section_networking-scenarios.html

• Ideally, the TAP device vnet0 would be connected directly to the integration bridge, br-int. Unfortunately, this isn't possible because of how Open-Stack security groups are currently implemented. OpenStack uses iptables rules on the TAP devices such as vnet0 to implement security groups, and Open vSwitch is not compatible with iptables rules that are applied directly on TAP devices that are connected to an Open vSwitch port.

• Networking uses an extra Linux bridge and a veth pair as a workaround for this issue. Instead of connecting vnet0 to an Open vSwitch bridge, it is connected to a Linux bridge, qbrXXX. This bridge is connected to the integration bridge, br-int, through the (qvbXXX, qvoXXX) veth pair.

LinuxBridgeOpenvSwitch

What is ML2• The Modular Layer 2 (ML2) Plugin is a framework allowing Open-

Stack Networking to simultaneously utilize the variety of layer 2 networking technologies found in complex real-world datacenters.

POST /v2.0/ports.json HTTP/1.1content-type: application/json

{ "port": { "admin_state_up": true, "device_id": "d6b4d3a5-c700-476f-b609-1493dd9dadc0", "name": "port1", "network_id": "6aeaf34a-c482-4bd3-9dc3-7faf36412f12" }}

LB Agent

Expected Scenario

/etc/neutron/plugins/ml2/ml2_conf.ini[ml2] type_drivers = flat,vlan,gre,vxlan tenant_network_types = mechanism_drivers = linuxbridge,openvswitch

/etc/neutron/dhcp_agent.ini[DEFAULT] verbose = True interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq dhcp_delete_namespaces = True

Mech Driver:Linuxbridge,Openvswitch+Type Driver:flat,vlan,gre,vxlan

Mech Driver:Linuxbridge,+Type Driver:vxlan

Mech Driver:Openvswitch+Type Driver:vxlan

neutron.agent.linux.interface.OVSInterfaceDriver

Oh… No…• /etc/neutron/neutron.conf• /etc/neutron/plugins/ml2/ml2_conf.ini• /etc/neutron/dhcp_agent.ini

APPENDIX

Bridge, Switch and Router• Bridge: A network bridge is network hardware that connects multiple network

segments.• Switch: A network switch is a multiport network bridge that uses hardware addresses

to process and forward data at the data link layer (layer 2) of the OSI model.• Router: A router is connected to two or more data lines from different networks.

When multiple routers are used in interconnected networks, the routers exchange information about destination addresses using a dynamic routing protocol.

주요 특징 Classical Router Layer 3 Switch

주요 OSI Layer Layer 3 Layer 3

Routing 방법 Software (CPU + Software) Hardware (ASIC chip)

L2 지원 Ethernet, TokerRing, FDDI, ATM, WAN FastEthernet, Gigabit Ethernet

forwarding 성능 Slow Fast

Latency 약 200 ms < 10 ms (100 Mbps)

관리 /program 가능 매우 높음 적음지원 Protocol All IP ( 일부 IPX)

Routing Protocol All RIP1,2 OSPF 등WAN 지원 지원함 지원하지 않음

비용 높음 낮음표 출처 : http://koreahojin.egloos.com/v/2341317

Booting a VM• 0. _build_instance() & _allocate_network()• 1. REST (C->N): Create Port• 2. vif_driver.plug() & Add a port tap with external_ids set

– ‘Neutron OVS agent’ loop to detect port update– Get the Neutron port id from the external_ids– Call get_device_details(port_id) to Neutron Server

• 3. Bind_port() & Set up the ovs port• 4. update_device_up()

ML2 in Codeneutron.ml2.type_drivers =

flat = neutron.plugins.ml2.drivers.type_flatlocal = neutron.plugins.ml2.drivers.type_localvlan = neutron.plugins.ml2.drivers.type_vlangeneve = neutron.plugins.ml2.drivers.type_genevegre = neutron.plugins.ml2.drivers.type_grevxlan = neutron.plugins.ml2.drivers.type_vxlan

neutron.ml2.mechanism_drivers = ovsvapp = neutron.plugins.ml2.drivers.ovsvapp.mech_driveropendaylight = neutron.plugins.ml2.drivers.opendaylight.driverlogger = neutron.tests.unit.plugins.ml2.drivers.mechanism_loggertest = neutron.tests.unit.plugins.ml2.drivers.mechanism_testlinuxbridge =

neutron.plugins.ml2.drivers.linuxbridge.mech_driver.mech_linuxbridgeopenvswitch =

neutron.plugins.ml2.drivers.openvswitch.mech_driver.mech_openvswitchhyperv = neutron.plugins.ml2.drivers.hyperv.mech_hypervl2population = neutron.plugins.ml2.drivers.l2pop.mech_driverofagent = neutron.plugins.ml2.drivers.ofagent.drivermlnx = neutron.plugins.ml2.drivers.mlnx.mech_mlnxbrocade = networking_brocade.vdx.ml2driver.mechanism_brocadebrocade_fi_ni =

neutron.plugins.ml2.drivers.brocade.fi_ni.mechanism_brocade_fi_nifslsdn = neutron.plugins.ml2.drivers.freescale.mechanism_fslsdnsriovnicswitch = neutron.plugins.ml2.drivers.mech_sriov.mech_driver.mech_driverfake_agent = neutron.tests.unit.plugins.ml2.drivers.mech_fake_agentsdnve = neutron.plugins.ml2.drivers.ibm.mechanism_sdnve

neutron.ml2.extension_drivers = test = neutron.tests.unit.plugins.ml2.drivers.ext_testtestdb = neutron.tests.unit.plugins.ml2.drivers.ext_testport_security = neutron.plugins.ml2.extensions.port_securityqos = neutron.plugins.ml2.extensions.qos neutron\plugins\ml2\driver_api.pysetup.cfg

ML2 in Code

neutron\plugins\ml2\driver_api.pyquantum-2013.1.5.tar\dist\quantum-2013.1.5\quantum\plugins\linuxbridge\lb_quantum_plugin.py

RPC: Agent->Plugin

RPC: Plugin->Agent

class OpenDaylightMechanismDriver(driver_api.MechanismDriver):

"""Mechanism Driver for OpenDaylight.

This driver was a port from the NCS MechanismDriver. The API exposed by ODL is slightly different from the API exposed by NCS, but the general concepts are the same. """

def initialize(self): self.url = cfg.CONF.ml2_odl.url self.timeout = cfg.CONF.ml2_odl.timeout self.username = cfg.CONF.ml2_odl.username self.password = cfg.CONF.ml2_odl.password required_opts = ('url', 'username', 'password') for opt in required_opts: if not getattr(self, opt): raise cfg.RequiredOptError(opt, 'ml2_odl')

self.odl_drv = OpenDaylightDriver()

# Postcommit hooks are used to trigger synchronization.

def create_network_postcommit(self, context): self.odl_drv.synchronize('create', odl_const.ODL_NETWORKS, context)

ML2 for ODLfrom networking_odl.common import constants as odl_constfrom networking_odl.ml2 import mech_driverfrom oslo_config import cfgfrom oslo_log import log

from neutron.plugins.ml2 import driver_api as api

https://github.com/openstack/networking-odl

(ODL backend implementation)

def sendjson(self, method, urlpath, obj): """Send json to the OpenDaylight controller."""

headers = {'Content-Type': 'application/json'} data = jsonutils.dumps(obj, indent=2) if obj else None url = '/'.join([self.url, urlpath]) LOG.debug("Sending METHOD (%(method)s) URL (%(url)s) JSON (%(obj)s)", {'method': method, 'url': url, 'obj': obj}) r = requests.request(method, url=url, headers=headers, data=data, auth=self.auth, timeout=self.timeout) r.raise_for_status()

def sync_single_resource(self, operation, object_type, con-text): """Sync over a single resource from Neutron to OpenDay-light.

Handle syncing a single operation over to OpenDaylight, and correctly filter attributes out which are not required for the req-uisite operation (create or update) being handled. """ # Convert underscores to dashes in the URL for ODL object_type_url = object_type.replace('_', '-') try: obj_id = context.current['id'] if operation == odl_const.ODL_DELETE: self.out_of_sync |= not self.client.try_delete( object_type_url + '/' + obj_id) else: filter_cls = self.FILTER_MAP[object_type] if operation == odl_const.ODL_CREATE: urlpath = object_type_url method = 'post' attr_filter = filter_cls.filter_create_attributes elif operation == odl_const.ODL_UPDATE: urlpath = object_type_url + '/' + obj_id method = 'put' attr_filter = filter_cls.filter_update_attributes resource = context.current.copy() attr_filter(resource, context) self.client.sendjson(method, urlpath, {object_type_url[:-1]: re-source})

def synchronize(self, operation, object_type, context): """Synchronize ODL with Neutron following a configuration change.""" if self.out_of_sync: self.sync_full(context._plugin) else: self.sync_single_resource(operation, object_type, con-text)

ML2 for ODLdef create_network_postcommit(self, context): self.odl_drv.synchronize('create', odl_const.ODL_NETWORKS, context)

self.context = context.get_admin_context()

curl -X POST -H "Content-Type: application/json" -u admin:admin http://localhost:8080/controller/nb/v2/neutron/networks -d '{ "networks": [ { "id": "d17696ea-1d2f-42ff-9a8f-895eca97b5ed", "name": "Network-1", "tenant_id": "93397de8-7b3c-4450-a700-40d5d0120e89","router:external": true,"shared": true} ]}'

https://github.com/openstack/networking-odl

ML2 for ONOSdef create_network_postcommit(self, context): entity_path = 'networks' resource = context.current.copy() onos_utils.send_msg(self.onos_path, self.onos_auth, 'post', entity_path, {'network': re-source})

def send_msg(onos_path, onos_auth, msg_type, entity_path, entity=None): """Send message to the ONOS controller."""

path = '/'.join([onos_path, entity_path]) hdr = {'Content-Type': 'application/json'} body = jsonutils.dumps(entity, indent=2) if entity else None LOG.debug("Sending MSG_TYPE (%(msg)s) URL (%(path)s) " "OBJECT (%(entity)s) BODY (%(body)s)", {'msg': msg_type, 'path': path, 'entity': entity, 'body': body}) req = requests.request(method=msg_type, url=path, headers=hdr, data=body, auth=onos_auth) # Let's raise voice for an error req.raise_for_status()

def __init__(self): self.onos_path = cfg.CONF.onos.url_path self.onos_auth = (cfg.CONF.onos.username, cfg.CONF.onos.-password) self.vif_type = portbindings.VIF_TYPE_OVS self.vif_details = {portbindings.CAP_PORT_FILTER: True}

class Network(model_base.BASEV2, HasId, HasTenant): """Represents a v2 neutron network."""

name = sa.Column(sa.String(attr.NAME_MAX_LEN)) ports = orm.relationship(Port, backref='networks') subnets = orm.relationship( Subnet, backref=orm.backref('networks', lazy='joined'), lazy="joined") status = sa.Column(sa.String(16)) admin_state_up = sa.Column(sa.Boolean) mtu = sa.Column(sa.Integer, nullable=True) vlan_transparent = sa.Column(sa.Boolean, nullable=True) rbac_entries = orm.relationship(rbac_db_models.NetworkRBAC, backref='network', lazy='joined', cascade='all, delete, delete-orphan')

https://github.com/openstack/networking-onos

ONOS JSON REST API

ML2 with SDNPart 2

SDN, OpenFlow, SDN ControllerHow to integrate SDN controller with OpenStack?Demo

Contents

SDN(Software Defined Networking) 개념소프트웨어 정의 네트워킹으로 불리 우며 개방형 API( 예를 들어 OpenFlow)를 통해 네트워크 장비에서 하드웨어 기능과 소프트웨어 기능을 분리하여 소프트웨어 기반 컨트롤러에 의해 제거 / 관리하는 접근 방식임

SDN Architecture

SDN, OpenFlow, SDN Controller

Infrastruc-ture

Layer

Network Control Layer

ApplicationLayer

Network Aware Application

SDN Controller(e.g., onos, opendaylight, floodlight …)

Physical Network Device

Virtual Network Device

Control Data Plane Inter-face(e.g., OpenFlow)

Network InformationAPI APINetwork Control

* 출처 : “OpenFlow 를 이용한 SDN 입문”

Control Plane

Data Plane

Neutron + SDN Controller

Neutron Components192.168.0.0/24

192.168.1.0/24

117.16.149.0/24https://github.com/openstack/networking-onosMechanism driver: ONOSMechanismDriver (instead of OpenvswitchMechanismDriver) Service plugin: ONOSL3Plugin(instead of L3Router-Plugin)Detail Configuration: http://heavenkong.blogspot.kr/2015/12/integrate-onosemu-with-openstack-kilo.html

xx x수정

Five VMs or host machines are needed

Prerequisite

No usage CPU Memory Disk

1 OpenStack Con-troller Node

2 Core 4 GB 20 GB

2 OpenStack Net-work Node

2 Core 4 GB 20 GB

3 ONOS Controller Node

2 Core 4 GB 20 GB

4 Compute Node1 1 Core 2 GB 20 GB

5 Compute Node2 1 Core 2 GB 20 GB

*Reference: SONA User Guide https://wiki.onosproject.org/display/ONOS/How+to+set+up+environment

System ConfigurationController/Network/ONOS Node Compute Node

Supporting ServicesDatabase

MySQLMessage Broker

rabbitmq

Basic ServicesIdentity: Key-

stoneImage Service: Glance

Compute: Nova mgmt

Networking: Neutron Server

Dashboard: Horizon

ONOS ControllerJava 8

Karaf & mvn onos (emu)

Basic ServicesCompute:

Nova HypervisorKVM/libvirt

Networking: Ml2 Plug-inOVS Agent

Network InterfaceManagement

(192.168.0.3/24)

External (117.16.149.115/24)

Network InterfaceManagement

(192.168.0.4/24) Instance Tunnels(192.168.1.3/24)

Instance Tunnels(192.168.1.4/24)

Networking: ONOS Ml2 Plug-in

OVS AgentLayer 3 Agent DHCP Agent

Metadata Agent

Openstackswitching from SONA architecture

https://wiki.onosproject.org/display/ONOS/SONA+ArchitectureThey only support switching part They didn’t support gre tunnel

Networking-onos 는 nova 로 부터 ONOS사이 REST API 형식으로 전달해주는 작용을 함 Openstackswitching 은 REST API 를 받아 instance 들을 위한 가상 네트워크를 만들어 줌

Openstackswithcing app uses the REST API call from onos-networking plugin only when pushing the DHCP mapping(VM IP-MAC pair), and it pushes flow rules when VMs are created and PORT_ADDED/UPDATED events occurs

We need both networking-onos and openstackswitching app

Demo

https://youtu.be/S-ANMA5EC24

Instance can not receive fixed IP It means ONOS can not receive fixed IP

Reference: http://uni2u.tistory.com/58http://blog.night9.cc/onos-on-centos7/https://www.youtube.com/watch?v=qP8nPYhz_Mohttps://wiki.onosproject.org/display/ONOS/Simple+workthrough

http://forum.onosfw.com/t/how-to-integrate-onos-master-1-4-0-with-openstack-kilo/80

https://github.com/hyunsun/documentations/wiki/Neutron-ONOS-Integration-for-CORD-VTN

Work to be continued …

THANK YOU