Download - Nvp deep dive_session_cee-day
NVP “Deep Dive” Yves Fauser
Network Virtualiza3on Pla6orm System Engineer
(slides prepared by Bad Hedlund & Dan Wendlandt)
OpenStack CEE Day 2013
Network Virtualiza6on
Hardware
SoMware
Logical Switches
SOFTWARE VIRTUALIZATION LAYER
Logical Routers
x86 Machine
NIC CPU
HD RAM
Network
L3 Router L2 Switch
Security Profiles
Virtual Network
vCPU
vRAM
vNIC image
Virtual Machine AUTOMATE
REPRODUCE
DECOUPLE Load Balancer
VIP
Load Balancer
OpenStack CEE Day 2013
A technical defini6on of “network virtualiza6on”
• Network virtualiza3on is: • A reproducKon of physical networks:
• Q: Do you have L2 broadcast / mul3cast, so apps do not need to be modified? • Q: Do you have the same visibility and control over network behavior?
• A fully isolated environment: • Q: Could two tenants decide to use the same RFC 1918 private IP space? • Q: Could you clone a network (IPs, MACs, and all) and deploy a second copy?
• Physical network locaKon independent: • Q: Can two VMs be on the same L2 logical network, while in different physical L2 networks? • Q: Can a VM migrate without disrup3ng its security policies, packet counters, or flow state?
• Physical network state independent: • Q: Do physical devices need to be updated when a new network/workloads is provisioned? • Q: Does the applica3on depend on a feature in the physical switch specific to a vendor? • Q: If a physical device died and was replaced, would applica3on details need to be known?
• Network virtualiza3on is NOT: • Running network func3onality in a VM (e.g., Router or Load-‐balancer VM)
OpenStack CEE Day 2013
Introducing NVP
• NVP “Network Virtualiza3on Pla6orm” • Compa3ble with KVM, XenServer, and VMware hypervisors. • NVP 1.0 release in July 2011 (prod deployments for 2 years) • Network pla6orm for largest produc3on OpenStack deployment, and many others… • 4 new releases per year (soMware is ea3ng the world…) • Current release is NVP 3.1 (Q2 release)
OpenStack CEE Day 2013
Physical Network
Hypervisors + OVS L2/L3 Gateways
Service Nodes
NVP Control Plane
NVP API
Quantum & Quantum API
Mgmt & Operator Tools
The NVP “Stack”
OpenStack CEE Day 2013
Compute
Service Nodes
OVS
NVP Manager
Controller Controller NVP Controller
Hypervisor 1 Hypervisor 2 Hypervisor N
L3 Gateway L2 Gateway L3 Gateway
OVS OVS OVS
OVS OVS OVS
L2 Gateway
OVS
Service Nodes
OVS
Tenants
NVP API
Internet
WEB WEB APP DB DB APP
Operators
L3 Fabric
…
Bare Metal VLANs
Remote Site
Physical (Non-‐virtualized)
View
OpenStack CEE Day 2013
L Switch
WEB
WEB
L Switch L Switch
L Router
APP
APP
DB
DB
Security QoS Monitoring
NAT
Virtual Network
World
Logical (Virtualized)
View
L Switch
WEB
WEB
L Switch L Switch
L Router
APP
APP
DB
DB
Security QoS Monitoring
NAT
Virtual Network
World
L Switch
WEB
WEB
L Switch L Switch
L Router
APP
APP
DB
DB
Security QoS Monitoring
NAT
Virtual Network
World
L Switch
WEB
WEB
L Switch L Switch
L Router
APP
APP
DB
DB
Security QoS Monitoring
NAT
Virtual Network
World
L Switch
WEB
WEB
L Switch L Switch
L Router
APP
APP
DB
DB
Security QoS Monitoring
Virtual Network
Remote Site
OpenStack CEE Day 2013
Physical Network
Hypervisors + OVS L2/L3 Gateways
Service Nodes
NVP Control Plane
NVP API
Quantum & Quantum API
Mgmt & Operator Tools
The NVP “Stack”
OpenStack CEE Day 2013
Treat your physical network like you treat your compute servers
• One big pool of resource capacity to be sliced up on-‐demand for tenants. • Rely on only commodity features (L3 forwarding) to enable vendor flexibility. • ConfiguraKon is done once when the devices are racked, can easily be automated. • No human in the loop when an applica3on/workload is provisioned. • Flexibility to choose/change architecture design without impac3ng applica3ons.
OpenStack CEE Day 2013
Fabric & POD Design
Leaf Switch Leaf Switch Leaf Switch Leaf Switch Leaf Switch Edge Switch
Edge Switch
Spine Switch
Spine Switch
Spine Switch
Spine Switch
World
Pod Switch Pod Switch
Compute Cabinets
Compute Cabinets
Infrastructure Cabinets
Edge Gateway Cabinets
Hypervisor
Hypervisor
Hypervisor
Hypervisor
Hypervisor
Hypervisor
Hypervisor
Hypervisor
NVP Controller
Service Nodes
OpenStack
OpenStack
L3 Gateways
L2 Gateways
VLANs STT/GRE
Leaf Switch L2
L3
L3 ECMP
BGP
BGP OSPF ISIS
RR
Sta3c / NAT
No VM addresses
OpenStack CEE Day 2013
Physical Network
Hypervisors + OVS L2/L3 Gateways
Service Nodes
NVP Control Plane
NVP API
Quantum & Quantum API
Mgmt & Operator Tools
The NVP “Stack”
OpenStack CEE Day 2013
About Open vSwitch
• Open Source, started with code contributed by Nicira • Widespread support in a lot of Linux Distribu3ons • Upstreamed in Linux Kernel • Building block for most Quantum Plugins today. • No single “feature set”: generic flow table lookup + tunneling engine. • Really what mamers is how this “engine” is programmed. Ranges from: • very simple (L2 forwarding) • very complex (L2 + L3 + ACL + QoS, etc.)
OpenStack CEE Day 2013
Hypervisor
br0 Linux IP stack + rouKng table
192.168.10.1
WEB WEB APP APP
Top of Rack Switch(s)
Config/State DB
ovsdb-‐server
ovs-‐vswitchd
eth0
MGMT Controller Controller NVP Controller
eth1 kernel user
TCP 6633 OpenFlow
TCP 6632 OVSDB
Tunnel Ports (to Linux IP Stack)
br-‐int (flow table)
OpenStack CEE Day 2013
Open vSwitch (OVS)
Open vSwitch (OVS)
Hypervisor Pla6orm Hypervisor Pla6orm
VM VM VM source
MAC
VM source IP
VM source MAC
VM source IP
source HV MAC
Source HV IP
VM source MAC
VM source IP
VM source MAC
VM source IP
source MAC
source HV IP
NVP Tunneling
Logical Network
Physical Network
OpenStack CEE Day 2013
A friendly note about tunneling protocols… • tunneling protocol != network virtualiza3on. They are just a part of the solu3on. • What does mamer: how forwarding rules setup. • For example:
• GRE was around for years, but missing programmable forwarding • VXLAN adop3on hobbled by reliance on mul3cast to program forwarding.
• NVP enables programma3c forwarding setup, can use many protocols. • For example:
• IPsec tunneling if security is required (e.g., over WAN) • VXLAN if interac3on with a physical switch is required.
OpenStack CEE Day 2013
Physical Network
Hypervisors + OVS L2/L3 Gateways
Service Nodes
NVP Control Plane
NVP API
Quantum & Quantum API
Mgmt & Operator Tools
The NVP “Stack”
OpenStack CEE Day 2013
NVP Controller Basics
• x86 SoMware • Exposes northbound API to Quantum • Southbound API to OVS • Maps between logical + physical • Never handles dataplane traffic
NVP Controller NVP Controller NVP Controller
OpenStack CEE Day 2013
NVP Controller
NVP Controller
NVP Controller
NVP Controller
NVP Controller
NVP Controller scale out
Node5 Node4
WebService API
Persistent Storage
Logical Network
Transport Network
Node1 Node2 Node3
Controller Cluster
• All nodes Ac3ve • Workload sliced and shared • Majority rule • Live SoMware Upgrades
OpenStack CEE Day 2013
Physical Network
Hypervisors + OVS L2/L3 Gateways
Service Nodes
NVP Control Plane
NVP API
Quantum & Quantum API
Mgmt & Operator Tools
The NVP “Stack”
OpenStack CEE Day 2013
NVP API NVP API
Non-‐Virtualized Abstrac3ons • Transport Nodes
(Mgmt & tunnel informa3on about hypervisors, gateways, service nodes)
• Transport Zones (Physical networks connec3ng Transport Nodes)
• Gateway Services (Collec3on of GW devices that func3on as a single unit)
• Controller Cluster status
Virtualized Abstrac3ons • Logical Switch (L2) • Logical Router (L3) • Logical Port
• Port security / port isola3on
• ACLs / Security Groups • QoS • Packet Sta3s3cs • Port mirroring
Descrip3on of physical world
Descrip3on of logical world
OpenStack CEE Day 2013
Physical Network
Hypervisors + OVS L2/L3 Gateways
Service Nodes
NVP Control Plane
NVP API
Quantum & Quantum API
Mgmt & Operator Tools
The NVP “Stack”
OpenStack CEE Day 2013
Nova Compute
Quantum w/NVP Architecture
Tenant Scripts
Horizon
NVP Plugin
OVS
Quantum API
NVP Controller NVP Controller NVP Controller
Cluster
L3 Fabric Orchestra3on Code
Nova Driver
Nova API
Create Net 1
Boot VM on Net 1
Create port Net 1, return port ID
Create vnic with port ID
Return port-‐ID
Push flow state
OpenStack CEE Day 2013
Create Net 1
Physical Network
Hypervisors + OVS L2/L3 Gateways
Service Nodes
NVP Control Plane
NVP API
Quantum & Quantum API
Mgmt & Operator Tools
The NVP “Stack”
OpenStack CEE Day 2013
L2 Gateways
br-‐int br-‐int
Service Node Service Node
Hypervisors
L3 Gateway L2 Gateway
NVP Controller
HV1 HV2
WAN / Internet
STT/GRE Tunnels
IPSec + STT/GRE
Data Data
VLAN 10
Logical Switch 1
WEB WEB Data Data
VLAN 10
WEB WEB
virtualized view
non-‐virtualized view
OpenStack CEE Day 2013
L3 Gateway HA + Scale-‐out
Failure Zone 2
R3
R4
R5
R6
R7
R8
R11
R12
R9
R10
R11
R12
R1
R2
R5
R6
Failure Zone 1
L3 Gateway Service
GW 1 GW N GW2 GW N+1
br-‐int br-‐int Hypervisors HV1 HV2
STT/GRE Tunnels with monitoring.
OpenStack CEE Day 2013
Physical Network
Hypervisors + OVS L2/L3 Gateways
Service Nodes
NVP Control Plane
NVP API
Quantum & Quantum API
Mgmt & Operator Tools
The NVP “Stack”
OpenStack CEE Day 2013
Failure Zone 2
Service Node HA + Scale-‐out
Brad Hedlund -‐ OpenStack Grizzly
Failure Zone 1
Bcat/Mcast ReplicaKon Service
SN 1 SN N SN 2 SN N+1
br-‐int br-‐int Hypervisors HV1 HV2
STT/GRE Tunnels with monitoring.
Logical Switch 1
Logical Switch N
Logical Switch 1
Logical Switch N
Logical Switch 1
Logical Switch N
Logical Switch 1
Logical Switch N
OpenStack CEE Day 2013
Physical Network
Hypervisors + OVS L2/L3 Gateways
Service Nodes
NVP Control Plane
NVP API
Quantum & Quantum API
Mgmt & Operator Tools
The NVP “Stack”
OpenStack CEE Day 2013
Management & Opera6ons • Tunnel status • Port-‐to-‐port troubleshoo3ng tool
• Traceflow packet injec3on
OpenStack CEE Day 2013
Management & Opera6ons (2)
• Automated deployment of new Version
• Built in compa3bility verifica3on
• Rollback • Online Upgrade (i.e. dataplane & control plane services stay up)
OpenStack CEE Day 2013
NVP: It’s not just about scale …
• Data plane performance • Fast + reliable high availability (data plane + control plane) • Rich logical network capabili3es (QoS, ACLs, sta3s3cs, etc.) • Ability to onboard remote customers + physical workloads (L2 GW) • Operator tools to troubleshoot, upgrade, etc.
OpenStack CEE Day 2013
Thank You! Have a great OpenStack CEE Day and check out our booth
OpenStack CEE Day 2013
![[2017 AWS Startup Day] AWS 비용 최대 90% 절감하기: 스팟 인스턴스 Deep-Dive](https://cdn.vdocuments.pub/doc/165x107/5a661a097f8b9a214f8b54dd/2017-aws-startup-day-aws-90-.jpg)
