networking for nested containers: magnum, kuryr, neutron integration

Magnum, Kuryr, Neutron IntegrationNetworking for Nested Containers

Fawad Khaliq - @fawadkhaliq Antoni Segura – @celebdor Gal Sagie - @GalSagie

Copyright © PLUMgrid, Inc. 2011-2016

IntroductionSpeakers

Sr. Software Engineer PLUMgrid

KhaliqFawad

2

Senior Engineer Midokura

SeguraAntoni

Architect Huawei

SagieGal


• Magnum, Neutron • Kuryr • Nested Containers and Networking Problem • Nested Containers Networking – Solution/Design • Capabilities and considerations • Current Status • Next Steps • Q&A

Agenda

3

MagnumContainer-as-a-service in OpenStack

4



5

Docker Swarm (Bay)

Nova Instance

Container

Container

Container

Nova Instance

Container

Container

Container



6

Kubernetes (Bay)

Nova Instance

Pod

Container

Container

Nova Instance

Pod

Container

Container

NeutronNetworking in OpenStack

8


• Provides “network as a service” • Provides rich network topologies • Technology agnostic; pluggable networking backends • Extensible • Offers advanced services like LBaas, VPNaas, FWaas etc

Neutron

9

KuryrContainer Networking in OpenStack

10


Kuryr

11

Neutron as the production-ready networking abstraction containers need


VM/Container Networking: Similar Concepts

12

Docker C1 Docker C2 Docker C3

libNetwork

Endpoint Endpoint EndpointEndpoint

Frontend Network

Backend Network

Network Sandbox Network Sandbox Network Sandbox

VM2

192.168.1.7 192.168.5.2

VM1

Tenant A Net1 192.168.1.0/0

Tenant A Net2 192.168.5.0/0

192.168.1.5

Neutron


• Open source • Part of OpenStack Big-Tent • Brings the Neutron networking model to containers • Aims to support different Container Runtimes (docker, rkt, etc)

• E.g. Kubernetes, Mesos, Docker Swarm • Weekly IRC meetings • Working together with OpenStack community

• Neutron, Magnum, Kolla

Kuryr Project Overview

13


Kuryr Components

14

Configuration ManagementKuryr libNetwork

Network Plugin

K8S CNI Driver

Keystone Authentication & Neutron Client Interface

Generic VIF Binding

Kuryr libNetwork IPAM Plugin

Problems with current Nested ContainersWhy do we need to consider this as a special scenario?

15


• Two Separate networking infrastructures • Hard to enforce network policy (N-tier applications) • Security and Isolation • Performance and unneeded overhead

Problems with Current Nested Containers Networking

16



17

Docker 0

OVS

VXLAN Overlay

VM

Docker 0

Neutron Plugin

VXLAN Overlay

VM

SDN Overlay

Neutron Overlay



18

Neutron Networks

VMVM VM

Tenant A Net1 192.168.1.0/0



19

Container Networks

VMVM VM

Backend Network 10.2.0.0/24

Endpoint Endpoint Endpoint Endpoint Endpoint Endpoint Endpoint

Frontend Network 10.1.0.0/24

Nested Container Networking SolutionDesign for the nested container networking in OpenStack

20


• Nested/baremetal container to nested/baremetal container same/different hosts

• Nested/baremetal container to virtual machine communication • Nested/baremetal container to baremetal communication • Container networking as a first class entity in Neutron • Consistent policy enforcement across containers, VMs, bare metal • Enable advanced networking services like FWaas, LBaas, VPNaas

etc

Nested Container Networking Use Cases

21


Nested Container Networking Design Magnum, Kuryr, Neutron Integration

22

VLAN:100 VLAN:200 VLAN:400 VLAN:100


Neutron Trunk Ports

23

Nova Instance

port-1

port-0

port-2

network-1

network-0

network-2Port combined into one vif by turning port-0 into trunk and other ports into supports of the trunk

Capabilities and Considerations

25


• Neutron resources spec approved and patches under review • Trunk • Subport

• Subports bring isolation to container-in-VM use cases • Port forwarding can take us further

• Vendors can implement new segmentation types • Tagged traffic that does not match a sub port, is considered of the

trunk port


26


• Limitations • Policy is applied at the Host level • Initial only VLAN tags for segmentation type • Tags are unique per “trunk port” scope • VM users can alter subport traffic • Logging of VM actions is dependent on integration • Can't work with current OVS


27

Current Status

28


• Trunk Port Extension spec approved and code in progress • Binding profile workaround to proceed in parallel

• Nested Container networking spec approved in Kuryr • Docker Swarm Integration completed • Kubernetes in progress • Mesos in design stages

Current Status

29

Next Steps

30


• Follow up on the Neutron Trunk port implementation • Finish COE baremetal integration

• Policy translation • Make Neutron resources available through native APIs

• Magnum deployment prototype of worker VM with Kuryr agent • Magnum administrator VM that communicates with Neutron

Next Steps

31

Questions

32

Join us at #openstack-kuryr

THANK YOU!

irc: #openstack-kuryr @ freenode