copyright © 2012, mbl@cs.nctu software defined networking for big data applications...

Copyright © 2012, MBL@CS.NCTU

Software Defined Networking for Big Data Applications

指導教授：王國禎 博士 學生：徐逸懷 連懷恩 賴寬嶧 洪維藩 張晏誌

國立交通大學網路工程研究所行動計算與寬頻網路實驗室

1

Copyright © 2012, MBL@CS.NCTU

2

Outlines

• SDN: Software-Defined Network• Motivation• Related Works• Design approach• OpenFlow Mininet• Schedule• References

Copyright © 2012, MBL@CS.NCTU

3

SDN: Software-Defined Network

• SDN separates the control plane from the data plane in network switches and routers.

Copyright © 2012, MBL@CS.NCTU

4

Motivation – Why SDN in Data Center?

• Google is using OpenFlow on custom-designed hardware for all the internal networks it runs connecting its global data centers, said Urs Holzle, senior vice president of technology infrastructure at Google, speaking in a keynote at the second annual Open Networking Summit 2012. [1]

Copyright © 2012, MBL@CS.NCTU

5

Motivation – Why SDN in Data Center?

• Virtual data center network.

• Application awareness.

• Better scalability for cloud environment.

• Faster VM migration.

Copyright © 2012, MBL@CS.NCTU

6

Motivation – Virtual Data Center Network

• VMware will acquire Nicira for approximately $1.05 billion in cash plus approximately $210 million of assumed unvested equity awards.

• “Tenants or customers utilizing the software-defined datacenter can have their own virtual datacenter with an isolated collection of all the compute, storage, networking, and security resources that they are used to. This is what the software-defined datacenter is all about, and it is the architecture for the cloud.” [2]

Copyright © 2012, MBL@CS.NCTU

7

Motivation – Virtual Data Center Network

• Security and privacy between different applications.

• Better QoS management.

• For operators of data centers, it will be easier to reconfigure the network equipments.

Copyright © 2012, MBL@CS.NCTU

8

Motivation – Application Awareness

• Guohui Wang et al. [3] have shown that by the help of SDN, the time to perform a MapReduce task can have a reduction up to 70% in their test case.

• Optimizing the routing and bandwidth resource between applications.

• Giving different priority to control messages and data transferring packets.

Copyright © 2012, MBL@CS.NCTU

9

Motivation – Better Scalability

• Traditional LAN (spanning tree protocol, ARP, forwarding table) limits the scalability of data centers.

• With SDN, a connection between two hosts can use duplicate paths simultaneously.

• Faster recovery from network failure.

Copyright © 2012, MBL@CS.NCTU

10

Motivation – Faster VM Migration

• Faster network reconfiguration after migration; transparent to applications [4].

• 48 bits PMAC: dcid.pod.position.port.vmid• 32 bits PIP: privateNetworkId.dcid.subnetid.hostid

Copyright © 2012, MBL@CS.NCTU

Related Works

Software Defined Networking

Recovery

Fonseca [5]

Integrate

Simeonidou [6]Farias [7]

Data Center

VM Migration

Mann [4]Pu [9]

QoS

Jeong [8]

Big Data

Wang [3]Proposed

• The classification tree of SDN

Copyright © 2012, MBL@CS.NCTU

12

Related Works

  Scale 

Use SDN to optimize network

Mentioned about cloud computing

Mentioned about big data

Application aware

Specify a routing protocol

CrossRoad[4] Intra data center

Between VMs Yes No No No

CloudRack[9] Intra data center

Between VMs Yes No No No

QoS-aware NOS for SDN[8]

Inter data center

Between ASes

No No No No

Programming Your Network at Run-time for Big Data Application[3]

Intra data center

Between VMs or Racks

Yes Yes Yes No

Proposed Intra data center

Between VMs Yes Yes Yes Yes

Copyright © 2012, MBL@CS.NCTU

13

Design Approach

• Our design approach is composed of three parts:– Virtual network and flow request– Network resource allocation algorithm– Network topology research

Copyright © 2012, MBL@CS.NCTU

14

Design Approach – Virtual Network and Flow Request

• Every application, including all its VMs and storage nodes form a virtual network, and assigned with a virtual network number.

• Every virtual network has a head node who will submit a flow request to the SDN controller. The controller then allocate the network resource to all the virtual networks according to their flow requests.

• The Priority of control messages and data transferring packets.

Copyright © 2012, MBL@CS.NCTU

15

Design Approach – Network Resource Allocation Algorithm

• For every connection in the network, we find all the possible paths between these two nodes and sort these paths by their cost in a non-decreasing order.

• The principle is, when some competing connections can not be satisfied with their minimum cost paths, we choose the next minimal cost path among all the competing connections and see if the demand can be satisfied.

• More like a circuit switch.

Copyright © 2012, MBL@CS.NCTU

16

Design Approach – Network Resource Allocation Algorithm

• Packet priority.

• When there are some nodes added in or removed, some flow requests are updated, or a network failure happens, we should run an update algorithm. The update algorithm would be the subset of the aforementioned algorithm and it should be fast enough that we can apply it on-line.

Copyright © 2012, MBL@CS.NCTU

17

Design Approach – Network Topology Research

• What is the best strategy when we apply the SDN scheme to the traditional data center network topology like Tree, VL2, Fat-Tree, and BCube?

• Is there a better network topology when we apply the SDN scheme to the data center?

Copyright © 2012, MBL@CS.NCTU

OpenFlow Mininet

• Mininet creates scalable (up to hundreds of nodes, depending on your configuration) software-defined (e.g. OpenFlow) networks on a single PC by using Linux processes in network namespaces [10].

• It allows you to quickly create, interact with, customize and share a software defined network prototype, and provides a smooth path to running on hardware [10].

• Build in Stanford University.

Copyright © 2012, MBL@CS.NCTU

OpenFlow Mininet

• Display nodes: nodes• you can ping from host 0 to host 1: h2 ping -c 1 h3

Copyright © 2012, MBL@CS.NCTU

20

• 第一年：–蒐集相關文獻，並歸納比較–熟悉 SDN的模擬工具–提出適用於 data center的 SDN網路架構，並與　傳統非 SDN的網路做比較

• 第二年：–實做出我們的 SDN網路架構，並利用 Hadoop　等適用於 big data的程式來初步評估效果–除了提升效能外，為我們的方法增加更多特　性，如 QoS management, Inter-data center

SDN

Schedule

Copyright © 2012, MBL@CS.NCTU

21

• 第三年：–加入更多來自 big data的 application來評估調整我們的方法

–將最新的文獻加入比較–與其他子計畫的項目整合

Schedule

Copyright © 2012, MBL@CS.NCTU

22

References

• [1] http://www.eetimes.com/electronics-news/4371179/Google-describes-its-OpenFlow-network

• [2] http://www.vmware.com/nl/company/news/releases/vmw-nicira-07-23-12.html• [3] Guohui Wang, T.S. Eugene Ng, and Anees Shaikh. 2012. Programming your network at

run-time for big data applications. In Proceedings of the first workshop on Hot topics in software defined networks (HotSDN '12). ACM, New York, NY, USA, 103-108.

• [4] Mann, V.; Vishnoi, A.; Kannan, K.; Kalyanaraman, S.; , "CrossRoads: Seamless VM mobility across data centers through software defined networking," Network Operations and Management Symposium (NOMS), 2012 IEEE , vol., no., pp.88-96, 16-20 April 2012

• [5] Fonseca, P.; Bennesby, R.; Mota, E.; Passito, A.; , "A replication component for resilient OpenFlow-based networking," Network Operations and Management Symposium (NOMS), 2012 IEEE , vol., no., pp.933-939, 16-20 April 2012

• [6]Simeonidou, D.; Nejabati, R.; Azodolmolky, S.; , "Enabling the future optical Internet with OpenFlow: A paradigm shift in providing intelligent optical network services," Transparent Optical Networks (ICTON), 2011 13th International Conference on , vol., no., pp.1-4, 26-30 June 2011

Copyright © 2012, MBL@CS.NCTU

23

References

• [7] Farias, F.N.N.; Salvatti, J.J.; Cerqueira, E.C.; Abelem, A.J.G.; , "A proposal management of the legacy network environment using OpenFlow control plane," Network Operations and Management Symposium (NOMS), 2012 IEEE , vol., no., pp.1143-1150, 16-20 April 2012

• [8] Kwangtae Jeong; Jinwook Kim; Young-Tak Kim; , "QoS-aware Network Operating System for software defined networking with Generalized OpenFlows," Network Operations and Management Symposium (NOMS), 2012 IEEE , vol., no., pp.1167-1174, 16-20 April 2012

• [9] Yan Pu; Yilong Deng; Nakao, A.; , "Cloud Rack: Enhanced virtual topology migration approach with Open vSwitch," Information Networking (ICOIN), 2011 International Conference on , vol., no., pp.160-164, 26-28 Jan. 2011

• [10] http://yuba.stanford.edu/foswiki/bin/view/OpenFlow/Mininet