openv switchの使い方とか
DESCRIPTION
Open vSwitchコードリーディングで使った資料です。TRANSCRIPT
Open vSwitchの使い方とか
1
自己紹介
• 菊田 宏(きくた こう)
• 某社研究開発職
• Twitter : @kotto_hihihi
→ほぼROM
• Open vSwitchを触り始めたキッカケ
→OpenFlowスイッチを探していたら見つけた
2
今日話すこと
• Open vSwitchの概要
• Open vSwitchの基本的な使い方
▫ 各コンポーネントの関係性
• Open vSwitchのデータ構造
3
Open vSwitchとは
• OSSの仮想スイッチ
• Linux標準のbridgeと互換性あり
• 開発者の大半はNicira社の人
• ハードにポーティングすることを意識している
4
Open vSwitchで何ができる？
• 転送機能 ：Bridge, VLAN, STP, LACP, GRE, GRE over IPsec, CAPWAP
• 管理機能 ：NetFlow, sFlow
• 制御機能 ：OpenFlow1.0, 1.1, 1.2
5
Open vSwitchの歴史？ • 2010/03/15 v1.0.0リリース
▫ OpenFlow1.0.0に対応 ▫ GRE対応 →1.0.1まで
• 2011/04/05 v1.1.0リリース ▫ QoS対応 ▫ Bonding対応 ▫ OpenFlowのベンダ拡張(NXM)対応 →1.1.2まで
• 2011/08/03 v1.2.0リリース ▫ とても性能が良くなったらしい →1.2.2まで
• 2011/12/09 v1.3.0リリース ▫ FlowTableが255個に(OpenFlow1.1.0の機能が使える？) ▫ STPに対応 ▫ NXM機能拡張
• 2012/01/30 v1.4.0リリース ▫ NXM機能拡張
• ※0.9以前のドキュメントが見つけることができませんでした…
6
Open vSwitchインストール
前提：Ubuntu 10.04 server amd64
• パッケージインストール
いろんなところでSSL対応したい場合はlibsslを入れる
• Open vSwitchのインストール
• ovsdbの作成
# apt-get install make pkg-config gcc autoconf libtool
# wget http://openvswitch.org/releases/openvswitch-1.4.0.tar.gz # tar zxvf openvswitch-1.4.0.tar.gz # cd openvswitch-1.4.0 # ./boot.sh # ./configure --with-linux=/lib/modules/`uname -r`/build # make # make install # insmod datapath/linux/openvswitch_mod.ko
# mkdir -p /usr/local/etc/openvswitch # ovsdb-tool create /usr/local/etc/openvswitch/conf.db ¥ vswitchd/vswitch.ovsschema
7
Open vSwitch起動
• ovsdb-server起動
SSL対応したい場合はlibsslを入れる
• vswitchdの起動
# ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock ¥ --remote=db:Open_vSwitch,manager_options ¥ --pidfile --detach
# ovs-vsctl --no-wait init # ovs-vswitchd --pidfile --detach
8
基本的な使い方
• Bridge作成
• 設定の確認
# ovs-vsctl add-br br0 # ovs-vsctl add-port br0 eth1 # ovs-vsctl add-port br0 eth2
# ovs-vsctl list-br br0 # ovs-vsctl list-ports br0 eth1 eth2
br0
eth1 eth2
10.0.0.1/24 10.0.0.2/24
9
Open vSwitch設定の流れ
① ovs-vsctlコマンドがovsdb-serverに設定情報を送信する
② ovsdb-serverがovs-vswitchdに設定情報を送信する
③ ovs-vswitchdがopenvswitch_modを設定する
④ ovs-vswitchdがovsdb-serverに設定結果を送信する
⑤ ovsdb-serverがovsdbに設定情報、設定結果を登録する
→設定情報は①と②の間に書き込んでいるかもしれない
⑥ ovsdb-serverがovs-vsctlコマンドに設定結果を送信する
ovs-vswitchd ovsdb-server ovsdb
ovs-vsctl
openvswitch_mod
①
JSON-RPC
②
③ ④
⑤
⑥
10
設定情報はovsdbで持つ。 ovsdbの実態はovsdb-toolで作成したconf.dbファイル。
少し脱線
設定済みのovsdbファイルをコピーすると同じ環境が作れる
11
br0
eth1 eth2
ovsdb
br0
eth1 eth2
ovsdb
br0
eth1 eth2
ovsdb
br0
eth1 eth2
ovsdb
ovsdbファイルをコピー
ovs-vswitchd
ovsdb-server
ovs-vswitchd
ovsdb-server
ovs-vswitchd
ovsdb-server
ovs-vswitchd
ovsdb-server
ovsdb-server, ovs-vswitchdを起動
コピー元と 同じ環境になる
ovs-vswitchdとovsdb-server間をTCP/IP or SSLで接続して、複数のOpen vSwitchホストをovsdb専用ホストで集中管理することもできる
もう少しだけ脱線
12
br0
eth1 eth2
ovsdb
br0
eth1 eth2
ovsdb
br0
eth1 eth2
ovsdb
br0
eth1 eth2
ovsdb
ovs-vswitchd
ovsdb-server
ovs-vswitchd
ovsdb-server
ovs-vswitchd
ovsdb-server
ovs-vswitchd
ovsdb-server
ovsdb 専用ホスト
Open vSwitch ホスト群
libvirtとの連携
brctlコマンドを使うことができればよい
• bridge-utilsをインストールする
• ovs-brcompatdを起動する
• brctlコマンドが使えるようになる
• brctlコマンドで設定した結果がovs-vsctlコマンドで確認できる
# apt-get install bridge-utils
# cd openvswitch-1.4.0 # insmod openvswitch-1.4.0/datapath/linux/brcompat_mod.ko # ovs-brcompatd --pidfile --detach
13
# brctl addbr br1 # brctl addif br1 eth1 # brctl addif br1 eth2
# ovs-vsctl list-br br1 # ovs-vsctl list-ports br1 eth1 eth2
brctlコマンド利用時の設定の流れ
① brctlコマンドがbrcompat_modを設定する
② brcompat_modがovs-brcompatdに設定情報を通知する
③ ovs-brcompatdがovs-vsctlコマンドを実行する
④ 以降、ovs-vsctlコマンド利用時と同じ
14
ovs-vswitchd ovsdb-server ovsdb
ovs-vsctl
openvswitch_mod
④
JSON-RPC
②
③
⑦ ⑧
⑨
ovs-brcompatd
brctl
brcompat_mod
①
⑤
⑥
• 最初からOpenFlowのフローエントリベースで動作している
• フローエントリを削除すると通信できなくなる
#ovs-ofctl del-flows br0 #ovs-ofctl dump-flows br0 NXST_FLOW reply (xid=0x4):
OpenFlowスイッチとして使う
#ovs-ofctl dump-flows br0 NXST_FLOW reply (xid=0x4): cookie=0x0, duration=2058.998s, table=0, n_packets=183, n_bytes=29257, priority=0 actions=NORMAL
15
OpenFlowスイッチとして使う
• ovs-ofctlコマンドでフローエントリが書ける
→フィルタリングにも使える
• フローエントリを書くときに使えるfield
→OpenFlow 1.1, 1.2に9割対応しているらしい
# ovs-ofctl add-flow br0 ¥ in_port=1,dl_type=0x0800,nw_src=10.0.0.1,nw_dst=10.0.0.2,actions=output:2 # ovs-ofctl add-flow br0 ¥ in_port=2,dl_type=0x0800,nw_src=10.0.0.2,nw_dst=10.0.0.1,actions=output:1
in_port, dl_vlan, dl_vlan_pcp, dl_src, dl_dst, dl_type, nw_src, nw_dst, nw_proto, nw_tos, nw_ecn, nw_ttl, tp_src, tp_dst, icmp_type, icmp_code, table, vlan_tci, ip_frag, arp_sha, arp_tha, ipv6_src, ipv6_dst, ipv6_label, nd_target, nd_sll, nd_tll, tun_id, regX
16
ovs-ofctlでフローエントリ設定の流れ
① ovs-ofctlコマンドがovs-vswitchdに設定情報を送信する
② ovs-vswitchdがopenvswitch_modを設定する
17
ovs-vswitchd ovsdb-server ovsdb
ovs-vsctl
openvswitch_mod
JSON-RPC
②
ovs-brcompatd
brctl
brcompat_mod
①
ovs-ofctl
コントローラから制御する
• OpenFlowコントローラと接続する
• OpenFlowコントローラが設定されるとL2SWとして動作するための フローエントリが消える
→以降、コントローラの制御で動作する。
# ovs-vsctl set-controller br0 tcp:172.0.0.10:6633
# ovs-ofctl dump-flows br0 NXST_FLOW reply (xid=0x4):
18
br0
eth1 eth2
ovs-vswitchd
コントローラ
Open vSwitchのデータ構造
ovsdb-clientを使ってovsdbのデータ構造を見る
どんなDBがあるか？ どんなtableがあるか？
19
# ovsdb-client list-dbs Open_vSwitch
# ovsdb-client list-tables Open_vSwitch Table ------------ Capability SSL Bridge Controller NetFlow Port Mirror Queue QoS Interface Open_vSwitch sFlow Manager
Open vSwitchのデータ構造
各tableのcolumnと設定可能な値の定義
20
# ovsdb-client list-columns Open_vSwitch Bridge Column Type ------------- ------------------------------------------------------------------------------ status {"key":"string","max":"unlimited","min":0,"value":"string"} fail_mode {"key":{"enum":["set",["secure","standalone"]],"type":"string"},"min":0} other_config {"key":"string","max":"unlimited","min":0,"value":"string"} _version "uuid" name "string" datapath_type "string" netflow {"key":{"refTable":"NetFlow","type":"uuid"},"min":0} ports {"key":{"refTable":"Port","type":"uuid"},"max":"unlimited","min":0} external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} flood_vlans {"key":{"maxInteger":4095,"minInteger":0,"type":"integer"},"max":4096,"min":0} _uuid "uuid" controller {"key":{"refTable":"Controller","type":"uuid"},"max":"unlimited","min":0} datapath_id {"key":"string","min":0} stp_enable "boolean" mirrors {"key":{"refTable":"Mirror","type":"uuid"},"max":"unlimited","min":0} sflow {"key":{"refTable":"sFlow","type":"uuid"},"min":0}
Open vSwitchのデータ構造
21
Capability Bridge SSL
Controller NetFlow
Port
Mirror
Queue
QoS Interface
Open_vSwitch
sFlow
Manager
1
0,*
1
0,*
1 1
0,1 0,*
1 1 1 1
0,1
0,*
0,* 0,* 0,1
1
1
0,*
1
*
1 0,1
1 0,*
Open vSwitchのデータ構造
実際の設定はどうなっているか
ovs-vsctlコマンドで確認
22
# ovs-vsctl list bridge _uuid : 333ad8b8-7486-40ab-9d29-5051144734c2 controller : [311d4af1-6d48-4b75-85ed-17e67a94cfb0] datapath_id : "000000151769d908" datapath_type : "" external_ids : {} fail_mode : [] flood_vlans : [] mirrors : [] name : "br0" netflow : [] other_config : {} ports : [65f57564-d408-42ac-84f7-7c6a4b433a5e, 86ecd3a9-7776-4567-83a3-ae19b9e4b46e, f490bdfa-8424-475a-9949-d10e2d3b8820] sflow : [] status : {} stp_enable : false
ovsdbに直接設定
ovs-vsctlコマンドでfail_modeにsecureを設定してみる
23
# ovs-vsctl set Bridge br0 fail_mode=secure # ovs-vsctl list Bridge _uuid : 333ad8b8-7486-40ab-9d29-5051144734c2 controller : [34d55998-475b-47d4-aa72-a37cfa6d294a] datapath_id : "000000151769d908" datapath_type : "" external_ids : {} fail_mode : secure flood_vlans : [] mirrors : [] name : "br0" netflow : [] other_config : {} ports : [65f57564-d408-42ac-84f7-7c6a4b433a5e, 86ecd3a9-7776-4567-83a3-ae19b9e4b46e, f490bdfa-8424-475a-9949-d10e2d3b8820] sflow : [] status : {} stp_enable : false
まとめ
• Open vSwitchの基本的な使い方を紹介
• Open vSwitchを構成するコンポーネントの関係性を紹介
• Open vSwitchのデータ構造を紹介
24
(参考)Open_vSwitchテーブル構造
25
# ovsdb-client list-columns Open_vSwitch Open_vSwitch Column Type --------------- ------------------------------------------------------------------------------------------ _uuid "uuid" system_type {"key":"string","min":0} external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} capabilities {"key":"string","max":"unlimited","min":0,"value":{"refTable":"Capability","type":"uuid"}} system_version {"key":"string","min":0} _version "uuid" manager_options {"key":{"refTable":"Manager","type":"uuid"},"max":"unlimited","min":0} other_config {"key":"string","max":"unlimited","min":0,"value":"string"} statistics {"key":"string","max":"unlimited","min":0,"value":"string"} cur_cfg "integer" ssl {"key":{"refTable":"SSL","type":"uuid"},"min":0} ovs_version {"key":"string","min":0} next_cfg "integer" db_version {"key":"string","min":0} bridges {"key":{"refTable":"Bridge","type":"uuid"},"max":"unlimited","min":0}
(参考)Bridgeテーブル構造
26
# ovsdb-client list-columns Open_vSwitch Bridge Column Type ------------- ------------------------------------------------------------------------------ status {"key":"string","max":"unlimited","min":0,"value":"string"} fail_mode {"key":{"enum":["set",["secure","standalone"]],"type":"string"},"min":0} other_config {"key":"string","max":"unlimited","min":0,"value":"string"} _version "uuid" name "string" datapath_type "string" netflow {"key":{"refTable":"NetFlow","type":"uuid"},"min":0} ports {"key":{"refTable":"Port","type":"uuid"},"max":"unlimited","min":0} external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} flood_vlans {"key":{"maxInteger":4095,"minInteger":0,"type":"integer"},"max":4096,"min":0} _uuid "uuid" controller {"key":{"refTable":"Controller","type":"uuid"},"max":"unlimited","min":0} datapath_id {"key":"string","min":0} stp_enable "boolean" mirrors {"key":{"refTable":"Mirror","type":"uuid"},"max":"unlimited","min":0} sflow {"key":{"refTable":"sFlow","type":"uuid"},"min":0}
(参考)Portテーブル構造
27
# ovsdb-client list-columns Open_vSwitch Port Column Type --------------- ------------------------------------------------------------------------------------------------------- bond_fake_iface "boolean" status {"key":"string","max":"unlimited","min":0,"value":"string"} other_config {"key":"string","max":"unlimited","min":0,"value":"string"} bond_mode {"key":{"enum":["set",["active-backup","balance-slb","balance-tcp","stable"]],"type":"string"},"min":0} _version "uuid" interfaces {"key":{"refTable":"Interface","type":"uuid"},"max":"unlimited"} name "string" bond_updelay "integer" vlan_mode {"key":{"enum":["set",["access","native-tagged","native-untagged","trunk"]],"type":"string"},"min":0} lacp {"key":{"enum":["set",["active","off","passive"]],"type":"string"},"min":0} mac {"key":"string","min":0} _uuid "uuid" external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} trunks {"key":{"maxInteger":4095,"minInteger":0,"type":"integer"},"max":4096,"min":0} statistics {"key":"string","max":"unlimited","min":0,"value":"integer"} fake_bridge "boolean" tag {"key":{"maxInteger":4095,"minInteger":0,"type":"integer"},"min":0} bond_downdelay "integer" qos {"key":{"refTable":"QoS","type":"uuid"},"min":0}
(参考)Interfaceテーブル構造
28
# ovsdb-client list-columns Open_vSwitch Interface Column Type ---------------------- ---------------------------------------------------------------- status {"key":"string","max":"unlimited","min":0,"value":"string"} link_resets {"key":"integer","min":0} link_speed {"key":"integer","min":0} duplex {"key":{"enum":["set",["full","half"]],"type":"string"},"min":0} admin_state {"key":{"enum":["set",["down","up"]],"type":"string"},"min":0} ofport {"key":"integer","min":0} _version "uuid" other_config {"key":"string","max":"unlimited","min":0,"value":"string"} name "string" link_state {"key":{"enum":["set",["down","up"]],"type":"string"},"min":0} type "string" mtu {"key":"integer","min":0} mac {"key":"string","min":0} cfm_mpid {"key":"integer","min":0} cfm_fault {"key":"boolean","min":0} _uuid "uuid" external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} options {"key":"string","max":"unlimited","min":0,"value":"string"} ingress_policing_rate {"key":{"minInteger":0,"type":"integer"}} statistics {"key":"string","max":"unlimited","min":0,"value":"integer"} cfm_remote_mpids {"key":"integer","max":"unlimited","min":0} ingress_policing_burst {"key":{"minInteger":0,"type":"integer"}} lacp_current {"key":"boolean","min":0}
(参考)QoSテーブル構造
29
# ovsdb-client list-columns Open_vSwitch QoS Column Type ------------ -------------------------------------------------------------------------------------------------------------------------------------- queues {"key":{"maxInteger":4294967295,"minInteger":0,"type":"integer"},"max":"unlimited","min":0,"value":{"refTable":"Queue","type":"uuid"}} external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} _uuid "uuid" type "string" other_config {"key":"string","max":"unlimited","min":0,"value":"string"} _version "uuid"
(参考)Queueテーブル構造
30
# ovsdb-client list-columns Open_vSwitch Queue Column Type ------------ ----------------------------------------------------------------- external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} _uuid "uuid" dscp {"key":{"maxInteger":63,"minInteger":0,"type":"integer"},"min":0} other_config {"key":"string","max":"unlimited","min":0,"value":"string"} _version "uuid"
(参考)Controllerテーブル構造
31
# ovsdb-client list-columns Open_vSwitch Controller Column Type ---------------------- --------------------------------------------------------------------------- _uuid "uuid" external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} status {"key":"string","max":"unlimited","min":0,"value":"string"} local_netmask {"key":"string","min":0} _version "uuid" is_connected "boolean" controller_burst_limit {"key":{"minInteger":25,"type":"integer"},"min":0} max_backoff {"key":{"minInteger":1000,"type":"integer"},"min":0} controller_rate_limit {"key":{"minInteger":100,"type":"integer"},"min":0} local_ip {"key":"string","min":0} local_gateway {"key":"string","min":0} connection_mode {"key":{"enum":["set",["in-band","out-of-band"]],"type":"string"},"min":0} inactivity_probe {"key":"integer","min":0} target "string" role {"key":{"enum":["set",["master","other","slave"]],"type":"string"},"min":0}
(参考)Managerテーブル構造
32
# ovsdb-client list-columns Open_vSwitch Manager Column Type ---------------- -------------------------------------------------------------------------- _uuid "uuid" external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} status {"key":"string","max":"unlimited","min":0,"value":"string"} max_backoff {"key":{"minInteger":1000,"type":"integer"},"min":0} _version "uuid" is_connected "boolean" connection_mode {"key":{"enum":["set",["in-band","out-of-band"]],"type":"string"},"min":0} inactivity_probe {"key":"integer","min":0} target "string"
(参考)Capabilityテーブル構造
33
# ovsdb-client list-columns Open_vSwitch Capability Column Type -------- ----------------------------------------------------------- _uuid "uuid" details {"key":"string","max":"unlimited","min":0,"value":"string"} _version "uuid"
(参考)SSLテーブル構造
34
# ovsdb-client list-columns Open_vSwitch SSL Column Type ----------------- ----------------------------------------------------------- certificate "string" external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} _uuid "uuid" private_key "string" ca_cert "string" bootstrap_ca_cert "boolean" _version "uuid"
(参考)Mirrorテーブル構造
35
# ovsdb-client list-columns Open_vSwitch Mirror Column Type --------------- ------------------------------------------------------------------------------------ _uuid "uuid" external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} select_vlan {"key":{"maxInteger":4095,"minInteger":0,"type":"integer"},"max":4096,"min":0} select_src_port {"key":{"refTable":"Port","refType":"weak","type":"uuid"},"max":"unlimited","min":0} _version "uuid" select_all "boolean" name "string" statistics {"key":"string","max":"unlimited","min":0,"value":"integer"} select_dst_port {"key":{"refTable":"Port","refType":"weak","type":"uuid"},"max":"unlimited","min":0} output_port {"key":{"refTable":"Port","refType":"weak","type":"uuid"},"min":0} output_vlan {"key":{"maxInteger":4095,"minInteger":1,"type":"integer"},"min":0}
(参考)NetFlowテーブル構造
36
# ovsdb-client list-columns Open_vSwitch NetFlow Column Type ------------------- ------------------------------------------------------------------ _uuid "uuid" external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} targets {"key":"string","max":"unlimited"} add_id_to_interface "boolean" _version "uuid" active_timeout {"key":{"minInteger":-1,"type":"integer"}} engine_id {"key":{"maxInteger":255,"minInteger":0,"type":"integer"},"min":0} engine_type {"key":{"maxInteger":255,"minInteger":0,"type":"integer"},"min":0}
(参考)sFlowテーブル構造
37
# ovsdb-client list-columns Open_vSwitch sFlow Column Type ------------ ----------------------------------------------------------- _uuid "uuid" external_ids {"key":"string","max":"unlimited","min":0,"value":"string"} targets {"key":"string","max":"unlimited"} agent {"key":"string","min":0} header {"key":"integer","min":0} _version "uuid" sampling {"key":"integer","min":0} polling {"key":"integer","min":0}