internet streaming media udp-based 轉送訊務的量測

1

Internet Streaming Media UDP-based轉送訊務的量測

中央大學電算中心楊素秋Dec, 2002

2

報告大綱• 1. 前言• 2. 相關研究• 3.Prevalent UDP Application• 4.Streaming Media傳訊模式• 5.Video Game 訊務特徵 • 6. Streaming Traffic統計方法• 7. 熱門應用訊務分布• 8.結語

3

1.前言• Streaming Media Traffic over Internet– 1970 年代的 audio conference– 1990 年代初期的 Mbone Video Conference

• variable delay Internet• Constant Bit Rate (CBR) traffic• 囿於網路頻寬限制致 streaming 品質無法為用戶接受

– 隨著 Audio/Video壓縮技術 ,網路頻寬的大量提昇• Internet用戶使用 free 播放程式• 播放音樂、新聞及影片視音節目 .

4

• 大量的 Streaming media UDP訊務– 容易造成網路資源分配不均 ,影響 TCP 應用延遲 ,產生traffic congestion

• 實做承載訊務的量測– 找出 streaming media 的 transportation features• 統計 /分析 streaming media訊務及其成長趨勢• 分析熱門應用訊務分布 • Capacity planning• Packet Classification

5

2.相關研究• Internet Traffic Measurement

– LAN Traffic Measurement• gathered packet-based traffic logs through tcpdump.– Provide traffic measurement over one broadcast segment

– Each traffic log recording the transportation information of each packet» TimeStamp» Src_ip.src_port > dst_ip.src_port» bytes » Transportation protocol

6

相關研究 (cont.)

•Barnet B. G. – measured traffic of an Ethernet segment– approximately 79% of the total Ethernet traffic in bytes was NFS traffic.

•Kushida Takayaki– gathered traffic logs over FDDI backbone through tcpdump

– The measurement result reveals that » about 98.2% of the traffic was TCP traffic.

7


– WAN Traffic Measurement•Thompson Kevin et al.

– Measured wide-area Internet traffic on an OC3 trunk within MCI ATM backbone using OC3MON

– TCP is the dominant protocol on the link; about 95% of the bytes.

– UDP is the second most dominant, with roughly 5% of the bytes,DNS dominates the total UDP traffic.

8

相關研究 (cont.)•Measuring WAN traffic using NetFlow or SFlow transportation traffic log gathered from router– WAN router轉送 Internet訊務

» router高效率暫存各介面轉送封包 header» 加總各 flow轉送訊務紀錄 ,週期性送往 Collect PC

– Collecting PC 執行 flow-tools shareware,接收與儲存區網 netflow logs (per-10-min)– 提供 region network 轉送訊務的追蹤與統計

9


• Light-Weight Netflow traffic logs recording– flow identifier

» source IP address, source port,» destination IP address, destination port

» protocol identifier– packet count, byte count,– source 及 destination interface identifier.

10

相關研究 (cont.)•Transit Traffic Log providing

– 特定 flow訊務特性的追蹤– 統計 Top-N user traffic (IP-based)

» src_ip, dst_ip» source/ destination interface» protocol identifier»

– 監測轉送的 TCP與 UDP 輸出 /輸入訊務– 分析熱門網路應用訊務分布

» source port, destination port

11

相關研究 (cont.)• Art Mena et al.

– Tcpdump 蒐集 RealAudio servers訊務 headers• 位於 Broadcast.com 區域網段的 5 部 RealAudio servers的訊務 headers

– 統計 RealAudio servers的輸出與輸入 media訊務特性– 統計數據顯示 :

• 絕大部分的 RealAudio Servers訊務為單向的大量傳送 , • RealAudio 封包長度明顯大於一般 UDP flows 封包 .

12

相關研究 (cont.)• Jacobus Merwe et al.

• extracted and analyzed 4.5 million application level session-level log entries for two streaming services

– over a period of 4 months. • The fields in each log entry include:

– requesting IP address,– particulars of requested resource,

» Real or Windows Media object– transport protocol

» TCP or UDP– total data transmitted– session end-time, total session time,– status/error codes, etc

13

相關研究 (cont.)– Characterize the workload by looking

at a number of different measures of on-demand media servers •Protocol family

– MMS 72%, Real 23%•Bandwidth

– low 35%, High 65%•Transportation protocol

– UDP 34% , TCP 29%, Http 37%

14

相關研究 (cont.)• Schulzrinne H. et al.

– Real Time Transportation Protocol• RTSP streams

– Real Time Streaming Protocol(RTSP)– Session control– Remote control streaming options

» play, fast forward, rewind, pause, restart • RTP/RTCP streams

– Real-Time Transportation Protocol(RTP)– Accompanying streams– Media / QoS Packets

15

相關研究 (cont.)• Conlklin G.J. et al.

– Scalable and adaptive streaming delivery•Vendor strategies

– MMS, Real commercial products» Client-Processing QoS Evaluation» Application-Level Multicasting

16

相關研究 (cont.)– client 端處理 QoS參數

•Accounting QoS» Estimated end-to-end variant delay & packet loss

» According to the media packet headers

•Feedback RTCP QoS參數» 調整傳輸品質或速率» Stop/redirect media streaming

17

相關研究 (cont.)– Application-level multicasting

•Geographically distributed servers– RTSP Agent蒐集 members辨識資訊 & 傳訊參數 (jitter, loss)– Redirect clients與較接近或負荷低的 server建立傳輸連接– 減輕 massive media streaming對沿徑連網品質的影響 .

18


• Our work –統計 streaming media過境訊務

•追蹤 WAN router 轉送的 media 訊務紀錄•依據 streaming media傳輸特性 ,歸納傳輸特徵

– QuickTime, RealPlayer, MediaPlayer

–分析 streaming media WAN 轉送訊務

19

3.Prevalent Application

• TCP Application– Mail, Telnet, Ftp, WWW– Streaming Media, P2P file sharing

• UDP Application– DNS, Tftp– Streaming Media, Game– VoIP/ SIP

20

TCP Application

• TCP Applications– Mail

• SMTP(25 port), POP3(110 port)– Telnet

• Telnet (23 port), ssh (22 port)– Ftp

• Ftp-data (20 port), ftp (port 21)• Fast Ftp **

– WWW/Proxy• Httpd (80 port), sHttpd (443 port)• Proxy (3128 port, 8080 ports)

22

Fast FTP traffic Logs

=====================================================

61.13.173.211.1229>140.135.101.74.2090 958575.000 684 1401.425

140.135.101.74.2090>61.13.173.211.1229 31192.000 764 40.827

61.13.173.211.1230>140.135.101.74.2095 1025917.000 733 1399.614

140.135.101.74.2104>61.13.173.211.1232 120.000 3 40.000

140.135.101.74.21>61.13.173.211.1193 917.000 10 91.700

140.135.101.74.2095>61.13.173.211.1230 31380.000 769 40.806

140.135.101.74.2102>61.13.173.211.1231 3124.000 74 42.216

61.13.173.211.1231>140.135.101.74.2102 116249.000 87 1336.195

61.13.173.211.1232>140.135.101.74.2104 144.000 3 48.000

61.13.173.211.1193>140.135.101.74.21 225.000 5 45.000

61.13.173.211.1233>140.135.101.74.2105 866487.000 619 1399.817

23

TCP Application (cont.)

– P2P applications• Napster (6677, 6688 port)• eDonkey (4661, 4662 port)• KaZaA (1214 port)

– Streaming Media• MediaPlayer (1755 port)• RealPlayer (7070 port)• QuickTime Player (554 port)

– Others

24

UDP Application

• DNS – server accepts client’s gethostbyip() or gethostbyname() calls on fixed 53/UDP service port.

• Tftp – server accepts client connection requests for control stream on 69/UDP service port

– generates data streams to client sites using dynamic UDP port piggybacked from client for transmitting data

26

TFPT Traffic Logs …

=======================================================

140.138.143.42 . 1068 > 140.130.120.14.69 135 .000 3 43.000

140.138.143.42.1052>140.130.120.14.4902 423232.000 13226 32.000

140.130.120.14.4902>140.138.143.42.1052 6837536.000 12569 544.000

140.138.143.42.1053>140.130.120.14.1978 432160.000 13505 32.000

140.130.120.14.1978>140.138.143.42.1053 7098656.000 13049 544.000

…

140.138.143.42.1064>140.130.120.14.3613 428128.000 13379 32.000

140.130.120.14.3613>140.138.143.42.1064 7235744.000 13301 544.000

140.138.143.42.1065>140.130.120.14.4692 431168.000 13474 32.000

140.130.120.14.4692>140.138.143.42.1065 7148160.000 13140 544.000

140.138.143.42.1066>140.130.120.14.1833 445024.000 13907 32.000

27

UDP Application (cont.)

• VoIP (NeetMeeting)– server accepts request using fixed UDP service ports that ranges from 49601 to 49609.

• SIP (Service Initiation Protocol)– IP multicast traffic– using class D IP addresses ranging from 224.1.0.0 through 239.255.255.255

28


• Virus (Linux Slapper Worm)– adopts the same UDP service and client ports

– UDP ports adopted by Slapper Worm including •2002, 2001, 1978, 4156, 1812, 6257•depends on the different virus versions.

29


• Video Games– short lifetime synchronous flows.– Counter_Strike

• uses 27015 UDP service ports– Quake_3

• uses the fixed 27960, 27961,27962 UDP service ports.

– StarCraft• uses the fixed 6112 service port.

30


• Streaming Media Traffic– Microsoft’s Media Player

•採用 1024 ~65535 range的可用 UDP service port 傳送 RTP 視音訊封包– 大量視音訊流的同時存取– 避開 firewall對承載 media stream的過濾

•media過境訊務統計– 無法依據 fixed service port 辨識轉送訊務log

– 加總 media訊務量

31

4.Streaming Media傳訊模式 • Streaming Media轉送訊務的追蹤

–追蹤 streaming media flow–歸納視音訊務傳輸特徵–統計 WAN streaming media轉送訊務

32

實測網路• TANet網路採階層的連線架構

•桃園區域網路 router界接•TANet其他區網中心•ISP 網路業者•區域連線學校

•15 個大學 ,30 個高中職 ,數百個國中小學• Gigabit Ethernet• T1/E1 (1.544 Mbps/ 2.048 Mbps)

33

TANet Backbone

Campus Network

Campus Network

Campus Network

Domeatic ISP Backbone

Campus Network

Campus Network

Connected Subnet

TYCRouter

ISPRouter

BackboneRouter

Router

Router

Router

Router

BackboneRouter

BackboneRouter

BackboneRouter

EdgeRouter

TYCRouter Router

Router

Router

Router Router

Router

骨幹 Router及階層式連網架構

34

Streaming轉送訊務的追蹤• Step 1: 依據 streaming media大packet size傳輸特性 ,擷取 packet size大於 500 bytes/packet UDP flow logs.

• Step 2: 依據 flow identifiers (source IP/ source port, destination IP /destination port), 追蹤包含各 client IP 的轉送 flows紀錄 .

35

• Step 3: 歸納的 media傳訊特性 – RTSP stream

• RealVideo選用 7070/TCP service port• Quick Time 則選用 554/TCP RTSP service port• MediaServer採用 1755/TCP的 RTSP service port

– RTP/RTCP stream• RealVideo 與 Quick Time

– 採用 6970-7170 UDP client ports 建立 RTP連接– Send RTCP feedback in reverse direction

• MediaServer– 採用 1755/UDP RTCP service port– 選用動態 UDP ports RTP建立連接 ,1024-65535 ports

36

• 歸納 MediaPlayer RTP 與 RTCP 傳輸streaming的規律性 .

• MS Media Server– RTSP agent以 1755/TCP port 等待MediaPlayer (client) 連接

– server依據回應的 client RTP/RTCP port pairs• 分別建立 RTP傳輸與 RTCP控制 streams,• 遞送 real-time audio/video content,session control訊息

37

MedieaPlay(client)

MedieaServer

$RTSP_clnt_port /TCP( 1024 < clnt_port < 65535)

Port 1755/TCP(RTSP session control)

(RTCP session control)

$Audio_server_port /UDP( 1024 ~65535)

(RTP streaing audio )

Port 1755 /UDP

( $RTCP_client_port +1 ) /UDP( 1024 ~65535)

$RTCP_client_port /UDP( 1024 ~65535)

38

MediaPlay(client)

MediaServer(RTSP stream)

(Audio_RTCP stream)

RTP_server_port /UDP

(Audio RTP stream)

Port 1755 /UDP

Audio_RTP_client_port(Audio_RTCP_client_port +1)

Audio_RTCP_client_port

Port 1755 /UDP( Video_RTCP stream)

Video_RTCP_client_port

Rtp_server_port /UDPVideo_RTP_client_port

(Video_RTCP_client_port +1) (Video RTP stream )

39

5. Video Game 訊務特性–絕大半的 UDP flow 轉送紀錄承載 DNS 及 Video Game synchronization 連接

•Short-life•Carrying small traffic amount

–相當高比率的 Video Gaming 訊務為承載小訊務量 flows

40

Game 訊務特性 (cont.)•依據 src_port, dst_port 為 key 統計轉送 flow count分布

– Top-10高連結頻次的 UDP應用包括 : • 使用 53 service port 的 DNS,• 使用 27015, 27020 service port 的Counter_Strike

• 使用 4665 port 的 eDonkey_Msg,•Windows Netbios-ns 的 137 service port

41

• 依據 transportation port為 key 統計video game flow 封包數分布– 98% 的 Counter_Strike flows傳輸的封包數目小

於 10 packets,– 90% 的 Quake_3 flows– 73% 的 StarCraft flows.– 傳送封包數大於 50 packets的大訊務量 game flows 為特殊 port 的 Game flows (Table 3)• 93.4 % 的 27015 – 27005 CS flows, • 100% 的 27960—27960 Quake 3 flows,• 95% 的 6112-6112 StarCraft flows.

42

TYC連網的 UDP訊務 flow count 分布UDP Port Flow_cnt

1 DNS (53) 18174512 CS_game (27015) 2480183 EdonkeyMsg (4665) 207804 NTP (123) 148275 CS_game (27020) 145406 Netbios-ns (137) 104967 Snmp (161) 67118 Mplayer RTSP

(1755) 6036

9 MSNmessenger (6257)

179210 Tftp (69) 1764

43

TYC連網的 Video Game flows分布pkts/flow

CS flows Quake flows StarCraft flows

1 54672 (22.032 %) 492 (57.143 %) 6293 (43.091 %)2 ~ 10 189312 (76.291%) 252 (29.268 %) 3834 (26.253 %)11 ~ 20 81 (0.033 %) 30 (3.484 %) 647 (4.430 %)21 ~ 30 13 (0.005 %) 11 (1.278 %) 317 (2.171 %)31 ~ 40 154 (0.062 %) 1 (0.116 %) 255 (1.746 %)41 ~ 50 65 (0.026 %) 1 (0.116 %) 167 (1.144 %)> 50 3846 (1.55%)

27005 -27015 93.4 % 74(8.59%)27960-27960 100% 3091 (21.16%)

6112-6112 94.6 %others 6.6 % others 0.0 % other

s5.4 %

44

6.MediaPlayer Traffic統計演算法– Step 1: Extract the transportation addresses of the high-bandwidth streams.•Parse the per-10-min UDP traffic logs– determine the likely RTP flow logs by checking mean packet size feature

– extracting the likely transportation address of RTP client sites» destination IP addresses» destination UDP client ports

45

Step 2: Estimate the transportation addresses of the likely RTCP streams •Estimated the client transportation addresses of likely RTCP streams– destination IP addresses– destination UDP client ports satisfy

» RTCP_client_port = RTP_client_port - 1.

46

– Step 3: Extract parallel RTCP/RTP flow logs • Extract parallel RTP/RTCP streaming logs from the likely streaming log-files

• determine the RTP/RTCP flow logs– checking the transportation address of RTSP

streams» IP addresses obtained from step 1 and 2» if there exist the relative RTSP session

streams using 1755/TCP service ports..

47

– Step 4: Filter and accumulate media player traffic volume•Filter out the prevalent video game flow logs from the extracted RTCP and RTP flow logs, such as Counter_Strike and Quake.

•Accumulate media player traffic for each time interval

48

** Streaming Media Flow logs 210.155.134.232.1755>140.135.155.99.1777 I 592.000 10 59.200 //RTSP 140.135.155.99.1778>210.155.134.232.1755 3604.000 80 45.050 //RTCP 210.155.134.232.3836>140.135.155.99.1779 484543.000 336 1442.092 //RTP

** Fast Streaming Media Flow logs211.72.248.106.1755>140.138.243.167.1098 1192.000 13 91.692 // RTSP

140.138.243.167.1110>211.72.248.106.1755 1524.000 34 44.824 //RTCP

211.72.248.106.4344>140.138.243.167.1111 943500.000 629 1500.000 //RTP

211.72.248.106.2857>140.138.243.167.1121 357802.000 239 1497.079 //RTP

211.72.248.106.4498>140.138.243.167.1131 58705.000 59 995.000 //RTP

211.72.248.106.3773>140.138.243.167.1211 38246.000 26 1471.000 //RTP

211.72.248.106.2963>140.138.243.167.1140 671246.000 448 1498.317 //RTP

49

TANet桃園區網 MediaPlayer訊務分布

50

• MediaPlayer traffic volume in byte– our measurement network ( Nov in 2002)

– streaming media訊務低峰•清晨時段 (4:00–8:00).

– streaming media訊務高峰•午休 ~ 凌晨時段

51

桃園區網 Top-10 Content Sites 排行

• Top-10 Content Sites–加總各 host IP media 訊務量

•大多 content sites 來自 Domestic ISPs– Giga.net.tw,– Hinet.net

52

1 203.187.31.130 (giga.net.tw) 1108.190 (MB)

2 202.39.43.241 (.HINET-IP.hinet.net) 611.629 (MB)


4 61.219.53.133 61-219-53-133. ET-IP.hinet.net 323.279 (MB)

5 210.58.102.73 73.c102.etceHINnter.net 105.744 (MB)

6 203.187.31.162 (giga.net.tw) 62.811 (MB)

7 192.192.40.2 www2.vit.edu.tw 48.616 (MB)


9 203.187.31.191 (giga.net.tw) 44.563 (MB)

10 140.115.11.117 center3-3.dd.ncu.edu.tw 43.650 (MB)

桃園區網的 Top-10 Content Sites 排行

53

11 203.187.31.161 (giga.net.tw) 36.286 (MB)

12 61.219.24.21 61-219-24-21.HINET-IP.hinet.net 31.780 (MB)

13 61.218.130.110 61-218-130-110.HINET-IP.hinet.net 21.119 (MB)

14 203.187.31.163 (giga.net.tw) 17.654 (MB)

15 211.106.66.139 (kt.co.kr) 16.851 (MB)

16 202.1.239.19 tai-wmarc03.tai.bcst.yahoo.com 16.016 (MB)

17 203.187.31.198 (giga.net.tw) 15.352 (MB)

18 163.26.220.51 elearning.stut.edu.tw 14.231 (MB)

19 210.172.39.105 cache55.smartstream.ne.jp 14.156 (MB)


54

• UDP應用訊務分布– DNS, TFTP– Game, Streaming, VoIP

• TCP應用訊務分布– Mail, Telent– FTP, WWW– P2P File sharing– Streaming Media

7. 熱門應用訊務分布

55

熱門 UDP應用訊務分布• Measured transaction statistics reveals– Media player

• dominates UDP input traffic in bytes• representing 35.3% of the total UDP traffic

volume– Counter_Strike GAME 8.5%– DNS 6.9%– Tftp 6.0%– Real Player 3.7%– Others involves the emerging applications

• using the UDP services port selected by users

56

TANet桃園區網的單日 UDP應用訊務分布

57

UDP應用訊務分布 (cont.)• In addition DNS and Tftp

– many applications are adopting competitive UDP transmission protocol to transmit real-time traffic

– Emerging streaming media and video games were the most popular UDP applications.

58

UDP應用訊務分布 (cont.)– Streaming Media contributed almost 40% UDP traffic volume.

– Others contributed 24% of TCP traffic volume•Video Game ports (user assigned)

– Game ports detection » Statistics features

•Emerging application•Virus/Worm

59

UDP應用訊務分布 (cont.)• Mean packet size of streaming media – definitely exceeds that of other UDP applications•media player is 1237 bytes/packet•Real Player is 648 bytes/packet.

60

桃園區網 UDP應用訊務封包大小分布 UDP_Services

Pkt_Size (Bytes / Pkt)

MediaPlay 1237SIP 1307

Virus (Slapper Worm) 937

RealPlay 648VoIP 563Tftp 293

CS_GAME 119DNS 108

Quake3 92StarCraft 63Edonkey_Msg 42

Others 131

61

TCP應用訊務分布• Date Service Bytes Pkt_size Ratio• =====================================================• 1203 :: Napster, 721925306, 693.840, 17.50, %• 1203 :: WWW, 709071230, 693.560, 17.18, %• 1203 :: Edonkey, 670955496, 621.942, 16.26, %• 1203 :: FTP, 162008675, 793.650, 3.93, %• 1203 :: Mail, 71543955, 696.853, 1.73, %• 1203 :: KaZaA, 25725377, 841.928, 0.62, %• 1203 :: RTSP, 20852075, 656.929, 0.51, %• 1203 :: Others, 1740544499, 632.336, 42.18, %

62

TCP應用訊務分布 (cont.)

63

• WWW & P2P file sharing– contributed almost 50% TCP traffic volume.

– The mean packet size of each TCP applications is 700 bytes/packet.

• Others contributed 42% of TCP traffic volume– Fast FTP adopt dynamic ports to transmit large traffic volume

– New emerging application– Virus/Worm traffic

65

8. 結語• 透過 router轉送 NetFlow紀錄

– 追蹤 streaming media訊務 flow log,– 找出其建立 RTP 與 RTCP傳輸 streaming的規律性

(1) 採 1755/udp 的 RTCP service port, (2) RTP 及 RTCP client port 為連續正整數 ;

– streaming訊務統計方法適用於一般 WAN連網 .– 量測 TANet 桃園區網 MediaPlayer streaming media與熱門的 UDP應用訊務 .

66

• Numerical Result shows – TCP based traffic dominate over UDP traffic about 98% of total traffic

– Windows Media streams dominate the Streaming UDP Traffic Volume• Media streams account 85%• Real streams account for 15%

– TCP based streaming Traffic dominate over UDP being used in about 60% of all bytes transfered

67

– Top-10 content sites •Accumulated media traffic volume with the indexing of source IP address based on the extracted media player RTP flow logs.

•Numeric Result helps– observing the main streaming traffic contributors

– observing the increasing growth of the commercial streaming traffic.

68

• video game traffic statistic– Short-live game UDP connection

•Low-bandwidth synchronous traffic– member that join each game creating connection to the others at very high frequency•2MN

– 2: In/out– M: local members (in the same network)– N: Foreign members (in other networks)

69

– significant transportation features of game• be used to identify

– new emerging video games– game traffic accounting.

– our experience in transportation traffic measurement of streaming media • could be easily applied to measure streaming traffic over general– campus router– backbone router

• help network user grasping the media traffic growth, observing long-term traffic growth and planning network capacity.

internet streaming media udp-based 轉送訊務的量測

Documents