口試投影片(詹智傑) final
Post on 21-May-2015
2.433 views
TRANSCRIPT
自適性串流架構下之視訊影像品質最佳化設計與實作
指導教授:楊士萱學生:詹智傑
Design and Implementation of Quality Optimization Strategy in Adaptive Streaming
2
Outline
• Introduction to MPEG-DASH • Related Works• Proposed State-based Strategy• Results and Future Works
3
Outline
• Introduction to MPEG-DASH • Related Works• Proposed State-based Strategy• Results and Future Works
4
Introduction to MPEG-DASH
• Dynamic Adaptive Streaming over HTTP– 由 3GPP 與 MPEG 一起訂定的自適性串流標準– 屬於 HTTP/TCP 的串流協定– 唯一的自適性串流標準
5
Introduction to MPEG-DASH
• Dynamic Adaptive Streaming over HTTP– 何謂自適性串流 (Adaptive bit-rate streaming)
• 依據環境與應用需求即時性的調整視訊串流之碼率– 視訊之解析度 (Resolution) 與品質 (Quality) 需求– 網路狀態與撥放延遲 (playback delay)– 客戶端之暫存器使用效率 (Video buffer)– 使用終端之計算能力與複雜度
• 目的是獲得更佳之使用經驗品質 (Quality of Experience, QoE) 與頻寬使用效率 (Bandwidth efficiency)
6
Introduction to MPEG-DASH
• Dynamic Adaptive Streaming over HTTP– 使用 HTTP/TCP (not RTSP/UDP) 的優勢
• 多數使用終端支援 HTTP 協定• TCP 能提供高可靠性 (reliable) 且有秩序的傳送
(ordered transmission) 並且能避免 NAT 與防火牆之阻擋問題
• HTTP-based streaming 能夠重複使用已建置之 HTTP servers 與 caches 做資料內容 (content delivery) 的傳遞
• HTTP-based streaming 將串流連線之控制權由 Server端移轉至 Client 端
7
Introduction to MPEG-DASH
• Dynamic Adaptive Streaming over HTTP– 世界上第一個也是唯一的 International standard
• Apple HTTP Live Streaming (Apple-HLS)• Microsoft Live Smooth Streaming (Microsoft-LSS)• Adobe Dynamic Streaming for Flash (ADOBE-DSF)• OPEN IPTV Forum: HTTP Adaptive Streaming• Dynamic Adaptive Streaming over HTTP (MPEG-DASH)• …
Draft International Standard : 2011 Jan
International Standard : 2011 Nov
ISO/IEC International Standard : 2012 April
ISO/IEC Technical Corrigendum 1 : 2013 May
8
Introduction to MPEG-DASH
• 系統架構 (System Architecture)
HTTP server DASH clientSegment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
Segment
MediaPresentationDescription
(MPD)
Control heuristics
HTTP client
MPD parser
Segment parser
Mediaplayer
HTTP 1.1
視訊串流服務之”目錄”資料模
型
視訊串流服務之”媒體”資料模
型
9
Introduction to MPEG-DASH
• 資料模型 (Data Model)– Media Presentation Description (MPD)
• Descript the video segment’s information in xmlex: timing, codec, resolutions, bit-rate … etc.
– Segments• The media data.• ex: mp4, m4s, mp3 … etc.
10
Introduction to MPEG-DASH
• 資料模型 (Data Model)– Media Presentation Description (MPD)Media presentation
DescriptionPeriod ID = 1, start=0 s
Period ID = 2, start=100 s
Period ID = 3, start=200 s
Period ID = 2start=100s
AdaptationSet 0
AdaptationSet 1
AdaptationSet 2
Adaptation set 1
Representation 1 5Mbps
Representation 2 2Mbps
Representation 3 500kbps
Representation 22 Mbytes
Segments
11
Introduction to MPEG-DASH
• 系統流程圖 (System Flowchart)
Process MPD
HTTP Get MPD
Measure Download Rate
Select Representation
Measure Download Rate
HTTP Get Segment #1
HTTP Get Segment #2
#1
#2
12
Introduction to MPEG-DASH
• 客戶端系統流程圖 (Client side system flowchart)
Main part of MPEG-DASH
Segment buffer
13
Major contribution of this thesis
• 實際建置一個包含伺服器與客戶端之視訊串流系統,並加入 MPEG-DASH 框架
• 我們採用 emulation 的形式,相較於simulation 能提供更真實、更有修改與加入環境參數彈性的系統
• 我們探索視覺品質 (visual quality) 、網路頻寬(network bandwidth) 以及暫存器使用量(buffer usage) 的關聯性,並提出一套自適性品質控制以提升感知上的品質 (perceived quality)
14
Outline
• Introduction of MPEG-DASH • Related Works
– VLC Player MPEG-DASH rate-based Quality Strategy
– Related Researches in MPEG-DASH• Proposed State-based Strategy• Results and Future Works
15
VLC Player Rate-based Quality Strategy
• 實作於 VLC Player DASH plug-in 中
• Main consider principle– 當暫存器容量低於預設
threshold ,接收最低碼率串流以加速暫存器容量補充
< minimum buffer threshold
> minimum buffer threshold
NoneAt least one
Algorithm start
InitialMPD ManagerCurrent Period
Get current Bandwidth
Check current Buffer capacity
Set current bandwidth as zero
Gather Representations withsame resolution of the player
Check the quantity of Representations
in candidate set
Find target Representationfrom every AdaptationSets
Find target Representationfrom candidate set
Algorithm end
Stage one
Stage two
16
VLC Player Rate-based Quality Strategy
• Advantage– 視訊片段緩衝器補充速度快
• Disadvantage– 串流初始狀態的品質等級低– 易受到網路瓶頸而品質崩潰
17
VLC Player Rate-based Quality Strategy
• Advantage• Disadvantage
– 串流初始狀態的品質等級低– 易受到網路瓶頸而品質崩潰
Bottleneck
18
Outline
• Introduction of MPEG-DASH • Related Works
– VLC Player MPEG-DASH rate-based Quality Strategy
– Related Researches in MPEG-DASH• Proposed State-based Strategy• Results and Future Works
19
Related Researches in MPEG-DASH
• [23] 以馬可夫鍊的方式對無線網路的頻寬變化特性做分析與歸類,並提出找尋最佳視訊品質的演算法。
• [25] 探討如何使用可調式編碼 (Scalable video coding) 提升網路儲存空間的使用效率。
• [26] 以代理伺服器 (Proxy) 的角度出發,以網路管理的 QoE 機制來替客戶選擇最佳的視訊品質。
20
Related Researches in MPEG-DASH
• 雖然過去兩年 MPEG-DASH 相關的投稿文獻絡繹不絕,但多數的共通點都如 [23][25] 採用純模擬 (Simulation) 或數學推導,無法於實際網路中驗證。
• 故本論文除了提出一套 MPEG-DASH 的品質調控機制,也將實作一套系統進行驗證。
21
Outline
• Introduction of MPEG-DASH • Related Works• Proposed State-based Strategy• Results and Future Works
22
Proposed State-based Quality Strategy
• 本論文提出之 DASH 品質調控機制分 two stage– First Stage :以下載 MPD 的網路資訊預測前幾
個 Segment 的最佳品質– Second Stage :依照網路與視訊片段暫存器使
用率來預測下一個 Segment 的最佳品質
23
Proposed State-based Quality Strategy
• Flowchart of first stage– Case 1:
• MPD 檔中的第一個Segment 。
– Case 2:• 新 Adaptation set 的第一
個 Segment 的決策流程。– Case 3:
• 一般之 Segment 的決策流程
Choise segmentStart
!First segment
!First segment in MPD
No
1 2 3
NoYes
Yes
InitialGet target RepresentationList
Sort RepresentationsRepresentationStream
24
Proposed State-based Quality Strategy
1
Download InitializationSource URL
Download first five segmentby using MPD’s bitrates as ref.
Compute and save five segment’s downloaded bitrates
End
2
Download InitializationSource URL
Choice target level according to pre-algorithm’s last five segments
Compute and save segment’s downloaded bitrates
End
Download target level segments
3
Choice target level according tofive pre-segments
Download target level segments
Compute and save segment’s downloaded bitrates
End
Case 1: Case 2: Case 3:
25
Proposed State-based Quality Strategy
• Finite state machine of second stage
State 1-1
State 2-3
State 2-2
State 2-1
Network Throughputs > Highest Representation
Network Throughputs < Highest Representation
Network Throughputs < 80% Representations
Network Throughputs < 80% Representations
NetworkThroughputs
(Bps)
SegmentBuffer
(%)0
Adaptation Set’sRepresentation
(Level)
Level n
Level 1
Level 2
State 3-2 State 3-1
Level k
Network Throughputs < Representations Level 2
Network Throughputs > Representations Level 2
Buffer Fill persentage > 30%
Buffer Fill persentage > 30%
30% 50%Buffer Fill persentage > 50%
Buffer Fill persentage > 50%
Case One
Case Two
Case Three
26
Outline
• Introduction of MPEG-DASH • Related Works• Proposed State-based Strategy• Results and Future Works
– Emulation Settings– Objective Evaluation– Subjective Evaluation
27
Emulation Settings
• 實驗前提– 本論文探討“頻寬高於 MPD 所能提供的最低碼
率視訊片段”的情境– 本論文實作的是 On demand 的視訊串流服務– 本論文探討“ Client有足夠計算能力解析 MPD
中任一等級的串流”– 本論文假設碼率與 PSNR 的關係為已知,故不須於解碼的同時檢測畫面品質。
28
Emulation Settings
• 測試平台 Client ServerName Asus PC Asus PCCPU Intel Core 2 Quad
2.67GHzIntel Core 2 Quad 2.67GHz
Memory 4G DDRII 2G DDRIIOS Windows 7 Ubuntu 12.04Network Ethernet Ethernet
29
Emulation Settings
• 測試平台
HTTP Server
Apache Server(on demand)
NetworkEmulator
Client
PC
Notebook
HTTP
Internet
DASH-AVC/264Encoder
30
Emulation Settings
• 所使用之開源式軟體Open Source Software
Operating System Linux Ubuntu 12.04 LTE
Web HTTP Server Apache 2
Server Database MySQL
Language Perl, Bash script
Network Emulator Linux tc commands
Network Monitor libpcap & winpcap
Server Monitor Monitorix
Client side libraries libdash, libcurl, libxml2, libav
31
Emulation Settings
• 測試序列Parameter Value
Length (min:sec) 09:46
Segment size (second/segment) 2
Numbers of Frames 14315
Video codec H.264/AVC
Container MP4 & M4S
Frame rate (fps) 25 fps
GOP size (frame) 25
Resolution (pixel) 480 x 360
32
Emulation Settings
• 測試序列Level Video bitrate Level Video bitrate
0 100 kbits/s 7 1300 kbits/s
1 200 kbits/s 8 1600 kbits/s
2 350 kbits/s 9 1900 kbits/s
3 500 kbits/s 10 2300 kbits/s
4 700 kbits/s 11 2800 kbits/s
5 900 kbits/s 12 3400 kbits/s
6 1100 kbits/s 13 4500 kbits/s
33
Emulation Settings
• 序列特性:品質等級與碼率關係
The lower the bit-rate is, the smaller the gap is.
34
Emulation Settings
• 網路模型– 高品質穩態網路模型
• 網路頻寬維持於 MPD 最高品質等級之上
– 遞增型網路模型• 網路頻寬由低頻寬以穩定之速率上升
– 遞減型網路模型• 網路頻寬由高頻寬以穩定之速率下降
35
Emulation Settings
• 網路模型
0 100 200 300 4000
1000000
2000000
3000000
4000000
5000000
6000000
7000000
最高品質穩態網路模型 遞增網路模型 遞減網路模型
Time(s)
Bitr
ate(
kbits
/sec
)
36
Outline
• Introduction of MPEG-DASH • Related Works• Proposed State-based Strategy• Results and Future Works
– Emulation Settings– Objective Evaluation– Subjective Evaluation
Objective Evaluation
• 頻寬使用效率分析– 最高品質穩態網路模型
0 10 20 30 40 50 600
1000000
2000000
3000000
4000000
5000000
6000000
7000000
Bandwidth Throughput(VLC[31]) Throughput(Proposed)ChosenBitrate(VLC[31]) ChosenBitrate(Proposed)
Time(sec)
Bitr
ate(
kbit
/sec
)
37
Objective Evaluation
• 頻寬使用效率分析– 最高品質穩態網路模型
0 10 20 30 40 50 600
1000000
2000000
3000000
4000000
5000000
6000000
7000000
Bandwidth Throughput(VLC[31]) Throughput(Proposed)ChosenBitrate(VLC[31]) ChosenBitrate(Proposed)
Time(sec)
Bitr
ate(
kbit
/sec
)
38
0 10 20 30 40 50 600
1000000
2000000
3000000
4000000
5000000
6000000
7000000
012345678910111213
Bandwidth Throughput(VLC[31]) Throughput(Proposed)ChosenLevel(VLC[31]) ChosenLevel(Proposed)
Time(sec)
Bitr
ate(
kbit
/sec
)
Leve
l(#)
Objective Evaluation
• 頻寬使用效率分析– 遞增網路模型
39
Objective Evaluation
• 頻寬使用效率分析– 遞減網路模型
40
Objective Evaluation
• 頻寬使用效率分析– 遞減網路模型
41
Objective Evaluation
• 使用者觀賞品質分析– 最高品質穩態網路模型
42
Objective Evaluation
• 使用者觀賞品質分析– 最高品質穩態網路模型
43
+4.02845 dB
Objective Evaluation
• 使用者觀賞品質分析– 最高品質穩態網路模型
• Average PSNR
44
PSNRY PSNRU PSNRV
Proposed 41.9227dB 47.8613 dB 49.9766 dBVLC[31] 41.4475 dB 47.4507 dB 49.5599 dB△PSNR +0.4752 dB +0.4106 dB +0.4167 dB
Objective Evaluation
• 使用者觀賞品質分析– 遞增網路模型
45
Objective Evaluation
• 使用者觀賞品質分析– 遞減網路模型
46
Objective Evaluation
• 使用者觀賞品質分析– 遞減網路模型
47
Objective Evaluation
• 使用者觀賞品質分析– 遞減網路模型
• Average PSNR
48
PSNRY PSNRU PSNRV
Proposed 39.311 dB 47.814 dB 48.930 dBVLC[31] 38.728 dB 47.325 dB 48.449 dB△PSNR +0.583 dB +0.489 dB +0.481dB
Objective Evaluation
• 使用者觀賞品質分析– 遞減網路模型
• Average PSNR (from 8000~12000 frame)
49
PSNRY PSNRU PSNRV
Proposed 38.411 dB 44.065 dB 44.54 dBVLC[31] 36.93 dB 42.875 dB 43.325 dB△PSNR +1.481 dB +1.19 dB +1.215 dB
畫面數 : 8000~12000張
50
Outline
• Introduction of MPEG-DASH • Related Works• Proposed State-based Strategy• Results and Future Works
– Emulation Settings– Objective Evaluation– Subjective Evaluation
51
Subjective Evaluation
• 串流起始狀態– 最高品質穩態網路模型 (frame 829)
Proposed (36.160dB) VLC[31] (27.810dB)
52
Subjective Evaluation
• 串流起始狀態– 遞增網路模型 (frame 279)
Proposed (30.796dB) VLC[31] (27.837dB)
53
Subjective Evaluation
• 串流起始狀態– 遞減網路模型 (frame 373)
Proposed (35.063dB) VLC[31] (26.696dB)
54
Subjective Evaluation
• 串流起始狀態 (Demo)– 最高品質穩態網路模型
Proposed VLC[31]
55
Subjective Evaluation
• 串流起始狀態 (Demo)– 遞減網路模型
Proposed VLC[31]
56
Subjective Evaluation
• 遞減網路模型低視訊品質狀態– frame 9872
Proposed (44.129dB) VLC[31] (27.555dB)
57
Subjective Evaluation
• 遞減網路模型低視訊品質狀態– Demo
Proposed VLC[31]
58
Results and Future Works
• 本論文實作一套包含伺服器與客戶端的自適性串流系統。
• 本論文在客戶端提出一套視訊品質評估機制。
• 整體 PSNR 相較於 [31] 提升約 0.5dB;在初始狀態上,也能達到約 1dB 到 4dB 的提升;而在網路瓶頸時也有 1dB 的提升。
59
Results and Future Works
• 提升品質調控機制的準確度• 擴充其他視訊編碼標準與其相關特性探討• 切割自適性串流機制與實際解碼的框架,
以做為智慧型家庭、微型視訊共享機制的系統雛形。
60
Reference• [1] Cisco, Cisco Visual Networking Index: Global Mobile Data Traffic
Forecast Update, 2012–2017, Feb 6, 2013.• [2] R. Pantos, and W. May, HTTP Live Streaming, version 10, IETF Internet-
Draft, Oct 15, 2012.• [3] Information technology — Dynamic Adaptive Streaming over HTTP
(DASH), ISO/IEC 23009-1:2012 ,Aprl 01, 2012.• [4] Jack Y. B. Lee , Scalable Continuous Media Streaming Systems-
Architecture, Desgn, Analysis and Implementation, John Wiley & Sons Ltd, June, 2005.
• [5] H.264: Advanced Video Coding for Generic Audiovisual Services, ITU-T H.264, Jan, 2012.
• [6] I. E. G. Richardson, H.264 and MPEG-4 Video Compression: Video Coding for Next Generation Multimedia, John Wiley & Sons, 2003
• [7] 經濟部 , 2003 數位內容產業年鑑 趨勢篇 第二章 數位內容技術趨勢
61
Reference• [8] Microsoft Corporation, IIS Smooth Streaming Technical Overview, Mar
25, 2009.• [9] Apple Corporation, HTTP Live Streaming Overview, April 1, 2011.• [10] Information technology — Dynamic Adaptive Streaming over HTTP
(DASH) — Part 1: Media Presentation Description and Segment Formats, AMENDMENT 1: Support for Event Messages and extended Audio Channel Configuration, ISO/IEC 23009-1:2012/DAM 1, Dec 07, 2012.
• [11] Information technology — Dynamic Adaptive Streaming over HTTP (DASH) — Part 1: Media Presentation Description and Segment Formats, TECHNICAL CORRIGENDUM 1, ISO/IEC 23009-1:2013/Cor.1:2012(E), Feb 02, 2013.
• [12] 蔡德明 , 鳥哥的 Linux私房菜基礎學習篇第二版 , 上奇科技出版事務處 , March, 2006.
• [13] 蔡德明 , 鳥哥的 Linux私房菜伺服器架設篇第二版 , 上奇科技出版事務處 , March, 2007.
62
Reference• [14] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, W. Weiss, An Archi-
tecture for Differentiated Services, IETF RFC 2475, Dec 1998, http://tools.ietf.org/html/rfc2475
• [15] M. Allman, V. Paxson, W. Stevens, TCP Congestion Control, IETF RFC 2581, Aprl 1999, http://tools.ietf.org/html/rfc2581
• [16] D. Robinson, K. Coar, The Common Gateway Interface (CGI) Version 1.1, IETF RFC 3875, Oct 2004, http://tools.ietf.org/html/rfc3875
• [17] Information technology — Dynamic adaptive streaming over HTTP (DASH) — Part 2: Conformance and reference software, ISO/IEC 23009-2:2012 ,May 28, 2012.
• [18] N. Cranley, P. Perry, and L. Murphy, “User perception of adapting video quality,” ACM International Journal of Human-Computer Studies, pp. 637-647,Aug. 2006
63
Reference• [19] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, "Overview of
the H.264/AVC video coding standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol.13, no.7, pp.560,576, July 2003
• [20] M. Wien, H. Schwarz, T. Oelbaum, "Performance Analysis of SVC,” IEEE Transactions on Circuits and Systems for Video Technology, vol.17, no.9, pp.1194,1203, Sept. 2007
• [21] A. Begen, T. Akgul, M. Baugher, " Watching Video over the Web: Part 1: Streaming Protocols,” IEEE Internet Computing, vol.15, no.3, pp.59-63, May-June 2011
• [22] I. Sodagar, "The MPEG-DASH Standard for Multimedia Streaming Over the Internet,” IEEE Multimedia, vol.18, no.4, pp.62-67, April 2011
• [23] S. Xiang, L. Cai, and J. Pan, “Adaptive scalable video streaming in wireless networks,” MMSys'12 Proceedings of the 3rd Multimedia Systems Conference, pp. 167-172, 2012
64
Reference• [24] V. Swaminathan,and Wei Sheng, "Low latency live video streaming
using HTTP chunked encoding,” 2011 IEEE 13th International Workshop on Multimedia Signal Processing (MMSP), pp.1,6, 17-19, Oct. 2011
• [25] Y. Sánchez, T. Schierl, C. Hellge, and T. Wiegand, “iDASH: improved dynamic adaptive streaming over HTTP using scalable video coding,” MMSys'11 Proceedings of the second annual ACM conference on Multimedia systems, pp.257-264, Feb 2011
• [26] R.K.P. Mok, X. Luo, E. W. W. Chan, and R. K. C. Chang, "QDASH: A QoE-aware DASH system,” MMSys'12 Proceedings of the 3rd Multimedia Systems Conference, pp.11-22, Feb. 2012
• [27] O. Nemcic, M. Vranjes, and S. Rimac-Drlje, “Comparison of H.264/AVC and MPEG-4 part 2 coded video,” Proc. Electronics in Marine (ELMAR), pp.41,44, 12-14, Sept. 2007
65
Reference• [28] T. Lohmar, T. Einarsson, P. Fröjdh,F. Gabin, M. Kampmann, “Dynamic
Adaptive HTTP Streaming of Live Content,” IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks, June 20-24, 2011
• [29] T. Stockhammer, “Dynamic Adaptive Streaming over HTTP–Standards and Design Principles,” MMSys'11 Proceedings of the second annual ACM conference on Multimedia systems, Feb 2011,
• [30] S. Lederer, C. Müller, and C. Timmerer, “Dynamic Adaptive Streaming over HTTP Dataset,” In Proceedings of the ACM Multimedia Systems Conference 2012, Chapel Hill, North Carolina, February 22-24, 2012.
• [31] C. Müller and C. Timmerer. “A VLC media player plugin enabling dynamic adaptive streaming over HTTP,” In Proceedings of the 19th ACM international conference on Multimedia, New York, NY, USA, 723-726, 2011.
66
Reference• [32] L. R., Romero, “A Dynamic Adaptive HTTP Streaming Video Service for
Google Android,” Oct 06, 2011, Available from: http://dat.etsit.upm.es/~luciano /en/master-thesis.pdf
• [33] Jan Ozer, “What is streaming media protocol?,” http://www.streaming media.com/Articles/Editorial/What-Is-.../What-Is-a-Streaming-Media-Protocol-84496.aspx
• [34] Liz Gannes, “The Next Big Thing in Video: Adaptive Bitrate Streaming,” http://pro. gigaom.com/blog/how-to-deliver-as-much-video-as-users-can-take/
• [35] wiki Adaptive streaming: http://en.wikipedia.org/wiki/Adaptive_bitrate_stre aming
• [36] Adaptive Streaming Comparison: http://learn.iis.net/page.aspx/792/adaptive -streaming-comparison/
• [37] Video LAN Official page: http://www.videolan.org/vlc/index.html
67
Reference• [38] September 2012 Web Server Survey :
http://news.netcraft.com/archives/2012/09/10/september-2012-web-server-survey.html
• [39] Ubuntu documentation: Guide line for installing LAMP in Linux Ubuntu : ht tps://help.ubuntu.com/community/ApacheMySQLPHP
• [40] Monitorix Project by Jordi Sanfeliu : http://www.monitorix.org/• [41] RRDtool – About RRDtool :
http://oss.oetiker.ch/rrdtool/index.en.html• [42] libdash | bitmovin.net multimedia :
http://www.bitmovin.net/libdash/• [43] libav- Open source audio and video processing tools :
http://libav.org/• [44] Hierarchical token bucket theory :
http://luxik.cdi.cz/~devik/qos/htb/man ual/theory.html• [45] Traffic control HOWTO :
http://linux-ip.net/articles/Traffic-Control- HOWTO/