WebRTC 개발, 현재와 미래

최진호 (jinho.c@sk.com) / IT기술원 SK Telecom

Q&A

WebRTC와 킬러서비스

Q&A

WebRTC와 킬러서비스

Viblast

Q&A

대상: WebRTC로 영상채팅앱 개발을 따라하기 해본 개발자. !두어시간 만에 뚝딱! 어? 생각보다 쉽잖아? 좋아! !이제 나만의 서비스를 개발해보자!

누구를 위한 발표인가?

핵심 사이트 • http://webrtc.org • http://dev.w3.org/2011/webrtc/editor/webrtc.html 주요 튜토리얼 정보!• apprtc.appspot.com / WebRTC Codelab • http://www.html5rocks.com/ko/tutorials/webrtc/basics Blog 및 커뮤니티 • http://webrtchacks.com • https://groups.google.com/forum/#!forum/discuss-webrtc

WebRTC 개발의 기본

TURN!STUN!Media

Signaling!HTTPS

Javascript App

Browser 1

Media!Streams

PeerConnection

Javascript App

Browser 2

Media!Streams

PeerConnection

navigator.getUserMedia()! video.src = localStream!RTCPeerConnection() for local,remote! addStream! create offer, answer! set Description for local and remote! onIceCandidate —> addIceCandidate

SIP, XMPP, WebSocket, DataChannel 등의 시그널링

WebRTC 서비스의 기본 flowSVCAAppBrowser B App Browser

로그인로그인

방관리방관리

Call BMedia

PC

SDP offerAlarm Media

PCSDPAnswer

Check Check

Media MediaDTLS/ SCTP, SRTPRealtime network transport

시그널링

Q&A

그러나! • 브라우저 지원 제약 - IE, Safari, Android Default Browser… • Codec, 품질, Multiparty 연결 처리, 익숙치않은 개발 방식과 개념…

!제대로된 서비스를 만드려면 여전히 어려움

그러나?

제대로된 서비스 개발을 위해…

App

Web API

Server

App

Web API

Voice Video Data

Browser

WebRTC C++ API

ICE

Svc Server

Signal Server

SDP

SRTP

SCTP

SIP

DTLS

STUN

TURN

ORTC

제대로된 서비스 개발을 위해…

A B

IP:Port IP:Port

NAT NAT

A’ B’

WebRTC CoreCall Flow와 표준/프로토콜Signaling

Media and Communication

processing

protocols!!

ICE: Interactive Connectivity Establishment (RFC 5245) STUN: Session Traversal Utilities for NAT (RFC 5389) TURN: Traversal Using Relays around NAT (RFC 5766) SDP: Session Description Protocol (RFC 4566) DTLS: Datagram Transport Layer Security (RFC 6347) SCTP: Stream Control Transport Protocol (RFC 4960) SRTP: Secure Real-Time Transport Protocol (RFC 3711)

NAT와 WebRTC

Client

src: 192.168.0.123:5575!dest: 65.55.32.20:5060

NAT Binding!192.168.0.123:5575 <—> 1.2.3.4:6432

src: 1.2.3.4:6432!dest: 65.55.32.20:5060

Client A Client B

STUN!서버

나는 누구?1.2.3.4:1234

Client A Client B

1.2.3.4:1234!Symetric NAT

TURN서버

ICE!!

a=candidate:4022866446 1 udp 2113937151 1.2.3.4 36768 typ srf x raddr 192.1.2.3 rport 12345 a=candidate:4022866446 2 udp 2113937151 1.2.3.4 36768 typ srf x raddr 192.1.2.3 rport 12345 !Foundation(4022866~): 동일 type, interface, STUN서버의 모든 candidate을 위한 Unique ID Component ID (1,2): 1은 RTP, 2는 RTCP Transport (udp): 전송 type Priority(211393~): 해당 component의 우선순위. 높을 수록 좋음 Type(host, srf…): component type 관련 주소: relayed candidate는 reflexive svr candidate 주소

Interactive Communication Establishment(RFC 5245) 1. Candidate 수집

2. 정렬

3. 인코딩

4. offer/answer

5. 중복 제거 및 Connectivity check

6. Coordination and Communication

SDP!!

v=0 a=group:BUNDLE audio video data a=msid-semantic: WMS Bojy6E0V9qwZtscZIfKflfecOMDTiv151amI m=audio 1 RTP/SAVPF 111 103 0 8 106 105 13 126 c=IN IP4 0.0.0.0 a=rtcp:1 IN IP4 0.0.0.0 a=ice-ufrag:CsfC78jBGAXz7Ir/ a=ice-pwd:8JR9PhCbW3xxc1VaSYwRmclM a=ice-options:google-ice a=fingerprint:sha-256 4C:3B:B9:CE:57:2B:D9:A0:D9:1C:B7:91:4D… a=setup:actpass a=mid:audio a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=fmtp:111 minptime=10 a=rtpmap:103 ISAC/16000 …… a=ssrc:2787795017 cname:BuWzBTE1UlgUf0V7 a=ssrc:2787795017 msid:Bojy6E0V9qwZtscZIfKflfecOMDTiv151a… a=candidate:2077988028 1 udp 2122260223 1… a=candidate:2077988028 2 udp 2122260223 172.2… ……

m=video 1 RTP/SAVPF 100 116 117 96 c=IN IP4 0.0.0.0 a=rtcp:1 IN IP4 0.0.0.0 a=ice-ufrag:CsfC78jBGAXz7Ir/ a=ice-pwd:8JR9PhCbW3xxc1VaSYwRmclM a=ice-options:google-ice a=fingerprint:sha-256 4C:3B:B9:CE:57:2B:D9:A0:D9:1C:B7:03:A… a=extmap:2 urn:ietf:params:rtp-hdrext:toffset a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=sendrecv a=rtcp-mux a=rtpmap:100 VP8/90000 a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli a=rtcp-fb:100 goog-remb a=rtpmap:116 red/90000 a=rtpmap:117 ulpfec/90000 a=rtpmap:96 rtx/90000 a=fmtp:96 apt=100 a=ssrc-group:FID 2912859667 954580083 a=ssrc:2912859667 cname:BuWzBTE1UlgUf0V7 a=ssrc:2912859667 msid:Bojy6E0V9qwZtscZIfKflfecOM …….. m=application 1 DTLS/SCTP 5000 c=IN IP4 0.0.0.0 a=ice-ufrag:CsfC78jBGAXz7Ir/ a=ice-pwd:8JR9PhCbW3xxc1VaSYwRmclM a=ice-options:google-ice a=fingerprint:sha-256 4C:3B:B9:CE:57:2B:D9:A0:D9:1C:B7:03:A6:… a=setup:actpass a=mid:data a=sctpmap:5000 webrtc-datachannel 1024 ….

DTLS

DTLS 표준은 RFC 4347로 UDP에 대한 안전한 전송 이외에는 TLS와 거의 유사 SRTP는 마스터키 및 마스터시드를 외부에 의존 ! DTLS마스터 비밀키 사용! 전송되는 데이터의 무결성(Integrity)와 기밀성(Confidentiality) 보장.

SRTP and SRTCP!

Secure Real-time Transport Protocol (SRTP) , RFC 3550 • Secure profile of the standardized format for delivery of real-time data, such as audio

and video over IP networks. Secure Real-time Control Transport Protocol (SRTCP), RFC • Secure profile of the control protocol for delivery of sender and receiver statistics and

control information for an SRTP flow.

UDP기반의 모든 실시간 미디어 스트림의 표준 프로토콜

RTP header

WebRTC Core

WebRTC 크로미엄 소스 • http://www.webrtc.org/reference/getting-started

WebRTC Core

빌드 방법 및 개발환경 구축

빌드 순서(Android, Linux)!1) gclient config https://webrtc.googlecode.com/svn/trunk!2) gclient sync --nohooks!3) echo target_os = [\'android\', \'unix\'] >> .client!4) ./trunk/build/install-build-deps.sh --no-chromeos-fonts!5) ./trunk/build/install-build-deps-android.sh!6) check 'sudo update-alternatives --config java!7) source ./build/android/envsetup.sh!8) export GYP_GENERATOR_FLAGS="$GYP_GENERATOR_FLAGS output_dir=out_android"!9) export GYP_DEFINES="$GYP_DEFINES OS=android"!10) export GYP_DEFINES="build_with_libjingle=1 build_with_chromium=0 libjingle_java=1 $GYP_DEFINES"!11) gclient runhooks!12) Ninja Build ! * ninja -C out_android/Debug -j10 All ! * ninja -C out_android/Debug -j10 AppRTCDemo or ninja -C out_android/Release -j10 AppRTCDemo

Prerequisite!!- Ubuntu 64bit / 보통은 virtualbox에서…!- JDK 1.6.0_45!- su 모드에서 진행을 권장!- apt-get install git subversion libpulse-dev g++ pkg-

config gtk+-2.0 libnss3-dev!- apt-get install bison fakeroot flex g++ g++-multilib

gperf libapache2-mod-php5 libasound2-dev libbz2-dev libcairo2-dev libdbus-glib-1-dev libgconf2-dev libgl1-mesa-dev libglu1-mesa-dev libglib2.0-dev libgtk2.0-dev libjpeg62-dev libnspr4-dev libnss3-dev libpam0g-dev libsqlite3-dev libxslt1-dev libxss-dev mesa-common-dev msttcorefonts patch perl pkg-config python python2.5-dev rpm subversion libcupsys2-dev libgnome-keyring-dev libcurl4-gnutls-dev libelf-dev libc6-i386 lib32stdc++6 libbluetooth-dev!

- git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git!

- mkdir webrtc_project

WebRTC Core

Namespace 관계도

cricket

rtc

WebRTC Interface & Implementation

Primitives, Thread, Socket etc

Middle Layer Implementation

webrtc::acm2

webrtc::audioproc

webrtc::RTCPHelp

webrtc::RTCPUtitlity

webrtc::RtpUtility

webrtc::vcm

webrtc::videocapturemodule

webrtc::voe

webrtc

WebRTC Core주요 클래스

webrtc:: PeerConnectionFactory

webrtc:: PeerConnection

cricket:: ChannelManager

webrtc:: WebRtcSession

webrtc:: MediaSignaling

webrtc:: WebRtcSessionDescriptionFactory

cricket:: VideoChannel

cricket:: VoiceChannel

cricket:: DataChannel

cricket:: WebRtcVideoMediaChan

nel

cricket:: WebRtcVoiceMediaChan

nel

cricket:: WebRtcVoiceEngine

cricket:: HybridDataEngine

cricket:: SctpDataEngine

cricket:: RtpDataEngine

cricket:: DataMediaChannel

cricket:: WebRtcVideoEngine

cricket:: WebRtcVideoCapturer

webrtc:: DeviceManager

webrtc:: MediaStream

webrtc:: VideoTrack

webrtc:: AudioTrack

webrtc:: VideoSource

webrtc:: AudioSource

webrtc:: PeerConnectionObserver

cricket:: TransportChannel

cricket:: MediaMonitor

webrtc:: SessionDescription

Application

creation use member

WebRTC 서비스의 성능과 품질

영상기반 서비스의 어려운 점 - 기대하지 않은 사용자측에서의 오류 발생 - 웹캠이나 잘못된 비디오 드라이버 혹은 다른 앱과의 충돌등등 - 무조건 서비스 탓만 한다…ㅜ!- 모든 것을 모니터링 하고 품질 측정해야함 ( 통신중의 n/w quality, 양 끝단의 영상품질, 반복적인 짧은 통신, 때로는 불법적인 서비스 사용까지도) - 사용자의 네트워크와 단말에 따른 동적인 constraint 적용 필요

주요 품질 지표

Packet Losses

RTT /!Delay Throughput Inter-Arrival

Time

Setup Time Call Failure MOS

PeerConnection.getStats() 에서 제공되는 metric들로 대부분 측정 가능

WebRTC-Internal tool

Constraints

general video constraints !minAspectRatio, maxAspectRatio maxWidth, minWidth maxHeight, minHeightmaxFrameRate, minFrameRate

audio constraints googEchoCancellation, googEchoCancellation2 googAutoGainControl, googAutoGainControl2 googNoiseSuppression, googNoiseSuppression2 googHighpassFilter googTypingNoiseDetection googAudioMirroring

google constraints for local video source googNoiseReduction googLeakyBucket googTemporalLayeredScreencast

createoffer/createAnswer를 위한 constraint OfferToReceiveAudio OfferToReceiveVideo VoiceActivityDetection IceRestart

PeerConnection 생성 때 필요한 constraints DtlsSrtpKeyAgreement RtpDataChannels

google 특화 constraint googDscpgoogIPv6googSuspendBelowMinBitrate googImprovedWifiBwe: 개선된 BWE 알고리즘 googScreencastMinBitrate googCpuOveruseDetection CPU모니터 가동여부googCpuUnderuseThreshold googCpuOveruseThreshold googCpuUnderuseEncodeRsdThreshold googCpuOveruseEncodeRsdThreshold googCpuOveruseEncodeUsagegoogHighStartBitrate 보다 빨리 높은 bitrate 원래는 500kbps미만googHighBitrate googVeryHighBitrate googPayloadPaddinggoogOpusFec

State 처리

new 객체가 만들어졌고 아직 네트워킹은 일어나지 않음

gathering ICE agent가 local candidate를 모으고 있는 중

complete ICE agent 가 gathering을 끝냈음

RTCIceGatheringState

RTCPeerStateRTCIceConnectionState

State 처리

시작

Offer

Answer

Candidates

Stable new

have-remote-offer

stable

gathering

completenew

checking failed

connected

disconnected

completed

Stable

have-local-offer

Stable

new

gathering

complete

failed

new

checking

connected

disconnected

completed

create PC

onIceCandidate!send candidate

addLocalStream!setLocal!

sending offer

setRemote!addRemoteStream

addIceCandi…

create PC!set Remote offer!

addRemoteStream!add local Stream!

create answer!setLocal!

send answer

onIceCandidate!send candidate

addIceCandidate

RTCIceGatheringStateRTCPeerStateRTCIceConnectionState

Codec…

H.264 vs VP8

전체 아키텍처

Client logic for Web or Native

API / Interface

Voice Video Network

Service Logic

Server Logic

Signaling (SIP, WS…)

STUN

TURN

MCU

API 인증

채널 lifecycle 관리품질관리

모니터링

연계 과금 및 고객 관리

서비스개발에만 전념하고 싶다면?

그 외 TEMASYS, easyRTC 등등 다양한 프레임워크

Feature Tokbox Addlive Bistri SightCall

Chrome O O O O

Firefox O O O O

I.E O O O

Safari O O O

iOS O O O

Android O O O O

multy party conferencing O (max 20) O O

(max 4) O

Screen Sharing O O O

H.264 O O

Network Recording O

Usage analytics/statistics O O O

현장에서 느끼는 WebRTC시장의 미래…

Why?IoT

B2B

Q&A

THANK YOU