web rtc+webaudio

Web RTC & Web Audio

2012. 6. @erucipe 최 승 준

The mission of the Audio Working Group, part of the Rich Web Client Activity, is to define a client-side script API adding more advanced audio capabilities than are currently offered by audio elements. The API will support the features required by advanced interactive applications including the ability to process and synthesize audio streams directly in script. The HTML5 specification introduces the <audio> and <video> media elements, including an API to play back prerecorded audio and video files and to get limited information about the media, such as duration. The Audio Working Group will build upon and expand that basic functionality.

background

2010 2011 2012

W3C Audio Incubator Group

12 May 2010 ~ 28 March 2012

http://www.w3.org/2005/Incubator/audio

Mozilla Audio Data API

14 May 2010 ~

https://wiki.mozilla.org/Audio_Data_API

Mozilla Firefox 4

22 March 2011

2010-08-26 nightly builds @humphd

Google Chrome 14

16 September 2011

2011-01-31 nightly builds

public-xg-audio@w3.org public-audio@w3.org

May 2010 ~ December 2011 March 2011 ~

http://lists.w3.org/Archives/Public/public-xg-audio http://lists.w3.org/Archives/Public/public-audio

( Chromium ) Web Audio API

W3C Web Audio API

1 June 2010 ~ 18 July 2011 18 July 2011 ~

http://chromium.googlecode.com/svn/ trunk/samples/audio/index.html

http://www.w3.org/TR/webaudio

W3C Audio Working Group

25 March 2011 ~

http://www.w3.org/2011/audio

MIDI Device Communication API

MediaStream Processing API

crogers@google.com

detail

WebGL Camp 3, June 2011 http://www.youtube.com/watch?feature=player_detailpage&v=6YYFsmjCKX4#t=873s

var context = new AudioContext(); var playSound = function { var source = context.createBufferSource(); source.buffer = dogBarkingBuffer; source.connect(context.destination); source.noteOn(0); };

demo

use cases http://www.w3.org/2011/audio/wiki/Use_Cases_and_Requirements

UC 1 Video Chat

UC 2 HTML5 game with audio effects, music

UC 3 Online music production tool

UC 4 Online radio broadcast

UC 5 writing music on the web

UC 6 wavetable synthesis of a virtual music instrument

UC 7 Audio / Music Visualization

UC 8 UI/DOM Sounds

UC 9 Language learning

UC10 Podcast on a flight

UC11 DJ music at 125 BPM

UC12 Soundtrack and sound effects in a video editing tool

UC13 Web-based guitar practice service

UC14 User Control of Audio

UC15 Video commentary

so what ?

위 이미지는 Web Audio와 직접적인 관련이 없음

MIDI Device Communication API The MIDI API specification defines a means for web developers to enumerate, manipulate and access MIDI devices. Having an API for MIDI gives a means to make various applications using existing software and hardware synths, as well as light systems and other mechanical apparatus controlled by MIDI, along with a method of communication with existing DAW (Digital Audio Workstation), trackers and other music software on the user's computer.

https://dvcs.w3.org/hg/audio/raw-file/tip/midi/specification.html

위 이미지는 Web Audio와 직접적인 관련이 없음

advanced interactive app MIDI

Web RTC

MediaStream Processing API W3C Working Group Note 31 May 2012

background

The mission of the Web Real-Time Communications Working Group, part of the Ubiquitous Web Applications Activity, is to define client-side APIs to enable Real-Time Communications in Web browsers.These APIs should enable building applications that can be run inside a browser, requiring no extra downloads or plugins, that allow communication between parties using audio, video and supplementary real-time communication, without having to use intervening servers (unless needed for firewall traversal, or for providing intermediary services).

茫茫大海

focus

media capture API a.k.a

getUserMedia

implementations

ericsson opera google mozilla

Demos Here are a couple of demo sites. Most of them were sent to us via the discuss-webrtc list. Please remember that the same disclaimer as above applies: Justin Uberti (Chrome-webrtc team member) has sent in a App Engine based 1:1 video calling app.http://apprtc.appspot.com/ source code: http://code.google.com/p/webrtc-samples/source/browse/trunk/apprtc/ Justin Uberti (Chrome-webrtc team member) has sent in a App Engine based face detection apphttp://apprtc.appspot.com/html/face.html Paul Neave sent in a great video effects app. http://neave.com/webcam/html5/ Greg Miernicki was officially the first to send in a demo: http://miernicki.com/cam.html Eric Bidelman sent in an additional photo effects demos: http://html5-demos.appspot.com/static/getusermedia/photobooth.html And another let's have fun with WebGL sent in by Jérome Étienne: http://webglmeeting.appspot.com An HTML5 SIP client http://www.sipml5.org/call.htm If you find another demo app, please send it to sergel@webrtc.org / the discuss list or our Google+ Page. We will try to put it up!

demo

if (navigator.webkitGetUserMedia) { var video = document.createElement('video'); var gotStream = function(stream) { video.src = webkitURL.createObjectURL(stream); video.onerror = function () { stream.stop(); console.log('camera error'); }; }; var noStream = function() { console.log('no camera available'); }; navigator.webkitGetUserMedia('video', gotStream, noStream); document.body.appendChild(video); video.autoplay = true; } else { console.log('no native camera support available'); }

P2P

AR http://www.html5rocks.com/en/tutorials/webgl/jsartoolkit_webrtc

http://www.html5rocks.com/en/tutorials/webgl/jsartoolkit_webrtc/AR_mediaStream_three.html

Let’s Put It All Together?

security issues https://wiki.mozilla.org/Security/Discussions/WebRTC

conclusion

Web RTC & Web Audio

2012. 6. @erucipe 최 승 준