http://www.kawa.net/http://www.kawa.net/ 11
JSARToolKit / LiveChromaKey
2009.05.27Yusuke Kawasaki
川﨑 有亮 (kawanet)http://www.kawa.net/
Spark project 勉強会 #09
http://www.kawa.net/http://www.kawa.net/ 22
Yusuke Kawasaki(川﨑 有亮)
• Perl monger– XML::TreePP, XML::FeedPP, ...
• JavaScript addict– Wiimote over HTTP, Gainer, ...
• ActionScript 3.0 newbie– Today!
• Media Technology Labs– Recruit Co., Ltd. in Japan
http://www.kawa.net/http://twitter.com/kawa0117
http://www.kawa.net/http://www.kawa.net/ 33
§§1 1 -- JSARToolKitJSARToolKit
JavaScript Augmented RealityJavaScript Augmented Reality
http://www.kawa.net/http://www.kawa.net/ 44
JSARToolKit
• JSAR(ジャバスクリプト拡張現実)– JavaScript Augmented Reality
• JSARToolKit– My first work using ActionScript 3.0– Powered by SWFObject! – A library to run AR by JavaScript– A proxy wrapper for a Flash app using FLARToolKit– Debuted at OSDC.TW 2009 in Taipei
• Genealogy– 2008.03 – NyARToolKit for Java - by A⻁@nyatla.jp– 2008.05 – FLARToolKit for ActionScript3 by Saqoosha– 2009.04 – JSARToolKit for JavaScript by Kawanet
• Saqoosha さん、本当にありがとうございます!!_o_
http://www.kawa.net/http://www.kawa.net/ 55
JSARToolKit Synopsis
It needs just several lines of JavaScript code to run AR.When a marker found, it calls onDetect callback function.
< s c r i p t t y p e = " t e x t / j a v a s c r i p t " s r c = " s w f o b j e c t . j s " > < / s c r i p t >< s c r i p t t y p e = " t e x t / j a v a s c r i p t " s r c = " j s a r . j s " > < / s c r i p t >< s c r i p t t y p e = " t e x t / j a v a s c r i p t " > < ! - -
v a r p a t s = [ ' j s a r l o g o . p a t ' ] ;v a r j s a r = n e w J S A R ( ' j s a r _ h e r e ' ) ;j s a r . d r a w M a r k e r R e c t = t r u e ;j s a r . o n D e t e c t e d = f u n c t i o n ( r e s u l t ) { . . . } ;j s a r . i n i t ( ) ;j s a r . s e t M a r k e r ( p a t s ) ;
- - > < / s c r i p t >< d i v i d = " j s a r _ h e r e " > < / d i v >
http://www.kawa.net/http://www.kawa.net/ 66
JSAR returns a JavaScript Object
{" d e t e c t e d " : t r u e ," m a r k e r s " : [{
" c o d e i d " : 0 ," d i r e c t i o n " : 2 ," c o n f i d e n c e " : 0 . 5 1 2 6 3 7 ," v e r t e x " : [{ " x " : 3 4 6 , " y " : 2 0 1 } ,{ " x " : 2 7 7 , " y " : 1 9 1 } ,{ " x " : 3 5 3 , " y " : 1 3 0 } ,{ " x " : 3 9 5 , " y " : 1 5 4 }
]} ,
] ,}
http://www.kawa.net/http://www.kawa.net/ 77
JSARToolKit Layers
Flash Layer jsar.swf
HTML Canvas Layer
HTML Div Layer
http://www.kawa.net/http://www.kawa.net/ 88
AiR Xiaolongbao(エア⼩籠包)
2009.04.19 @ Taipei (OSDC.TW)
http://www.kawa.net/http://www.kawa.net/ 99
§§2 2 -- LiveChromaKeyLiveChromaKey
BluescreenBluescreen--less augmented less augmented ININ realityreality
http://www.kawa.net/http://www.kawa.net/ 1010
LiveChromaKey
• A new image synthesizing engine for AR– 2009.05 released by Kawanet (me!)
• Pure ActionScripe 3.0– No library dependencies– No JavaScript
• Only webcam needed– No blue back screen– No need to print
AR marker PDF
http://www.kawa.net/http://www.kawa.net/ 1111
Step 1/3 - Stationary background
At first, connect you webcam and run LiveChromaKey.Never move until it recognizes stationary background.
http://www.kawa.net/http://www.kawa.net/ 1212
Step 2/3 – Bluescreen on the fly
It generates bluescreen automatically.Live Video – Background = Bluescreen
- ⇒
http://www.kawa.net/http://www.kawa.net/ 1313
Step 3/3 - Transparent foreground
Get the foreground image as a translarent Sprite.Live Video – Bluescreen = Foreground
- ⇒
http://www.kawa.net/http://www.kawa.net/ 1414
LiveChromaKey Synopsis
LiveChromaKey provides four kinds of Sprites.
v a r c h r o m a k e y : L C K _ C o r e = n e w L C K _ C o r e ( ) ;c h r o m a k e y . i n i t ( ) ;
v a r s p L i v e : S p r i t e = c h r o m a k e y . g e t L i v e ( ) ;v a r s p B a c k : S p r i t e = c h r o m a k e y . g e t B a c k g r o u n d ( ) ;v a r s p M a s k : S p r i t e = c h r o m a k e y . g e t M a s k ( ) ;v a r s p F o r e : S p r i t e = c h r o m a k e y . g e t F o r e g r o u n d ( ) ;
t h i s . a d d C h i l d ( s p L i v e ) ;t h i s . a d d C h i l d ( s p B a c k ) ;t h i s . a d d C h i l d ( s p M a s k ) ;t h i s . a d d C h i l d ( s p F o r e ) ;
http://www.kawa.net/http://www.kawa.net/ 1515
Example:Example: Travelling in EgyptTravelling in Egypt
egypt.swfegypt.swf
http://www.kawa.net/http://www.kawa.net/ 1616
Sprites augmented IN reality
The key feature of LiveChromaKey is to insert Sprites augmented between background and foreground images.
Sprite
http://www.kawa.net/http://www.kawa.net/ 1717
Example:Example: Slideshow in realitySlideshow in reality
minority.swfminority.swf
http://www.kawa.net/http://www.kawa.net/ 1818
MMultiarmed ultiarmed ddeityeity (or (or 美川健⼀美川健⼀))
exile.swfexile.swf
http://www.kawa.net/http://www.kawa.net/ 1919
ひとりひとり おそ松くんおそ松くん
exile.swfexile.swf
http://www.kawa.net/http://www.kawa.net/ 2020
ひとりひとり ExileExile
exile.swfexile.swf
http://www.kawa.net/http://www.kawa.net/ 2121
§§3 3 -- LivePointersLivePointers
Live Color Pointer Detection LibraryLive Color Pointer Detection Library
http://www.kawa.net/http://www.kawa.net/ 2222
LivePointers
• Live Color Pointer Detection Library–will be released soon!
• Pure ActionScripe 3.0– No library dependencies– No JavaScript
• Webcam as a 3D human interface– No need to print AR marker PDF
http://www.kawa.net/http://www.kawa.net/ 2323
Pointer = Pixel Cluster
• Ex.–Cluster #0–250 pixels–RGB: 25D985–H: 152–S: 0.83–V: 0.85
http://www.kawa.net/http://www.kawa.net/ 2424
Labeled Pointer Detection
• Labeled pointer needs a guide color.
• ex. fingercap– 0x2197A9 (uint)– ¥105(キャン・ドゥ)
• Fingercap is definitelyimportant user interfacedevice of the future.
http://www.kawa.net/http://www.kawa.net/ 2525
LivePointers(⽇本語のみスマソ)
• 最短距離法 (Nearest Neighbor Clustering) をベースにして、ActionScript 3.0 でリアルタイムに表⽰できる程度に単純化したクラスタ分析処理を⾏う。
• 画⾯内の単⾊のカラーマーカーを検出します。– ⽩・⿊・グレー・淡⾊は検出しません。– (S・V値が⾼い)くっきりした⾊が得意です。– 模様は検出しません。(⾊のみを認識する)– 通常は、画⾯内の全ての検出⽤ガイドとしてRGBで
⾊指定した「ラベル付マーカー」を利⽤する。
• 3次元⼊⼒のポインティングデバイスとして利⽤可能。
http://www.kawa.net/http://www.kawa.net/ 2626
Now on the Spark!
JSARToolKithttp://www.libspark.org/svn/js/JSARToolKit/
LiveChromaKeyhttp://www.libspark.org/svn/as3/LiveChromaKey/
LivePointers... coming soon!
http://www.kawa.net/http://www.kawa.net/ 2727
Thank you!
Yusuke Kawasaki川﨑 有亮 (kawanet)
http://www.kawa.net/