the not so short introduction to kinect
DESCRIPTION
重庆LUG8期聚会KINECT主题PPTTRANSCRIPT
About Me
Chongqing University bachelor （单身汉）A boy couldn't stop dreamingBike travelling
About Him
•Dr.•Sensor related•Poet （魔幻现实主义）
About the Lab
•50% Idle•25% Computer•20% Programmer•5% Student/Paper
About You?
Programming dummy?Linux || Windows?Understand 3D?Work with the Kinect?
The Contentswhat u will learn today
体感操作， Kinect ， OPENNI 介绍基本程序设计Working with Depth ImageWorking with Point Clouds （ * ）Working with Skeleton Data & TrackingKinect Hacks
昨夜西风凋碧树 , 独上高楼 , 望尽天涯路
Buy it!
Inside Kinect
TeardownPrincipleHardwarePerformace
Hardware & teardown
http://www.ifixit.com/Teardown/Microsoft-Kinect-Teardown
Kinect Hardware
•RGB camera•IR depth sensor•IR projector•Audio device
Take a closer look
Noise at edges(shoulders)Depth shadowsMisalignment between the color and depth imagesFrame rate(30 vs 60)
Behind Kinect
TOF
•Time of Flight•以色列公司 3DV （ 09 年微软收购）•光飞行时间，飞秒 (1e−15) 级快门•砷化镓，使其时钟频率提高到上百•高成本，难以适应消费电子品•Natal 并不是基于 ToF 的原理
Light Coding （光编码）
•连续的照明（而非脉冲）•普通的 CMOS 感光芯片（低成本）•结构光技术（三维纵深的“体编码” ）•激光散斑（ laser speckle ）
http://www.javaforge.com/wiki/103639 http://www.wipo.int/pctdb/en/wo.jsp?WO=2007043036 http://www.freepatentsonline.com/7433024.pdf
Exemplar
•机器学习系统•TB 计的数据训练集
1.寻找移动部位（背景分离，分割遮罩）2.辨认身体部位可能性3.模型匹配：生成骨架系统
Audio
•Something we forget•Supported by OpenNI, not by SensorKinect
More About Audio （题外话）•“Sound Wave”•微软研究院和华盛顿大学•利用多普勒效应，使用笔记本内置的扬声器和麦克风开发出类似 Kinect 的姿势感知系统•低成本•识别精度能达到 90% 以上•用手势玩俄罗斯方块游戏（视频）
Concepts
http://en.wikipedia.org/wiki/Natural_user_interface
Next generation interface
Wii Remote
ASUS Xtion
Kinect
Project Glass
Development
http://www.openni.org/
http://code.google.com/p/simple-openni/
http://openkinect.org/wiki/Main_Page
https://github.com/OpenKinect/libfreenect
Architecture
Quick view(*)
OpenNI
•Short for “Open Natural Interface”•Multi-langue•Cross-platform•Standard & Infrastructure
向后兼容
使用 openNI 开发的程序在以后的版本中都可以继续使用 , 因为开发人员可以毫无顾虑的更新 openNI到最新的版本。
OpenNI Framework(1)
抽象分层视图
•Top: 代表软件，在 OpenNI 顶部，实现了自然交互的应用程序。•Middle: 代表 OpenNI ，提供通信接口，同时连接传感器和中间件组件，后者分析来自传感器的数据。•Bottom: 硬件设备，捕捉现场的视频和音频内容。
OpenNI Framework(2)
通用的框架工具集
•一个 USB 抽象访问层•一些基本数据类型的实现（包括列表，哈希，等等）•系统日志和转储•内存和性能分析•活动（授权要注册的回调函数到一个特定的事件）•任务调度•不是标准 OpenNI 的一部分（ * ）
Callback func
•函数指针•注册•事件响应
OpenNIの构成
Generator Metadata Capability
Generator
注：绿色已经有，红色计划中，后略
生产节点
•单位集•生产性角色•使用较低水平的生产节点•“有意义的三位数据”
Sensor Related
•Device•Depth Generator•Image Generator•IR Generator•Audio Generator
Middleware Related
•Gesture Generator•Scene Analyzer•Hands Generator•User Generator•//provided by NITE
Record & Play
•Recoder•Player•Codec
Matadata
Capability
NITEの构成
SessionDetector
Middleware
•Full body analysis•Hand point analysis•Gesture detection•Scene Analyzer
Session
Detector
衣带渐宽终不悔，为伊消得人憔悴。
Hello world!
•Preparation•The program
Software Preparation
•OpenNI+NITE•Avin’s SensorKinect•Processing(*)•SimpleOpenNI(*)
https://github.com/avin2/SensorKinect
Here’s the code,too young too simple
import SimpleOpenNI.*;SimpleOpenNI kinect;
void setup(){ size(640*2, 480); kinect = new SimpleOpenNI(this);
kinect.enableDepth(); kinect.enableRGB();}
void draw(){ kinect.update();
image(kinect.depthImage(), 0, 0); image(kinect.rgbImage(), 640, 0);}
Understand the code
import SimpleOpenNI.*;SimpleOpenNI kinect;
•Import the library•Declare the SimpleOpenNI object
void setup(){ size(640*2, 480); kinect = new SimpleOpenNI(this);
kinect.enableDepth(); kinect.enableRGB();}
•Declare the size of app•Instantiate the object•Call two method
void draw(){ kinect.update();
image(kinect.depthImage(), 0, 0); image(kinect.rgbImage(), 640, 0);}
•Get fresh data from kinect•Place the image
Working with depth map
•R G B•纯黑代表无穷远，纯白代表无穷近•Demo:ex03_basic_depth_plus_rgb
More about depth map
•Projective•Realworld•Z值相同，代表深度，单位 mm•ConvertProjectiveToRealWorld()
The Kinect coordinate system
Demo
chPC_ex12_obj_in_point_cloud
范围以及精确性
•最近距离： about 0.5(?)•最远距离： 10m•GetDeviceMaxDepth()•官方建议： 1.2 – 3.6米•近远精确性比较
Working with Point Clouds(*)
Welcome to the Third Dimension3d reconstructionE.g:KinectFusion PCL
http://pointclouds.org/
DEMO
chPC_ex05_rotating_point_cloud
Working with Skeleton Data
•User Detection•Joints & Skeleton•Distance in 3D•Scene Map, Com•Tracking
User Detection
•Calibration or not•Stages in Process
Joints & Skeleton
•Joints•Distance between two joints in 3D
WindowsSDK: 20 joints(all)
OpenNI: 24 joints
DEMO
chSK_ex02_skel_anatomy
Scene Map
•Background remove•User pixels•Center of Mass
Demo
•chSK_scene_map•chSK_scene_image•chSK_center_of_mass
How
•kinect.enableScene();•sceneMap = kinect.sceneMap();•if(sceneMap[i] == n)
Tracking
Demo:PointViewer
众里寻他千百度，蓦然回首，那人却在，灯火阑珊处。
Kinect Hacks
Apps out of boxCreate your own cool stuffsLiving coding with VS & Cpp
Media Art
Demo: chDE_face_changing
How
•Collision detection•kinect.getJointPositionSkeleton()•kinect.convertRealWorldToProjective()•Picture scala
Computer Interface
Demo:Minority Report(*)Demo:2D TrackPadDemo:Hotpoint 3D
FAAST
•Key event mapping•Game playing
http://projects.ict.usc.edu/mxr/faast/
Sex Game Controller
•DEMO: 宅男妄想地带•Microsoft: NO!!
Sorry, Bill
Control emacs(*)
•Map(Joint position + gesture, key event)
Kinect Visual Camera(*)
http://www.kinectdevs.com/forums/kinect-virtual-camera-kinect-webcam-f17
The following demo need a better computer for higher GPU
support
kinect-ultra(*)
一秒钟凹凸曼合体！完成你童年时候的梦想Video
http://code.google.com/p/kinect-ultra/
kinect-kamehameha(*)
MikuMikuDance(*)
初音妹子Video
http://www.kinect-hacks.com/openni
What do you want to make?
reference•http://viml.nchc.org.tw•Visualization and Interactive Media Laboratory of NCHC•国家科學視算與互動媒體實驗室
•http://www.javaforge.com/wiki/103532•Kinect for PC Wiki•中文
Thanks!You are the controller.Q & AHang [email protected]://www.xhbang.com
This is basic template
Do not delete this page