the not so short
TRANSCRIPT
Chongqing University bachelor（单身汉）
A boy couldn't stop dreaming
Bike travelling
•Dr.
•Sensor related
•Poet（魔幻现实主义）
•50% Idle
•25% Computer
•20% Programmer
•5% Student/Paper
Programming dummy?
Linux || Windows?
Understand 3D?
Work with the Kinect?
体感操作，Kinect，OPENNI介绍
基本程序设计
Working with Depth Image
Working with Point Clouds（*）
Working with Skeleton Data & Tracking
Kinect Hacks
昨夜西风凋碧树,独上高楼,望尽天涯路
Buy it!
Teardown
Principle
Hardware
Performace
Hardware & teardown
http://www.ifixit.com/Teardown/Microsoft-Kinect-
Teardown
•RGB camera
•IR depth sensor
•IR projector
•Audio device
Noise at edges(shoulders)
Depth shadows
Misalignment between the color and depth images
Frame rate(30 vs 60)
Behind Kinect
•Time of Flight
•以色列公司3DV（ 09年微软收购）
•光飞行时间，飞秒(1e−15)级快门
•砷化镓，使其时钟频率提高到上百
•高成本，难以适应消费电子品
•Natal并不是基于ToF的原理
•连续的照明（而非脉冲）
•普通的CMOS感光芯片（低成本）
•结构光技术（三维纵深的“体编码” ）
•激光散斑（laser speckle）
http://www.javaforge.com/wiki/103639
http://www.wipo.int/pctdb/en/wo.jsp?WO=2007043036
http://www.freepatentsonline.com/7433024.pdf
•机器学习系统
•TB计的数据训练集
1. 寻找移动部位（背景分离，分割遮罩）
2. 辨认身体部位可能性
3. 模型匹配：生成骨架系统
•Something we forget
•Supported by OpenNI, not by SensorKinect
•“Sound Wave”
•微软研究院和华盛顿大学
•利用多普勒效应，使用笔记本内置的扬声器和麦克风开发出类似Kinect的姿势感知系统
•低成本
•识别精度能达到90%以上
•用手势玩俄罗斯方块游戏（视频）
Concepts
http://en.wikipedia.org/wiki/Natural_user_interface
Next generation interface
Wii Remote
ASUS Xtion
Kinect
Project Glass
Development
http://code.google.com/p/simple-openni/
http://openkinect.org/wiki/Main_Page
https://github.com/OpenKinect/libfreenect
Architecture
Quick view(*)
•Short for “Open Natural Interface”
•Multi-langue
•Cross-platform
•Standard & Infrastructure
使用openNI开发的程序在以后的版本中都可以继续使用,因为开发人员可以毫无顾虑的更新openNI到最新的版本。
OpenNI Framework(1)
•Top: 代表软件，在OpenNI顶部，实现了自然交互的应用程序。
•Middle: 代表OpenNI，提供通信接口，同时连接传感器和中间件组件，后者分析来自传感器的数据。
•Bottom:硬件设备，捕捉现场的视频和音频内容。
OpenNI Framework(2)
•一个USB抽象访问层
•一些基本数据类型的实现（包括列表，哈希，等等）
•系统日志和转储
•内存和性能分析
•活动（授权要注册的回调函数到一个特定的事件）
•任务调度
•不是标准OpenNI的一部分（*）
•函数指针
•注册
•事件响应
Generator
Metadata
Capability
Generator
注：绿色已经有，红色计划中，后略
•单位集
•生产性角色
•使用较低水平的生产节点
•“有意义的三位数据”
•Device
•Depth Generator
•Image Generator
•IR Generator
•Audio Generator
•Gesture Generator
•Scene Analyzer
•Hands Generator
•User Generator
•//provided by NITE
•Recoder
•Player
•Codec
Matadata
Capability
Session
Detector
•Full body analysis
•Hand point analysis
•Gesture detection
•Scene Analyzer
Session
Detector
衣带渐宽终不悔，为伊消得人憔悴。
•Preparation
•The program
•OpenNI+NITE
•Avin’s SensorKinect
•Processing(*)
•SimpleOpenNI(*)https://github.com/avin2/SensorKinect
Here’s the code,too young too simple
import SimpleOpenNI.*;SimpleOpenNI kinect;
void setup(){size(640*2, 480);kinect = new SimpleOpenNI(this);
kinect.enableDepth(); kinect.enableRGB();
}
void draw(){kinect.update();
image(kinect.depthImage(), 0, 0);image(kinect.rgbImage(), 640, 0);
}
Understand the code
import SimpleOpenNI.*;SimpleOpenNI kinect;
•Import the library
•Declare the SimpleOpenNI object
void setup(){size(640*2, 480);kinect = new SimpleOpenNI(this);
kinect.enableDepth(); kinect.enableRGB();
}
•Declare the size of app
•Instantiate the object
•Call two method
void draw(){kinect.update();
image(kinect.depthImage(), 0, 0);image(kinect.rgbImage(), 640, 0);
}
•Get fresh data from kinect
•Place the image
•R G B
•纯黑代表无穷远，纯白代表无穷近
•Demo:ex03_basic_depth_plus_rgb
•Projective
•Realworld
•Z值相同，代表深度，单位mm
•ConvertProjectiveToRealWorld()
The Kinect coordinate system
Demo
chPC_ex12_obj_in_point_cloud
•最近距离：about 0.5(?)
•最远距离： 10m
•GetDeviceMaxDepth()
•官方建议：1.2 – 3.6米
•近远精确性比较
Welcome to the Third Dimension
3d reconstruction
E.g:KinectFusion PCL
http://pointclouds.org/
DEMO
chPC_ex05_rotating_point_cloud
•User Detection
•Joints & Skeleton
•Distance in 3D
•Scene Map, Com
•Tracking
•Calibration or not
•Stages in Process
•Joints
•Distance between two joints in 3D
WindowsSDK: 20 joints(all)
OpenNI: 24 joints
DEMO
chSK_ex02_skel_anatomy
•Background remove
•User pixels
•Center of Mass
•chSK_scene_map
•chSK_scene_image
•chSK_center_of_mass
•kinect.enableScene();
•sceneMap = kinect.sceneMap();
•if(sceneMap[i] == n)
Demo:PointViewer
众里寻他千百度，蓦然回首，那人却在，灯火阑珊处。
Apps out of box
Create your own cool stuffs
Living coding with VS & Cpp
Demo: chDE_face_changing
•Collision detection
•kinect.getJointPositionSkeleton()
•kinect.convertRealWorldToProjective()
•Picture scala
Demo:Minority Report(*)
Demo:2D TrackPad
Demo:Hotpoint 3D
•Key event mapping
•Game playing
http://projects.ict.usc.edu/mxr/faast/
•DEMO: 宅男妄想地带
•Microsoft: NO!!
Sorry, Bill
•Map(Joint position + gesture, key event)
http://www.kinectdevs.com/forums/kinect-virtual-camera-kinect-webcam-f17
The following demo need a better computer for higher GPU
support
一秒钟凹凸曼合体！完成你童年时候的梦想
Video
http://code.google.com/p/kinect-ultra/
初音妹子
Video
http://www.kinect-hacks.com/openni
What do you want to make?
reference•http://viml.nchc.org.tw
•Visualization and Interactive Media Laboratory of NCHC
•国家科學視算與互動媒體實驗室
•http://www.javaforge.com/wiki/103532
•Kinect for PC Wiki
•中文
Do not delete this page