the not so short

Hang xie

sonicisdreaming@gmail.com

http://Xhbang.com

Chongqing University bachelor（单身汉）

A boy couldn't stop dreaming

Bike travelling

•Dr.

•Sensor related

•Poet（魔幻现实主义）

•50% Idle

•25% Computer

•20% Programmer

•5% Student/Paper

Programming dummy?

Linux || Windows?

Understand 3D?

Work with the Kinect?

体感操作，Kinect，OPENNI介绍

基本程序设计

Working with Depth Image

Working with Point Clouds（*）

Working with Skeleton Data & Tracking

Kinect Hacks

昨夜西风凋碧树,独上高楼,望尽天涯路

Buy it!

Teardown

Principle

Hardware

Performace

Hardware & teardown

http://www.ifixit.com/Teardown/Microsoft-Kinect-

Teardown

•RGB camera

•IR depth sensor

•IR projector

•Audio device

Noise at edges(shoulders)

Depth shadows

Misalignment between the color and depth images

Frame rate(30 vs 60)

Behind Kinect

•Time of Flight

•以色列公司3DV（ 09年微软收购）

•光飞行时间，飞秒(1e−15)级快门

•砷化镓，使其时钟频率提高到上百

•高成本，难以适应消费电子品

•Natal并不是基于ToF的原理

•连续的照明（而非脉冲）

•普通的CMOS感光芯片（低成本）

•结构光技术（三维纵深的“体编码” ）

•激光散斑（laser speckle）

http://www.javaforge.com/wiki/103639

http://www.wipo.int/pctdb/en/wo.jsp?WO=2007043036

http://www.freepatentsonline.com/7433024.pdf

•机器学习系统

•TB计的数据训练集

1. 寻找移动部位（背景分离，分割遮罩）

2. 辨认身体部位可能性

3. 模型匹配：生成骨架系统

•Something we forget

•Supported by OpenNI, not by SensorKinect

•“Sound Wave”

•微软研究院和华盛顿大学

•利用多普勒效应，使用笔记本内置的扬声器和麦克风开发出类似Kinect的姿势感知系统

•低成本

•识别精度能达到90%以上

•用手势玩俄罗斯方块游戏（视频）

Concepts

http://en.wikipedia.org/wiki/Natural_user_interface

Next generation interface

Wii Remote

ASUS Xtion

Kinect

Project Glass

Development

http://www.openni.org/

http://code.google.com/p/simple-openni/

http://openkinect.org/wiki/Main_Page

https://github.com/OpenKinect/libfreenect

Architecture

Quick view(*)

•Short for “Open Natural Interface”

•Multi-langue

•Cross-platform

•Standard & Infrastructure

使用openNI开发的程序在以后的版本中都可以继续使用,因为开发人员可以毫无顾虑的更新openNI到最新的版本。

OpenNI Framework(1)

•Top: 代表软件，在OpenNI顶部，实现了自然交互的应用程序。

•Middle: 代表OpenNI，提供通信接口，同时连接传感器和中间件组件，后者分析来自传感器的数据。

•Bottom:硬件设备，捕捉现场的视频和音频内容。

OpenNI Framework(2)

•一个USB抽象访问层

•一些基本数据类型的实现（包括列表，哈希，等等）

•系统日志和转储

•内存和性能分析

•活动（授权要注册的回调函数到一个特定的事件）

•任务调度

•不是标准OpenNI的一部分（*）

•函数指针

•注册

•事件响应

Generator

Metadata

Capability

Generator

注：绿色已经有，红色计划中，后略

•单位集

•生产性角色

•使用较低水平的生产节点

•“有意义的三位数据”

•Device

•Depth Generator

•Image Generator

•IR Generator

•Audio Generator

•Gesture Generator

•Scene Analyzer

•Hands Generator

•User Generator

•//provided by NITE

•Recoder

•Player

•Codec

Matadata

Capability

Session

Detector

•Full body analysis

•Hand point analysis

•Gesture detection

•Scene Analyzer

Session

Detector

衣带渐宽终不悔，为伊消得人憔悴。

•Preparation

•The program

•OpenNI+NITE

•Avin’s SensorKinect

•Processing(*)

•SimpleOpenNI(*)https://github.com/avin2/SensorKinect

Here’s the code,too young too simple

import SimpleOpenNI.*;SimpleOpenNI kinect;

void setup(){size(640*2, 480);kinect = new SimpleOpenNI(this);

kinect.enableDepth(); kinect.enableRGB();

}

void draw(){kinect.update();

image(kinect.depthImage(), 0, 0);image(kinect.rgbImage(), 640, 0);

}

Understand the code

import SimpleOpenNI.*;SimpleOpenNI kinect;

•Import the library

•Declare the SimpleOpenNI object

void setup(){size(640*2, 480);kinect = new SimpleOpenNI(this);

kinect.enableDepth(); kinect.enableRGB();

}

•Declare the size of app

•Instantiate the object

•Call two method

void draw(){kinect.update();

image(kinect.depthImage(), 0, 0);image(kinect.rgbImage(), 640, 0);

}

•Get fresh data from kinect

•Place the image

•R G B

•纯黑代表无穷远，纯白代表无穷近

•Demo:ex03_basic_depth_plus_rgb

•Projective

•Realworld

•Z值相同，代表深度，单位mm

•ConvertProjectiveToRealWorld()

The Kinect coordinate system

Demo

chPC_ex12_obj_in_point_cloud

•最近距离：about 0.5(?)

•最远距离： 10m

•GetDeviceMaxDepth()

•官方建议：1.2 – 3.6米

•近远精确性比较

Welcome to the Third Dimension

3d reconstruction

E.g:KinectFusion PCL

http://pointclouds.org/

DEMO

chPC_ex05_rotating_point_cloud

•User Detection

•Joints & Skeleton

•Distance in 3D

•Scene Map, Com

•Tracking

•Calibration or not

•Stages in Process

•Joints

•Distance between two joints in 3D

WindowsSDK: 20 joints(all)

OpenNI: 24 joints

DEMO

chSK_ex02_skel_anatomy

•Background remove

•User pixels

•Center of Mass

•chSK_scene_map

•chSK_scene_image

•chSK_center_of_mass

•kinect.enableScene();

•sceneMap = kinect.sceneMap();

•if(sceneMap[i] == n)

Demo:PointViewer

众里寻他千百度，蓦然回首，那人却在，灯火阑珊处。

Apps out of box

Create your own cool stuffs

Living coding with VS & Cpp

Demo: chDE_face_changing

•Collision detection

•kinect.getJointPositionSkeleton()

•kinect.convertRealWorldToProjective()

•Picture scala

Demo:Minority Report(*)

Demo:2D TrackPad

Demo:Hotpoint 3D

•Key event mapping

•Game playing

http://projects.ict.usc.edu/mxr/faast/

•DEMO: 宅男妄想地带

•Microsoft: NO!!

Sorry, Bill

•Map(Joint position + gesture, key event)

http://www.kinectdevs.com/forums/kinect-virtual-camera-kinect-webcam-f17

The following demo need a better computer for higher GPU

support

一秒钟凹凸曼合体！完成你童年时候的梦想

Video

http://code.google.com/p/kinect-ultra/

初音妹子

Video

http://www.kinect-hacks.com/openni

What do you want to make?

reference•http://viml.nchc.org.tw

•Visualization and Interactive Media Laboratory of NCHC

•国家科學視算與互動媒體實驗室

•http://www.javaforge.com/wiki/103532

•Kinect for PC Wiki

•中文

Hang xie

sonicisdreaming@gmail.com

http://www.xhbang.com

Do not delete this page