a multiple camera with real-time volume reconstruction for articulated skeleton pose tracking

A Multiple Camera with Real-Time Volume Reconstruction for Articulated Skeleton Pose Tracking

指導教授：王聖智教授學生：謝佳峻

Zheng Zhang, Hock Soon Seah1 Chee Kwang Quah,Alex Ong, and Khalid JabbarK.-T. Lee et al. (Eds.): MMM 2011, Part I, LNCS 6523, pp. 182–192, 2011.Springer-Verlag Berlin Heidelberg 2011

OutlineIntroductionMulti-camera SystemVolume ReconstructionSkeleton Pose EstimationResultsConclusion

IntroductionMarkerless don’t need markers or special suits.Multi-view deal better with occlusion and appearance

ambiguity problems.

建立場景資訊剪出主要物件還原個體輪廓形狀偵測動作與行為

Multi-camera SystemSystem Setup 1.Cameras work synchronously for acquiring multiple

image in time. 2. The frame rate of image acquisition should be at

least 15 fps. 3. The bandwidth is sufficient for supporting the

transfer of multi-video streams. 4.The acquisition room ought to be large.

Only one PC !!

Volume ReconstructionBackground Subtraction

otherwise

vuBvuIifb nn

vun ,0),(),(,1

),(

nI

nB: 目前影像: 參考背景

: 為一門檻值

1. Background modeling constructs a reference image representing the background.

2. Threshold selection determines appropriate threshold values used in the subtraction operation.

3. Subtraction operation or pixel classication classies the type of a given pixel, i.e., the pixel is the part of background, or it is a moving object.

Volume ReconstructionShape-from-Silhouette and

Visual Hulls1.Each multi-view silhouette contour is firstly obtained.

2.Silhouette polygons are back-projected into their corresponding camera positions.3. Volume reconstruction method4.Testing each voxel’s 6-connected neighbors.

voxel texture

(a)

(b)

(c)

Illustration of volume reconstruction rendered in point clouds (a), voxels without texturing (b) and voxels with texturing (c)

Skeleton Pose EstimationThe body model1. Barrel model2. 10 body segments

(1)

(2)

29 DOFs

Skeleton Pose EstimationPSO(particle swarm optimization)

))(),0()(),0(( 211ikg

iki

ik

ik xpUxpUvXv

ik

ik

ik vxx 11

ikxikv

),0( iU ],0[ iipgp

is the position of the i-th particle at k-th iteration . is the velocity of the i-th particle at k-th iteration . represents a vector of random numbers uniformly

distributed in is the history best position found by the i-th particle. is the global best position found by its neighborhood so far. is a constriction coefficient .

X

Results

Conclusion1.Real-time volume sequences are

reconstructed for articulated pose recovery.2.Relies on single PC.3.Different body segments are not allowed to

intersect in the space .4.Different model points should avoid taking

the same closest feature point.Future work will concentrate on enhancing the tracking robustness and accurateness.

References Horprasert, T., Harwood, D., Davis, L.S.: A statistical

approach for real-time robust background subtraction and shadow detection.

Laurentini, A.: The visual hull concept for silhouette-based image understanding .

Matusik, W., Buehler, C., McMillan, L.: Polyhedral visual hulls for real-time rendering.

Clerc, M., Kennedy, J.: The particle swarm - explosion, stability, and convergence in a multidimensional complex space.

http://www.csie.ntu.edu.tw/~cyy/courses/vfx/05spring/lectures/scribe/12scribe.pdf

a multiple camera with real-time volume reconstruction for articulated skeleton pose tracking

Documents