A Multiple Camera with Real-Time Volume Reconstruction for Articulated Skeleton Pose Tracking
指導教授:王聖智 教授學生:謝佳峻
Zheng Zhang, Hock Soon Seah1 Chee Kwang Quah,Alex Ong, and Khalid JabbarK.-T. Lee et al. (Eds.): MMM 2011, Part I, LNCS 6523, pp. 182–192, 2011.Springer-Verlag Berlin Heidelberg 2011
OutlineIntroductionMulti-camera SystemVolume ReconstructionSkeleton Pose EstimationResultsConclusion
OutlineIntroductionMulti-camera SystemVolume ReconstructionSkeleton Pose EstimationResultsConclusion
IntroductionMarkerless don’t need markers or special suits.
Multi-view deal better with occlusion and appearance
ambiguity problems.
建立場景資訊
剪出主要物件
還原個體輪廓形狀
偵測動作與行為
OutlineIntroductionMulti-camera SystemVolume ReconstructionSkeleton Pose EstimationResultsConclusion
Multi-camera SystemSystem Setup 1.Cameras work synchronously for acquiring multiple
image in time. 2. The frame rate of image acquisition should be at
least 15 fps. 3. The bandwidth is sufficient for supporting the
transfer of multi-video streams. 4.The acquisition room ought to be large.
Only one PC !!
OutlineIntroductionMulti-camera SystemVolume ReconstructionSkeleton Pose EstimationResultsConclusion
Volume ReconstructionBackground Subtraction
otherwise
vuBvuIifb nn
vun,0
),(),(,1),(
nI
nB
: 目前影像: 參考背景
: 為一門檻值
1. Background modeling constructs a reference image representing the background.
2. Threshold selection determines appropriate threshold values used in the subtraction operation.
3. Subtraction operation or pixel classication classies the type of a given pixel, i.e., the pixel is the part of background, or it is a moving object.
Volume ReconstructionShape-from-Silhouette and
Visual Hulls1.Each multi-view silhouette contour is firstly obtained.
2.Silhouette polygons are back-projected into their corresponding camera positions.
3. Volume reconstruction method4.Testing each voxel’s 6-connected neighbors.
voxel texture
(a)
(b)
(c)
Illustration of volume reconstruction rendered in point clouds (a), voxels without texturing (b) and voxels with texturing (c)
OutlineIntroductionMulti-camera SystemVolume ReconstructionSkeleton Pose EstimationResultsConclusion
Skeleton Pose EstimationThe body model1. Barrel model2. 10 body segments
(1)
(2)
29 DOFs
Skeleton Pose EstimationPSO(particle swarm optimization)
))(),0()(),0(( 211ikg
iki
ik
ik xpUxpUvXv
ik
ik
ik vxx 11
ikxikv
),0( iU ],0[ iipgp
is the position of the i-th particle at k-th iteration . is the velocity of the i-th particle at k-th iteration .
represents a vector of random numbers uniformly distributed in
is the history best position found by the i-th particle. is the global best position found by its neighborhood so far. is a constriction coefficient .
X
OutlineIntroductionMulti-camera SystemVolume ReconstructionSkeleton Pose EstimationResultsConclusion
Results
Results
OutlineIntroductionMulti-camera SystemVolume ReconstructionSkeleton Pose EstimationResultsConclusion
Conclusion1.Real-time volume sequences are
reconstructed for articulated pose recovery.2.Relies on single PC.3.Different body segments are not allowed to
intersect in the space .4.Different model points should avoid taking
the same closest feature point.
Future work will concentrate on enhancing the tracking robustness and accurateness.
References Horprasert, T., Harwood, D., Davis, L.S.: A statistical
approach for real-time robust background subtraction and shadow detection.
Laurentini, A.: The visual hull concept for silhouette-based image understanding .
Matusik, W., Buehler, C., McMillan, L.: Polyhedral visual hulls for real-time rendering.
Clerc, M., Kennedy, J.: The particle swarm - explosion, stability, and convergence in a multidimensional complex space.
http://www.csie.ntu.edu.tw/~cyy/courses/vfx/05spring/lectures/scribe/12scribe.pdf