Real Time Appearance Based Hand Tracking

The 19th International Conference on Pattern Recognition (ICPR)December 7-11, 2008, Tampa Convention Center, Tampa, FL, USA

報告者：彭成瑋日期： 2009/12/29指導教授：陳立祥 教授實驗室：網際網路多媒體應用實驗室

Outline

Introduction Tracking method Experiments Conclusion Q&A

Introduction

Hand tracking is an important problem in the field of human-computer interaction.

Application ：sign language recognition or controlling computer games.

Model-based(3D model) and Appearance-based (Image features)

Introduction （ Cont. ） the hand presents a motion of 27 degrees of fre

edom (DOF), 21 for the joint angles and 6 for orientation and location[11, 10].

Substantial problems ：out-of-plane rotations scale changes, self-occlusions or segmentation accuracy.

Real-time tracking performance Maximally Stable Extremal

Region (MSER) tracking algorithm.

Tracking method

Novel tracking method Multivariate Gaussians with the Kullback-

Leibler distance

Color likelihood

calculate a probability value p(O|xi) for every pixel in the current frame

object-to-be-tracked (hand) O Kullback-Leibler distance instead of the

Bhattacharyya distance The integral image for Bhattacharyya dist

ance calculation

Color likelihood （ Cont. ） Mahalanobis Distance

Bhattacharyya Distance

222

111

;

;

N

N

21

1

212121 2

,

TMAd

2

1

22

1

1

21

21

1

212121

2

1

ln2

1

28

1,

TBAd

Color likelihood （ Cont. ） color likelihood value -- p(O|xi) every pixel – xi r × c window color distribution of the hand O in the frame t−1 -- G

aussian 3×1 mean vector – μO 3×3 covariance matrix -- Gaussian multivariate Gaussian --

OOON ,

BBBN

iwN

B

iwO

iw

Oiw

NNNN

NNxiOp

,,

,exp|

Maximally Stable Extremal Region (MSER) tracking

(a) Input Image (b) Image histogram (c) MSER result

Modified MSER tracking

(a) Color likelihood (b) MSER detection result

Experiments

25 frames per second on a 320 × 240 video sequences

Experiments （ Cont. ） A simple gesture recognition allows to us

e the tracker for controlling the mouse pointer and activating mouse-clicks.

Conclusion

Novel real time method for tracking hands through image sequences

Efficiently calculated color similarity maps

Q&A

Q ：為什麼選擇使用 Appearance-based 來實作 .

A ：為了符合即時運算之效能考量，因為 Model-based 使用 3D model 來辨識，需花費較多運算量。