3d tracking : chapter4 natural features, model-based tracking

Monocular Model-Based 3D Tracking of Rigid Ob-jects: A Survey

2008. 12. 15.백운혁

Chapter 4. Natural Features, Model-Based Tracking

Agenda

Monocular Model-Based 3D Track-ing of Rigid Objects : A Survey

Chapter 4. Natural Features, Model-Based Tracking 4.1. Edge-Based Methods 4.2. Optical Flow-Based Methods 4.3. Template Matching 4.4. Interest Point-Based Methods 4.5. Tracking Without 3D Models

4.1 Edge-Based Methodsstraight line segments and to fit the model outlines

4.1.1 RAPiD

4.1.1 RAPiD

Origin

Control point

Control point in camera coordinates

Motion

z

y

x

T

T

T

T

z

y

x

P

P

P

P

Rtp

Z

Y

X

PTM

4.1.1 RAPiD

RPtTM

PtMM IR

)( PtKmm

pWmm

z

y

x

z

y

x

t

t

tp

4.1.1 RAPiD

IRiii lpWn ~ )(~ mmnl T dis-

tance

2)~(minarg ii

iip

lpWnp

is vector made of the dis-tances

L il

pAL

ALAAp T 1)(

4.1.2 Making RAPiD Robust

Minimize the distance

Control points lying on the same object edge are grouped into primitives. And a whole primitive can be rejected from the pose es-timation.

RANSAC methodology

The number of edge strength maxima visible

ji

ijipp

mMPdistp,

)),((minarg

4.1.3 Explicit Edge Extraction

),,,( lccX yx The middle point, the orientation and the length of the segment

mX Of a model seg-ment

dX Of a an extracted seg-ment

)()()( 1dmdm

Tdm XXXXd

Is the covariance matrix

Mahalanobis distance

The pose is then estimated by minimizing

i

im

id

id

Tim

id pXXpXX ))(())((

p

4.2 Optical Flow-Based Meth-ods

dtv

umm

m Its corresponding location in the next image

m The projection of a point in an image at time

I t

4.2.1 Using Optical Flow Alone

Normal optical flow

For large motions Causes error accumulation

0)(,

t

Imm

v

I

u

I

4.2.2 Combining Optical Flow and Edges

To avoid error accumulation0 tIpB

B Depends of the pose and the image spatial gradi-ents at time

p t

tI Is a vector made of the temporal gradient at the cho-sen locations

4.3 Template MatchingTo register a 2D template to an image under a family of deformations

4.3.1 2D Tracking

j

jjt mTpmfIpO 2))());((()(

To find the parameters of some deformation

That warps a template into the input image

p f

T tI

j

jjt mTpmfIpO 2))());((()(

iAi

is the pseudo-inverse of the Jacobian matrix of computed at A J ));(( pmfI jt jP

ITT

i JJJ 1)(

4.4 Interest Point-Based MethodsUse localized features

Rely on matching individual features across images and are therefore easy to robustify against partial occlusions or matching errors

4.4.1 Interest Point Detection

Harris-Stephen detector / Shi-Tomasi detector

The pixels can be classified from the behavior of the eigen values of

2

2

vvu

vuu

III

IIIZ

The coefficients of are the sums over a window

of the first derivatives and of image intensi-ties

with respect to pixel coordinates

Z

uI vI

),( vu

Z

4.4.2 Interest Point Matching

to use 7x7 correlation windows reject matches for which measure is less than 0.8 search of correspondents for a maximum movement

of 50 pixels

Kanade-Lucas-Tomasi tracker

Keep the points that choose each other

2))());((( j

jiijf mTpmfI

4.4.3 Pose Estimation by Tracking Planes Pose Estimation for Planar Structures

010

121 w

tt

tt

tw HHHHH

)),(( 11 IdjydixI

Thanks for your attention