tracking multiple occluding people by localizing on multiple scene planes professor ：王聖智...

Tracking Multiple Occluding People by

Localizing on Multiple Scene Planes

Professor ：王聖智教授Student ：周節

Outline

• Introduction

• Related work

• Localization Algorithm

• Tracking Algorithm

• Result

• Conclusion

• Reference

Introduction

Surveillance• Detection• Tracking

IntroductionIf the object is isolated…• Physical properties are useful.• Such as color, shape…• It is much simpler.

Introduction

?

But if the objects are not isolated…• Occlusion • Lack of visibility • In crowded and cluttered scenes

IntroductionDifficulty• It’s difficult to track

individual people

when occlusion occurred.

Outline

• Introduction

• Related work



• Result

• Conclusion

• Reference

Related work

• Monocular Approach

• Multi-camera Approach

Monocular Approach

• Handling occlusions (1) Color and shape

(2) Objects contours and appearances

(3) Blob split and merge analysis (4) ……

Monocular Approach

• Drawbacks (1) Can’t deal with full occlusion.

(2) Can’t deal with long periods.

• Therefore (1) Single view is limited.

(2) Multi-camera is preferred.

Multi-camera Approach

• Methods (1) Construct 3D model using voxel

(2) Calibrated camera to obtain 3D locations

(3) Switching camera

(4) Stereo camera

(5) ……

Multi-camera Approach

• Problems (1) Features might be corrupted by occlusions.

(2) Very similar color and shape.

(3) Full occlusion.

• Therefore This paper…

Outline

• Introduction

• Related work



• Result

• Conclusion

• Reference

Localization Algorithm

1. Obtain the foreground likelihood maps

2. Obtain reference plane homographies

3. Compute and find the localization


1. Obtain the foreground likelihood maps

– Model Background using a Mixture of Gaussians.– Perform Background Subtraction to obtain

foreground likelihood information.


2. Obtain reference plane homographies

– Homography Constraint

Homography Constraint

(R , T)

Homography Constraint


• x1~xn are the observations in images.• X is the event that the pixel is inside a foreground object.• L(xi) is the likelihood of observation xi belonging to the fo

reground.

Example

Localization AlgorithmFor robustness• Modeling Clutter and FOV Constraints

Example


• For robustness– Localization at Multiple Planes

Outline

• Introduction

• Related work



• Result

• Conclusion

• Reference

Tracking Algorithm

• Object trajectories are spatially and temporally coherent.

T=0 T=10 T=20 T=30

Tracking Algorithm• Spatially and temporally coherent

Tracking Algorithm

Tracking Algorithm

DEMO

Outline

• Introduction

• Related work



• Result

• Conclusion

• Reference

Result

• Case 1– Parking lot

Result

• Multi-camera improves performance.

Result

• Case 2– Indoor

Result

• Multi-plane improves performance.

Result

• Case 3– Basketball

• View limited – Error– Miss

Result

• Case 4– Soccer

• Background limited – Error– Miss

Outline

• Introduction

• Related work



• Result

• Conclusion

• Reference

Conclusion

To resolve occlusions and localize people:

• Planar homography constraints are used to fuse foreground likelihood information.

• Detection and tracking are performed simultaneously in the space-time occupancy likelihood data.

Conclusion

• Advantage– Very good performance when occlusions occur.

– No calibration is needed.

– Only 2D constructs , purely image-based .

Conclusion

• Disadvantage– If a person’s appearance is very similar to the

background or – If a person is occluded by some portion of the

background itself.– If a part of the scene is occluded in all views by the

foreground objects.

• Solutions– Add other models – like color model ,human motion model…

Reference

• Saad M. Khan and Mubarak Shah, “Tracking Multiple Occluding People by Localizing on Multiple Scene Planes”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume: 31, Issue: 3, March 2009

• S.M. Khan and M. Shah, “A Multi-View Approach to Tracking People in Crowded Scenes Using a Planar Homography Constraint,” Proc. Ninth European Conf. Computer Vision, 2006.

Property1. Do not use color models or shape cues of individual people2. Our method of detection and occlusion resolution is based on geometrical constructs

and requires only the distinction of foreground from back-ground, which is obtained using standard background modeling techniques.

3. the core of our method is a planar homographic occupancy constraint that combines foreground likelihood information from different views to resolve occlusions and determine regions on scene planes that are occupied by people.

4. consistently warp (under homographies of the reference plane) to foreground regions in every view.

5. The reason we use foreground likelihood maps instead of binary foreground maps is to delay the thresholding step to the last possible stage.

6. multiple planes parallel to the reference plane to robustly localize scene objects.7. To track,we obtain object scene occupancies for awindow of time and stack them tog

ether, creating a space-time volume. 8. designing an energy functional that combines scene occupancy information and spati

o-temporal proximity. 9. Homographies induced by the reference plane between views are computed using SI

FT feature matches and employing the RANSAC algorithm.10. The result is that our approach is purely image based and performs fusion in the imag

e plane without requiring to go in 3D space, and thus eliminating the need for fully calibrated cameras.

Monocular Approach

• Methods (1) Blob tracking

(2) Shape and Color

(3) Adaboost and Particle filter

(4) ……

tracking multiple occluding people by localizing on multiple scene planes professor ：王聖智...

Documents