Object Segmentation Based on Multiple Features Fusion and Conditional Random Field

CASIA_IGIT

National Laboratory of Pattern Recognition(NLPR)Institute of Automation, Chinese Academy of Sciences(CASIA)

Reporter： Kun Ding（丁昆）2013.10.17

Outline• System Overview

• System Characteristics

•Results and Conclusions

Outline• System Overview

• System Characteristics

•Results and Conclusions

System Overview

•Object Segmentation Pipeline

FeatureEngineering

Superpixel Segmentation

Superpixels Features Probabilistic Output Final ResultsInput Image

Feature Extraction SVM Classification GrabCut

Stage 1 : Superpixel Classification

Stage 2 : Pixel-based CRF Smoothing

System Overview

• Superpixel Classification• Superpixel Segmentation • Graph-based image segmentation

• Feature Extraction: • To be detailed in next section

• SVM Classification[1]• RBF kernel with Probabilistic Output

System Overview

•Pixel-Based CRF Smoothing• Fusing several kinds of information as data term• Solving with GrabCut with only a few iterations

SVM Probabilistic Output CRF Smoothing Output

BinarizeFirst Iteration

SecondIteration

Outline

• System Overview

• System Characteristics

•Results and Conclusions

System Characteristics • Superpixel Segmentation -- Efficient Graph-Based Image Segmentation[2]• Fast, property of edge-preserving• Speeding up the whole procedure• Improving the separability between foreground and

background

Superpixels and their edge-preserving property

System Characteristics • Feature Engineering – Superpixel-Based Multiple Features Fusion

Gradient

Texture

Color and skin

Geometrical

Saliency

Results of Object Detection

PCA

Dense SIFT[3][4] dictionary with Bag-of-Words description

Multi-scale LBP histogram

RGB histogram and HS histogram with skin detection

Position, direction and roundness

Color spatial distribution, multi-scale local and global contrast

Probability derived from AdaBoost, with manifold ranking[6] refinement

System Characteristics • Feature Engineering – Superpixel-Based Multiple Features Fusion• Illustration of object detection

Object Detection result Rectangle Density as Probability Refined with Manifold Ranking

System Characteristics •Pixel-Based CRF Smoothing – GrabCut[7]•Modified data term • Solving by maxflow iteratively

SVM Result Object Detection Result

GMM Result for Foreground and Background

CRF Smoothing Output

Outline• System Overview

• System Characteristics

•Results and Conclusions

Conclusion and Results Exhibition•Results Exhibition

Conclusion and Results Exhibition•Conclusion• Superpixel classification• Feature fusion works• CRF smoothing improves the results of SVM

• Object parts sometimes lost• Context information is inadequate

[1] C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http: //www.csie.ntu.edu.tw/˜cjlin/libsvm.[2] Felzenszwalb P F, Huttenlocher D P. Efficient graph-based image segmentation[J]. International Journal of Computer Vision, 2004, 59(2): 167-181.[3] Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International journal of computer vision, 2004, 60(2): 91-110.[4] Vedaldi A, Fulkerson B. VLFeat: An open and portable library of computer vision algorithms[C]//Proceedings of the international conference on Multimedia. ACM, 2010: 1469-1472.

Selected References

Selected References[5] Liu T, Yuan Z, Sun J, et al. Learning to detect a salient object[J]. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2011, 33(2): 353-367.[6] Chuan Yang, Lihe Zhang, Huchuan Lu, Minghsuan Yang, Saliency Detection via Graph-Based Manifold Ranking, CVPR2013, P3166-3173[7] Rother C, Kolmogorov V, Blake A. Grabcut: Interactive foreground extraction using iterated graph cuts[C]//ACM Transactions on Graphics (TOG). ACM, 2004, 23(3): 309-314.

Thank you very much!Any questions?

CASIA_IGIT

Leader: Ying Wang （王颖）Members: Kun Ding （丁昆）

Huxiang Gu （谷鹄翔）Yongchao Gong （宫永超）

E-mails: {ywang, kding, hxgu, yongchao.gong}@nlpr.ia.ac.cn