automatic 3d face recognition system

Automatic 3D Face Recognition System

695410042 邱彥霖491410044 龔士傑

Biometric Authentication

Outline

Introduction Nose Tip extraction Pose Correction Face Segmentation Recognition

Introduction

Nose Tip extraction

Reference

[1] X. Lu and A. K. Jain. "Multimodal facial feature extraction for automatic 3D face recognition," Technical Report MSU-CSE-05-22, Department of Computer Science, Michigan State University, East Lansing, Michigan, August 2005.

[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition," Proc. 7th IEEE International Conference on Automatic Face and Gesture Recognition (FG2006), pp. 585-590, Southampton, UK, Apr. 2006.

[3] M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreener for 3D face recognition using radial symmetry and the Hausdorff fraction, in: IEEE Workshop on Face Recognition Grand Challenge Experiments, June 2005.

[4] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition”, 3DPVT, 2006.

[1] X. Lu and A. K. Jain. "Multimodal facial feature extraction for automatic 3D face recognition.”

Face Segmentation

[1] X. Lu and A. K. Jain. "Multimodal facial feature extraction for automatic 3D face recognition.”

Nose Tip Extraction : Frontal scan

[1] X. Lu and A. K. Jain. "Multimodal facial feature extraction for automatic 3D face recognition.”

Nose Tip Extraction : Frontal scan

Nose Tip Extraction : Pose change

For a frontal facial scan, nose tip usually has the largest z value.

But, in the presence of large pose changes, this heuristic does not hold.

[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition ."

Nose Tip and Pose Estimation Pose quantization

The yaw angle change ranges from -90 degrees (full right profile) to 90 degrees (full left profile) in the X-Z plane.

[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition ."

Nose Tip and Pose Estimation Directional maximum

i

i

i

jj

jj

i

i

i

z

y

x

z

y

x

j

j

j

cos0sin

010

sin0cos

[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition ."

Nose Tip and Pose Estimation Pose correction

z

y

x

pz

py

px

z

y

x

cos0sin

010

sin0cos

'

'

'

[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition ."

Nose Tip and Pose Estimation Nose profile extraction

[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition ."

Nose Tip Extraction : Pose change

Nose Tip Extraction : Radial Symmetry Map Gradient and Zero-Crossing Map

[3] M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreener for 3D face recognition using radial symmetry and the Hausdorff fraction.

Radial Symmetry Map First, the gradient of the image, g is computed at ea

ch pixel p.

[3] M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreener for 3D face recognition using radial symmetry and the Hausdorff fraction.

Radial Symmetry Map

For each pair of affected pixels, the corresponding point P+ve in the orientation projection image is incremented by 1, respectively, while the point corresponding to P-ve is decremented by 1.

[3] M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreener for 3D face recognition using radial symmetry and the Hausdorff fraction.

Gradient and Zero-Crossing Map

The shape of the face is another effective indicator of key facial features.

[3] M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreener for 3D face recognition using radial symmetry and the Hausdorff fraction.

Nose Tip Extraction

Each 3D face is horizontally sliced at multiple steps dv. The nose tip is detected using a coarse to fine approach. Circles centered at horizontal intervals dh on the slice. The point which has the maximum altitude is considered

to be a potential nose tip and assigned a confidence value equal to the altitude.

[4] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition.”

Face Detection

A sphere of radius r (80 mm) centered at the nose tip is then used to crop the 3D face and its corresponding registered 2D face.

[4] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition.”

Pose Correction

Reference

[1] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition”, 3DPVT, 2006.

Pose Correction

Pose is corrected using the Hotelling transform.

To calculate the mean vector and covariance matrix.

The matrix of eigenvectors V of the covariance matrix C

[1] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition.”

Pose Correction

V is also a rotation matrix that aligns the point cloud P on its principal axes.

[1] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition.”

Face Normalization

[1] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition.”

Face Segmentation

Reference

[1] Ajmal S. Mian, M. Bennamoun and R. Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition", to appear in IEEE Transactions in Pattern Analysis and Machine Intelligence (IEEE TPAMI), 2007.

[2] K.-C. Wong, W.-Y. Lin, Y. H. Hu, N. Boston, and X. Zhang, " Optimal Linear Combination of Facial Regions for Improving Identification Performance", IEEE Trans. Systems, Man, and Cybernetics Part B: Cybernetics, Accepted, 2007.

Face Segmentation and Recognition

Robustness to facial expressions by automatically segmenting the face into expression sensitive and insensitive regions.

To measure the variance in the depth of the corresponding pixels (neutral and non-neutral expression).

[1] Ajmal S. Mian, M. Bennamoun and R. Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition."

Face Segmentation and Recognition

The features were automatically segmented by detecting the inflection points around the nose tip.

[1] Ajmal S. Mian, M. Bennamoun and R. Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition."

Results and Analysis

[1] Ajmal S. Mian, M. Bennamoun and R. Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition."

Multiple Region Face Recognition

[2] K.-C. Wong, W.-Y. Lin, Y. H. Hu, N. Boston, and X. Zhang, " Optimal Linear Combination of Facial Regions for Improving Identification Performance."

Similarity ScoreROC

[2] K.-C. Wong, W.-Y. Lin, Y. H. Hu, N. Boston, and X. Zhang, " Optimal Linear Combination of Facial Regions for Improving Identification Performance."

Results and Analysis

[2] K.-C. Wong, W.-Y. Lin, Y. H. Hu, N. Boston, and X. Zhang, " Optimal Linear Combination of Facial Regions for Improving Identification Performance."

Recognition

Reference

[1] M. Worring and A. W. M. Smeulders, " Digital curvature estimation ", CVGIP: Image Understanding, 58(3):366–382, 1993.

[2] N. Gelfand, N. J. Mitra, L. J. Guibas, and H. Pottmann, "Robust global registration", In Proc. Symp. Geom. Processing, pages 197–206, 2005.

[3] Robust Curvature Estimation Through Line Integrals.

Gaussian Convolution

1

'

( 1) ( )( ) tan ( )

( 1) ( )

( ) ( )

( )( , ) 100%, 1/

y i y ii

x i x i

k i i G

k k iB i r k r

k

[1] M. Worring and A. W. M. Smeulders, " Digital curvature estimation ."

Results and Analysis

[1] M. Worring and A. W. M. Smeulders, " Digital curvature estimation ."

Area Integrals

[2] N. Gelfand, N. J. Mitra, L. J. Guibas, and H. Pottmann, "Robust global registration".

Results and Analysis

[2] N. Gelfand, N. J. Mitra, L. J. Guibas, and H. Pottmann, "Robust global registration".

Line Integrals

[3] Robust Curvature Estimation Through Line Integrals.

Results and Analysis

[3] Robust Curvature Estimation Through Line Integrals.