Facial Recognition and

3D Face Modeling

by Tanasai Sucontphunt

interactive media science @ nida http://interactivemedia.nida.ac.th/

How a computer recognizes a human face?

How a computer sees a human face?

How a computer detects a human face?

How a computer identifies a human identity?

http://chenlab.ece.cornell.edu/

How a computer sees a human face?

3D Surface?

http://gl.ict.usc.edu/Research/FaceScanning/images/EGSR2007_SGI_faces.png

Temperature?

http://matlab-recognition-code.com/infrared-face-recognition-system/

Image?

Input Choices

Most Favorite!

How a computer detects a human face?API choices

Lambda Labs Face Recognition Animetrics Face Recognition

Skybiometry Face Detection and Recognition ImageVision Face Detection

Face++ FaceMark

EmoVu by Eyeris Face and scene recognition by Rekognition.com

FaceRect Infatics Face Detection

OpenCV Face Recognizer Libface

Automatic naming of characters in video CCV

OpenBR Flandmark

Computer Expression Recognition Toolbox (CERT) Nviso 3D facial imaging technology

FaceReader Affdex

Faceware Face detection in iOS Face-Recognition-SDK

FaceSDK FaceIT SDK

Stupeflix Face.com

Microsoft Research Face SDK etc.

http://cdn0.dailydot.com/cache/fc/3a/fc3aa93d4cf68ff5406f291efcd81b08.jpg

Detection + Landmark Extractionhttp://www.wired.com/2012/11/auto-facial-recognition/

Face TrackingPost Estimation

Facial Feature TrackingEmotion Recognition

How a computer identifies a human identity?

Usage Choices

Feature-Based Holistic

for image = all the pixelshttp://www.ait.gr/ait_web_site/faculty/apne/Images/FacePreproc.jpg

e.g. facial landmarkshttps://raw.githubusercontent.com/ci2cv/face-analysis-sdk/stable/doc/avatar-annotation.png

Both?

How a computer identifies a human identity?

Algorithm or Classifier choices

EigenFace (PCA) FisherFace (FisherLDA) Support Vector Machine

Neural Network Deep Neural Network

Convolution Neural Network etc.

http://chenlab.ece.cornell.edu/projects/KinshipVerification/images/kinship.png

EigenFace (Classic)

Face recognition using eigenfaces, Matthew Turk and Alex Pentland in CVPR 1991

http://mikedusenberry.com/on-eigenfaces/Face Dataset EigenVectors

Principle Component

Analysis (PCA)

EigenFace (Classic)

Face recognition using eigenfaces, Matthew Turk and Alex Pentland in CVPR 1991http://mikedusenberry.com/on-eigenfaces/

EigenVectors

A Face

Projection

0.4 0.7 0.5

Score (PCA Coefficient)

0.1 0.6 0.3

0.40.5 0.7Same Face = Same Score Similar Face = Similar Score

EigenFace (Classic)

https://www.youtube.com/watch?v=N5GoaNPWXFw

Analogy

Famous Issues

• Lighting

• Pose

• Facial Expression

• Deep Learning requires a huge training data set to reach an acceptable accuracy (~98%)

http://www.nap.edu/read/11577/chapter/3

http://www.christian-behrenberg.de/img/portfolio/bscthesis

3D Face Modeling

How can we model a 3D Face?• Crafting manually by artist

• 3D Scanner

• Construct from a photograph?

by Stanislav Klabík

ICT, USC

3D Face Scanner

Light Stage XStructured Light

From Photograph

A Morphable Model for the Synthesis of 3D Faces [Blanz and Vetter 99] (Classic)

Input Image

w1 * w2 * w3 * w4 * w5 *+ + + +

=Synthesized Image (Rendered)

Update Scores

Calculate differences

Score (PCA Coef.)

3D Morphable Model

3D Morphable Applications

Total Moving Face Reconstruction [S, Suwajanakorn, I. Kemelmacher-Shlizerman, S. M. Seitz, 2014]

3D Morphable Applications

More to come…• Your identity in characters in Games

• Cartoon face from photograph

• Celebrity face similarity

• โหวงเฮ้ง Games

• 3D Face Scanner with a single camera

• etc.