using dimensionality reduction to improve local regression on facial age estimation...

Using Dimensionality Reduction to Improve Local Regression on Facial Age Estimation國立台灣大學 資訊工程學系R02922098 簡嘉宏指導教授：張智星 博士

OutlineIntroduction

Related work

Proposed method

Performance evaluation

Conclusions and future work

2/28

OutlineIntroduction

MotivationObjectiveChallengesOur framework

Related work

Proposed method

Performance evaluation

Conclusion and future work

3/28

IntroductionMotivation

Application for face recognition topics, ex: surveillance system, advertising recommendation system.

Programing contest, ex: UTMVP

ObjectiveUse dimensionality reduction algorithm to improve local regression

performance on facial age estimation system

ChallengesOrdinal relationshipDataset imbalance

4/28

32 26 62

IntroductionOur framework

5/28

Input image

Feature extraction

Age estimation model

Predicted ageDistance metric

learning

Manifold learning

Dimensionality reduction

Feature vector:

Feature vector:

Determined age:

Input img :

Chao, Wei-Lun, Jun-Zuo Liu, and Jian-Jiun Ding. "Facial age estimation based on label-sensitive learning and age-oriented regression." Pattern Recognition46.3 (2013): 628-641.

Introduction

Related workFeature extractionManifold learningDistance metric learningAge determination model

Proposed method

Performance evaluation

Conclusion and future work

Outline

6/28

Related workTopic Method

Feature extraction Local binary pattern (LBP)Gabor filterASM/AAM

Manifold learning Local sensitive discriminate analysis (LSDA)Maximum margin projection (MMP)

Distance metric learning Relevant component analysis (RCA)Discriminative component analysis (DCA)

Age estimation model KNN-SVRSVM-BDTOrdinal hyperplanes Ranker (OHRanker)

7/28

OutlineIntroduction

Related work

Proposed methodActive appearance modelMaximum margin projectionLabel-sensitive DCAkNN-SVR

Performance evaluation

Conclusion and future work

8/28

Active appearance modelIntroduction

Capable of extracting the shape and the texture of an image

Data and landmarkWe labeled 54 imgs with 68 feature points,

9/28

Active appearance modelBuilding AAM concept

Data representationData normalizationStatically analysis by PCA

Model• Shape model: • Texture model:

Combine: Appearance model:

10/28

Maximum margin projectionIntroduction

Manifold learningSemi-supervised dimensionality reductionObjective

◦ Nearby points with same label are close to each other◦ Nearby points with different labels are far apart

Discovering the local manifold structure by Nearest Neighbor Graph

11/28

Maximum margin projectionMMP algorithm

12/28

Presetting• Training set: , n=1:N• Find a projection matrix , then Algorithm• Construct neighbor relationship• Construct within-class weight and

between-class weight matrix• Optimize: • Solve the problem by Laplacian

Eigenmaps

Laplacian eigenmaps

Discriminative component analysisIntroduction

Distance metric learningObjective:

◦ Find a good distance metric which can be used for similarity measureOvercome the disadvantage of relevant component analysis (RCA) Supervised dimensionality reductionLearn an optimal transformation matrix by feature variance analysis

13/28

Relevant component analysis

Original total variance matrix of DCA

Optimize

14/28

Discriminative component analysis

Our total variance matrix of label-sensitive DCADefine class-to-class weight , sample-to-

class

weighted mean vector

𝐸𝑏 𝐸𝑤

Discriminative component analysis

15/28

𝑋 𝑖=[𝑎11 ⋯ 𝑎𝑑

1

⋮ ⋱ ⋮𝑎1𝑛 ⋯ 𝑎𝑑

𝑛]𝑚𝑖=[𝑚1

𝑖 ,…,𝑚𝑑𝑖 ]

…

Age label i

Age label 1

Age label k

𝑚1=[𝑚11 ,…,𝑚𝑑

1 ]

𝑚𝑘=[𝑚1𝑘 ,…,𝑚𝑑

𝑘]

𝐶𝑏

𝐶𝑤 ,𝑖

𝐶𝑤 ,𝑘

𝐶𝑤 ,1

𝐶𝑤

Label sensitive - DCA

16/28

𝑚𝑖=[𝑚1𝑖 ,…,𝑚𝑑

𝑖 ]

𝑚1=[𝑚11 ,…,𝑚𝑑

1 ]

𝑚𝑘=[𝑚1𝑘 ,…,𝑚𝑑

𝑘]

𝑚𝑖′

𝑚1′

𝑚𝑖′

𝑚𝑘′

�̂�𝑏

�̂�𝑏 , 1

�̂�𝑏 , 𝑖

�̂�𝑏 ,𝑘

𝑚1′

𝑚𝑖′

𝑚𝑘′

𝑋 𝑖

𝑋𝑘

𝑋 1 �̂�𝑤 ,1

�̂�𝑤 ,𝑖

�̂�𝑤 ,𝑘

�̂�𝑤

𝐽 ( 𝐴 )=argmax𝐴

|𝐴𝑇 �̂�𝑏𝐴||𝐴𝑇 �̂�𝑤 𝐴|

kNN-SVRBecause of the advantage about MMP and ls-DCA, we use kNN to do local regression

Algorithm

17/28

Input query

kNN

…

…

Regression

OutlineIntroduction

Related work

Proposed method

Performance evaluationMORPH datasetDataset imbalanceSettingExperimentError analysis

Conclusion and future work

18/28

MORPH dataset

19/28

<20 20-29 30-39 40-49 50+0

2000

4000

6000

8000

10000

12000

14000

16000

18000

7469

1632515357

12050

3933

831

23092910

1988

451

6638

14016

12447

10062

3482

Total Female Male

Dataset imbalanceTo deal with this challenge, we modify neighbor size when doing k-nearest neighbor

This method called Neighbor Size Modification[1] which gives a huge weight to insufficient range

20/28[1] Chao, Wei-Lun, Jun-Zuo Liu, and Jian-Jiun Ding. "Facial age estimation based on label-sensitive learning and age-oriented regression." Pattern Recognition46.3 (2013): 628-641.

Age range Image number21~25 226~30 14931~35 59536~40 73841~45 60146~50 37351~55 18256~60 6461~65 2566~70 471~75 376~80 2total 2738

21~25 26~30 31~35 36~40 41~45 46~50 51~55 56~60 61~65 66~70 71~75 76~800

100

200

300

400

500

600

700

800

2

149

595

738

601

373

182

6425 4 3 2

SettingControl Variables

AAM feature: 5-fold cross validation

Independent VariablesProposed methodChao’s methodOHRankerSVM-BDT

Performance evaluationMean Absolute Error (MAE)

21/28

ExperimentMethod MAE

AAM＋ kNN-SVR 6.3374

AAM＋ MMP＋ kNN-SVR 5.8872

AAM＋ DCA＋ kNN-SVR 5.9929

AAM＋ MMP＋ DCA＋ kNN-SVR 5.9228

AAM＋ DCA＋ MMP＋ kNN-SVR 5.8279

22/28

Method MAE

AAM＋ *kNN-SVR 5.9172

AAM＋ *MMP＋ *kNN-SVR 5.7279

AAM＋ ls-DCA＋ *kNN-SVR 5.6931

AAM＋ *MMP＋ ls-DCA＋ *kNN-

SVR5.6242

AAM＋ ls-DCA＋ *MMP＋ *kNN-

SVR5.7987

Notation: Neighbor size modification (*) , Label sensitive (ls-)

AAM ＋ kNN-SVR

AAM ＋MMP ＋kNN-SVR

AAM ＋ DCA＋ kNN-SVR

AAM ＋MMP ＋ DCA＋ kNN-SVR

AAM ＋ DCA＋ MMP ＋

kNN-SVR

5.2

5.4

5.6

5.8

6

6.2

6.4 6.3374

5.88725.9929 5.9228

5.82795.9172

5.7279 5.6931 5.62425.7987

without ls- and NSM with ls- and NSM

Method MAE

AAM＋ OHRanker

(without cost-sensitive)6.4644 (4.48)

AAM＋ SVM-BDT

(33/41/51)6.7607 (4.2)

Chao’s method:

AAM＋ ls-RCA＋ ls-MFA＋ kNN-SVR5.8150 (4.44)

Error analysis - 1

23/28

The limit of neighbor size modification and its trade off.

21~25 26~30 31~35 36~40 41~45 46~50 51~55 56~60 61~65 66~70 71~75 76~800

100200300400500600700800

2

149

595

738

601

373

18264 25 4 3 2

61

kNN

Error analysis - 2The image which has beard and the neighbors almost have beard.

24/28

kNN

Error analysis – 3,4Face shape and expression of one’s eyes maybe influence the neighbor search.

Mouth maybe be the clue of judgement.

25/28

kNN

kNN

OutlineIntroduction

Related work

Proposed method

Performance evaluation

Conclusion and future work

26/28

Conclusions and future workConclusions

The local regression accuracy is actually promoted by our method.The images which was found by Neighbor Search look like input query ,but

we think it will more appropriate for facial expression recognition.

Future workNeed to make the best use of the dataset, while 2738/55134 is not a

effective usage.Need more experience on how to build a proper AAM model for different

applications or feature combination.Deal with the dataset imbalance problem, we also can do some modification

on SVR.Build this system on mobile device.

27/28

Thanks for listening!

28/28

Related workFeature extraction

LBP (Local binary pattern)◦ Ojala, Timo, Matti Pietikainen, and David Harwood. "Performance evaluation of texture

measures with classification based on Kullback discrimination of distributions." Pattern Recognition, 1994. Vol.

Gabor filter◦ Lee, Tai Sing. "Image representation using 2D Gabor wavelets." Pattern Analysis and

Machine Intelligence, IEEE Transactions on 18.10 (1996): 959-971.ASM/AAM ( Active shape/appearance model )

◦ Cootes, Timothy F., et al. "Active shape models-their training and application."Computer vision and image understanding 61.1 (1995): 38-59.

◦ Cootes, Timothy F., Gareth J. Edwards, and Christopher J. Taylor. "Active appearance models." Computer Vision—ECCV’98. Springer Berlin Heidelberg, 1998. 484-498.

BIF ( Bio-inspired feature )◦ Guo, Guodong, et al. "Human age estimation using bio-inspired features."Computer Vision

and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009.

29/28

Related workManifold learning

LSDA (Locality sensitive discriminant analysis)◦ Cai, Deng, et al. "Locality Sensitive Discriminant Analysis." IJCAI. 2007.

MMP (Maximum margin projection)◦ He, Xiaofei, Deng Cai, and Jiawei Han. "Learning a maximum margin subspace for image

retrieval." Knowledge and Data Engineering, IEEE Transactions on20.2 (2008): 189-201.

Distance metric learningRCA (Relevant component analysis)

◦ Shental, Noam, et al. "Adjustment learning and relevant component analysis."Computer Vision—ECCV 2002. Springer Berlin Heidelberg, 2002. 776-790.

DCA (Discriminative component analysis)◦ Hoi, Steven CH, et al. "Learning distance metrics with contextual constraints for image

retrieval." Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on. Vol. 2. IEEE, 2006.

30/28

Related workAge determination model

kNN-SVR◦ Chao, Wei-Lun, Jun-Zuo Liu, and Jian-Jiun Ding. "Facial age estimation based on label-

sensitive learning and age-oriented regression." Pattern Recognition46.3 (2013): 628-641.

SVM-BDT◦ Han, Hu, Charles Otto, and Anil K. Jain. "Age estimation from face images: Human vs.

machine performance." Biometrics (ICB), 2013 International Conference on. IEEE, 2013.Label distribution learning

◦ Geng, Xin, Chao Yin, and Zhi-Hua Zhou. "Facial age estimation by learning from label distributions." Pattern Analysis and Machine Intelligence, IEEE Transactions on 35.10 (2013): 2401-2412.

Ordinal hyperplanes Ranker◦ Chang, Kuang-Yu, Chu-Song Chen, and Yi-Ping Hung. "Ordinal hyperplanes ranker with

cost sensitivities for age estimation." Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 2011.

31/28

Chao 127

OHRanker

32/28