using dimensionality reduction to improve local regression on facial age estimation...
TRANSCRIPT
Using Dimensionality Reduction to Improve Local Regression on Facial Age Estimation國立台灣大學 資訊工程學系R02922098 簡嘉宏指導教授：張智星 博士
OutlineIntroduction
Related work
Proposed method
Performance evaluation
Conclusions and future work
2/28
OutlineIntroduction
MotivationObjectiveChallengesOur framework
Related work
Proposed method
Performance evaluation
Conclusion and future work
3/28
IntroductionMotivation
Application for face recognition topics, ex: surveillance system, advertising recommendation system.
Programing contest, ex: UTMVP
ObjectiveUse dimensionality reduction algorithm to improve local regression
performance on facial age estimation system
ChallengesOrdinal relationshipDataset imbalance
4/28
32 26 62
IntroductionOur framework
5/28
Input image
Feature extraction
Age estimation model
Predicted ageDistance metric
learning
Manifold learning
Dimensionality reduction
Feature vector:
Feature vector:
Determined age:
Input img :
Chao, Wei-Lun, Jun-Zuo Liu, and Jian-Jiun Ding. "Facial age estimation based on label-sensitive learning and age-oriented regression." Pattern Recognition46.3 (2013): 628-641.
Introduction
Related workFeature extractionManifold learningDistance metric learningAge determination model
Proposed method
Performance evaluation
Conclusion and future work
Outline
6/28
Related workTopic Method
Feature extraction Local binary pattern (LBP)Gabor filterASM/AAM
Manifold learning Local sensitive discriminate analysis (LSDA)Maximum margin projection (MMP)
Distance metric learning Relevant component analysis (RCA)Discriminative component analysis (DCA)
Age estimation model KNN-SVRSVM-BDTOrdinal hyperplanes Ranker (OHRanker)
7/28
OutlineIntroduction
Related work
Proposed methodActive appearance modelMaximum margin projectionLabel-sensitive DCAkNN-SVR
Performance evaluation
Conclusion and future work
8/28
Active appearance modelIntroduction
Capable of extracting the shape and the texture of an image
Data and landmarkWe labeled 54 imgs with 68 feature points,
9/28
Active appearance modelBuilding AAM concept
Data representationData normalizationStatically analysis by PCA
Model• Shape model: • Texture model:
Combine: Appearance model:
10/28
Maximum margin projectionIntroduction
Manifold learningSemi-supervised dimensionality reductionObjective
◦ Nearby points with same label are close to each other◦ Nearby points with different labels are far apart
Discovering the local manifold structure by Nearest Neighbor Graph
11/28
Maximum margin projectionMMP algorithm
12/28
Presetting• Training set: , n=1:N• Find a projection matrix , then Algorithm• Construct neighbor relationship• Construct within-class weight and
between-class weight matrix• Optimize: • Solve the problem by Laplacian
Eigenmaps
Laplacian eigenmaps
Discriminative component analysisIntroduction
Distance metric learningObjective:
◦ Find a good distance metric which can be used for similarity measureOvercome the disadvantage of relevant component analysis (RCA) Supervised dimensionality reductionLearn an optimal transformation matrix by feature variance analysis
13/28
Relevant component analysis
Original total variance matrix of DCA
Optimize
14/28
Discriminative component analysis
Our total variance matrix of label-sensitive DCADefine class-to-class weight , sample-to-
class
weighted mean vector
𝐸𝑏 𝐸𝑤
Discriminative component analysis
15/28
𝑋 𝑖=[𝑎11 ⋯ 𝑎𝑑
1
⋮ ⋱ ⋮𝑎1𝑛 ⋯ 𝑎𝑑
𝑛]𝑚𝑖=[𝑚1
𝑖 ,…,𝑚𝑑𝑖 ]
…
Age label i
Age label 1
Age label k
𝑚1=[𝑚11 ,…,𝑚𝑑
1 ]
𝑚𝑘=[𝑚1𝑘 ,…,𝑚𝑑
𝑘]
𝐶𝑏
𝐶𝑤 ,𝑖
𝐶𝑤 ,𝑘
𝐶𝑤 ,1
𝐶𝑤
Label sensitive - DCA
16/28
𝑚𝑖=[𝑚1𝑖 ,…,𝑚𝑑
𝑖 ]
𝑚1=[𝑚11 ,…,𝑚𝑑
1 ]
𝑚𝑘=[𝑚1𝑘 ,…,𝑚𝑑
𝑘]
𝑚𝑖′
𝑚1′
𝑚𝑖′
𝑚𝑘′
�̂�𝑏
�̂�𝑏 , 1
�̂�𝑏 , 𝑖
�̂�𝑏 ,𝑘
𝑚1′
𝑚𝑖′
𝑚𝑘′
𝑋 𝑖
𝑋𝑘
𝑋 1 �̂�𝑤 ,1
�̂�𝑤 ,𝑖
�̂�𝑤 ,𝑘
�̂�𝑤
𝐽 ( 𝐴 )=argmax𝐴
|𝐴𝑇 �̂�𝑏𝐴||𝐴𝑇 �̂�𝑤 𝐴|
kNN-SVRBecause of the advantage about MMP and ls-DCA, we use kNN to do local regression
Algorithm
17/28
Input query
kNN
…
…
Regression
OutlineIntroduction
Related work
Proposed method
Performance evaluationMORPH datasetDataset imbalanceSettingExperimentError analysis
Conclusion and future work
18/28
MORPH dataset
19/28
<20 20-29 30-39 40-49 50+0
2000
4000
6000
8000
10000
12000
14000
16000
18000
7469
1632515357
12050
3933
831
23092910
1988
451
6638
14016
12447
10062
3482
Total Female Male
Dataset imbalanceTo deal with this challenge, we modify neighbor size when doing k-nearest neighbor
This method called Neighbor Size Modification[1] which gives a huge weight to insufficient range
20/28[1] Chao, Wei-Lun, Jun-Zuo Liu, and Jian-Jiun Ding. "Facial age estimation based on label-sensitive learning and age-oriented regression." Pattern Recognition46.3 (2013): 628-641.
Age range Image number21~25 226~30 14931~35 59536~40 73841~45 60146~50 37351~55 18256~60 6461~65 2566~70 471~75 376~80 2total 2738
21~25 26~30 31~35 36~40 41~45 46~50 51~55 56~60 61~65 66~70 71~75 76~800
100
200
300
400
500
600
700
800
2
149
595
738
601
373
182
6425 4 3 2
SettingControl Variables
AAM feature: 5-fold cross validation
Independent VariablesProposed methodChao’s methodOHRankerSVM-BDT
Performance evaluationMean Absolute Error (MAE)
21/28
ExperimentMethod MAE
AAM＋ kNN-SVR 6.3374
AAM＋ MMP＋ kNN-SVR 5.8872
AAM＋ DCA＋ kNN-SVR 5.9929
AAM＋ MMP＋ DCA＋ kNN-SVR 5.9228
AAM＋ DCA＋ MMP＋ kNN-SVR 5.8279
22/28
Method MAE
AAM＋ *kNN-SVR 5.9172
AAM＋ *MMP＋ *kNN-SVR 5.7279
AAM＋ ls-DCA＋ *kNN-SVR 5.6931
AAM＋ *MMP＋ ls-DCA＋ *kNN-
SVR5.6242
AAM＋ ls-DCA＋ *MMP＋ *kNN-
SVR5.7987
Notation: Neighbor size modification (*) , Label sensitive (ls-)
AAM ＋ kNN-SVR
AAM ＋MMP ＋kNN-SVR
AAM ＋ DCA＋ kNN-SVR
AAM ＋MMP ＋ DCA＋ kNN-SVR
AAM ＋ DCA＋ MMP ＋
kNN-SVR
5.2
5.4
5.6
5.8
6
6.2
6.4 6.3374
5.88725.9929 5.9228
5.82795.9172
5.7279 5.6931 5.62425.7987
without ls- and NSM with ls- and NSM
Method MAE
AAM＋ OHRanker
(without cost-sensitive)6.4644 (4.48)
AAM＋ SVM-BDT
(33/41/51)6.7607 (4.2)
Chao’s method:
AAM＋ ls-RCA＋ ls-MFA＋ kNN-SVR5.8150 (4.44)
Error analysis - 1
23/28
The limit of neighbor size modification and its trade off.
21~25 26~30 31~35 36~40 41~45 46~50 51~55 56~60 61~65 66~70 71~75 76~800
100200300400500600700800
2
149
595
738
601
373
18264 25 4 3 2
61
kNN
Error analysis - 2The image which has beard and the neighbors almost have beard.
24/28
kNN
Error analysis – 3,4Face shape and expression of one’s eyes maybe influence the neighbor search.
Mouth maybe be the clue of judgement.
25/28
kNN
kNN
OutlineIntroduction
Related work
Proposed method
Performance evaluation
Conclusion and future work
26/28
Conclusions and future workConclusions
The local regression accuracy is actually promoted by our method.The images which was found by Neighbor Search look like input query ,but
we think it will more appropriate for facial expression recognition.
Future workNeed to make the best use of the dataset, while 2738/55134 is not a
effective usage.Need more experience on how to build a proper AAM model for different
applications or feature combination.Deal with the dataset imbalance problem, we also can do some modification
on SVR.Build this system on mobile device.
27/28
Thanks for listening!
28/28
Related workFeature extraction
LBP (Local binary pattern)◦ Ojala, Timo, Matti Pietikainen, and David Harwood. "Performance evaluation of texture
measures with classification based on Kullback discrimination of distributions." Pattern Recognition, 1994. Vol.
Gabor filter◦ Lee, Tai Sing. "Image representation using 2D Gabor wavelets." Pattern Analysis and
Machine Intelligence, IEEE Transactions on 18.10 (1996): 959-971.ASM/AAM ( Active shape/appearance model )
◦ Cootes, Timothy F., et al. "Active shape models-their training and application."Computer vision and image understanding 61.1 (1995): 38-59.
◦ Cootes, Timothy F., Gareth J. Edwards, and Christopher J. Taylor. "Active appearance models." Computer Vision—ECCV’98. Springer Berlin Heidelberg, 1998. 484-498.
BIF ( Bio-inspired feature )◦ Guo, Guodong, et al. "Human age estimation using bio-inspired features."Computer Vision
and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009.
29/28
Related workManifold learning
LSDA (Locality sensitive discriminant analysis)◦ Cai, Deng, et al. "Locality Sensitive Discriminant Analysis." IJCAI. 2007.
MMP (Maximum margin projection)◦ He, Xiaofei, Deng Cai, and Jiawei Han. "Learning a maximum margin subspace for image
retrieval." Knowledge and Data Engineering, IEEE Transactions on20.2 (2008): 189-201.
Distance metric learningRCA (Relevant component analysis)
◦ Shental, Noam, et al. "Adjustment learning and relevant component analysis."Computer Vision—ECCV 2002. Springer Berlin Heidelberg, 2002. 776-790.
DCA (Discriminative component analysis)◦ Hoi, Steven CH, et al. "Learning distance metrics with contextual constraints for image
retrieval." Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on. Vol. 2. IEEE, 2006.
30/28
Related workAge determination model
kNN-SVR◦ Chao, Wei-Lun, Jun-Zuo Liu, and Jian-Jiun Ding. "Facial age estimation based on label-
sensitive learning and age-oriented regression." Pattern Recognition46.3 (2013): 628-641.
SVM-BDT◦ Han, Hu, Charles Otto, and Anil K. Jain. "Age estimation from face images: Human vs.
machine performance." Biometrics (ICB), 2013 International Conference on. IEEE, 2013.Label distribution learning
◦ Geng, Xin, Chao Yin, and Zhi-Hua Zhou. "Facial age estimation by learning from label distributions." Pattern Analysis and Machine Intelligence, IEEE Transactions on 35.10 (2013): 2401-2412.
Ordinal hyperplanes Ranker◦ Chang, Kuang-Yu, Chu-Song Chen, and Yi-Ping Hung. "Ordinal hyperplanes ranker with
cost sensitivities for age estimation." Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 2011.
31/28
Chao 127
OHRanker
32/28