单击此处编辑母版标题样式 class-oriented regression embedding 报告人：陈燚 2011...

单击此处编辑母版标题样式单击此处编辑母版标题样式

Class-oriented Regression Embedding

报告人：陈燚

2011 年 8 月 25日

单击此处编辑母版标题样式单击此处编辑母版标题样式报告提纲1. Background

2. Related Works

2.1 Linear Regression-based Classification

2.2 Neighborhood Preserving Embedding & Sparsity Preserving Projections

3. Class-oriented Regression Embedding

4. Experiments


1. Background

单击此处编辑母版标题样式单击此处编辑母版标题样式Background

• The minimum reconstruction error criterion is widely used in the recent progress of subspace classification, such as in SRC and LRC

J. Wright, A. Yang, S. Sastry, Y. Ma, Robust face recognition via sparse representation, IEEE Trans. Pattern Anal. Mach. Intell. 31 (2), 210–227, 2009.

I. Naseem, R. Togneri, and M. Bennamoun. Linear Regression for Face Recognition. IEEE Trans. on PAMI, 2010.

单击此处编辑母版标题样式单击此处编辑母版标题样式A brief review

• SRC:

• LRC

Classification rule:

• is the coefficients of the ith class

i iy X

y X

i

min i iiy X

单击此处编辑母版标题样式单击此处编辑母版标题样式Nearest Space Classifiers

• Definition: The nearest subspace of a given sample

• Measurement: Reconstruction Error

Stan Z. Li: Face Recognition Based on Nearest Linear Combinations. CVPR 1998: 839-844


2.Related Works

单击此处编辑母版标题样式单击此处编辑母版标题样式LRC

1T Ti i i ii

y X X X X y

最小二乘法

1 2[ , ,..., ]ipi i i iX x x x i iy X β线性子空间假设

1T Ti i i i

β X X X y第 i类的

重构结果

min , 1,2,...,iid i cy

样本的类别即是最小重构误差的类

2

i id

y y y

单击此处编辑母版标题样式单击此处编辑母版标题样式NPE & SPP

2

min

s.t. 1

i ij ji j

ijj

x W x

W

min

s.t. 1

T T

T T aa XMX a

a XX a

T M I W I W

Objective Function

The difference between NPE and SPP the reconstructive strategy.

NPE: KNN SPP: Global SparseXiaofei He, Deng Cai, Shuicheng Yan, and HongJiang Zhang. Neighborhood preserving embedding, ICCV, 1208–1213, 2005.

Qiao, L.S., Chen, S.C., Tan, X.Y., Sparsity preserving projections with applications to face recognition. Pattern Recognition 43 (1), 331–341, 2010.


3. Class-oriented Regression Embedding

单击此处编辑母版标题样式单击此处编辑母版标题样式Assumption of SRC and LRC

• A given sample belongs to the class with minimum reconstruction error

Problem: Does this assumption holds well in real world

applications?

单击此处编辑母版标题样式单击此处编辑母版标题样式Examples

• The training face images

单击此处编辑母版标题样式单击此处编辑母版标题样式Examples

0 5 10 15 20 25 30 35 400

200

400

600

800

1000

1200

0 5 10 15 20 25 30 35 400

100

200

300

400

500

600

700

800

900

0 5 10 15 20 25 30 35 400

200

400

600

800

1000

1200

1400

0 5 10 15 20 25 30 35 400

200

400

600

800

1000

1200

1400

1600

320

14

17

单击此处编辑母版标题样式单击此处编辑母版标题样式Motivation

• LRC uses downsampled images directly for classification, which is not optimal for LRC.

• We aim to find the subspace that conforms to the assumption. In this low-dimensional subspace, A sample can be best represented by its intra-class samples.

单击此处编辑母版标题样式单击此处编辑母版标题样式Algorithm

• Objective function:

• To avoid degenerate solutions, we constraint

• Then we have:

Where

and

1T T a XX a

T T T T X I β β ββ X a XX a

2min j j j

i i i ii j i j

ε y Yβ

1

2

0

0 c

W

Wβ

W

1 2, ,..., ini i i i W β β β

单击此处编辑母版标题样式单击此处编辑母版标题样式Example

• Reconstructive Strategy of CRE NPE and SPP

CRE

NPE

SPP

单击此处编辑母版标题样式单击此处编辑母版标题样式SSS problem

T T TT X I β β ββ X XXa a

TXX is singular in SSS case. We apply PCA to reduce the dimensionality of the origin sample to avoid SSS problem.

单击此处编辑母版标题样式单击此处编辑母版标题样式Ridge Regression-based Classification

1T Ti i i ii

y X X X X y

最小二乘法

1 2[ , ,..., ]ipi i i iX x x x i iy X β线性子空间假设

1Ti i

Ti i

X Xβ X y第 i类的

重构结果

min , 1,2,...,ii

d i cy样本的类别即是最小重

构误差的类

2

i id

y y y

May be singular

2min i i i iJ β y X β

• Solution: Ridge Regression

22min i i i i iJ β y X β β

1T Ti i i l i i

β X X I X y 1T

i iT

i i

X Xβ X y

单击此处编辑母版标题样式单击此处编辑母版标题样式Steps• Input: Column sample matrix • Output: Transform matrix Step 1: Project the training samples onto a PCA

subspace: Step 2: Construct the global reconstruction coefficient

matrix using .Step 3: Solve the generalized eigenvectors of corresponding to the

first d smallest eigenvalues.

1 2[ , ,..., ]cX X X X

TPCAX P X

β X

T T T T X I β β ββ X φ XX φ

CREP


4. Experiments

单击此处编辑母版标题样式单击此处编辑母版标题样式Experiments on YALE-B Experiments on the YALE-B database

Method 5 Train 10 Train 20 Train

PCA+NNC 36.1(176) 52.7(362) 68.9(727)

LDA+NNC 73.4(37) 87.0(37) 91.3(37)

NPE+NNC 65.7(77) 79.0(93) 82.7(152)

SPP+NNC 60.2(51) 76.5(72) 84.4(91)

CRE+NNC 66.3(43) 58.6(112) 54.3(161)


PCA+ SRC 72.4(91) 85.8(153) 92.6(192)

LDA+ SRC 72.7(37) 84.6(35) 91.7(37)

NPE+ SRC 68.8(51) 81.5(80) 90.2(102)

SPP+ SRC 69.2(51) 83.4(63) 92.0(82)

CRE+ SRC 78.6(65) 89.5(79) 93.4(90)


PCA+LRC 59.8(101) 82.7(148) 85.6(190)

LDA+LRC 65.3(37) 84.1(37) 87.4(37)

NPE+LRC 70.4(112) 82.7(205) 85.3(240)

SPP+LRC 72.5(51) 86.0(72) 91.3(91)

CRE+LRC 80.7(43) 92.4(83) 97.2(161)

LRC 58.0 81.7 90.9

Comparisons of recognition rates using CRE plus NNC/LRC/SRC on the YALE-B database with 10 and 20 training samples each class respectively.

20 30 40 50 60 70 80 90 100 1100.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Dimensions

Rec

ogni

tion

rate

s

CRE+LRC

CRE+NNC

CRE+SRC

30 40 50 60 70 80 90 100

0.4

0.5

0.6

0.7

0.8

0.9

1

Dimensions

Rec

ogni

tion

rate

s

CRE+LRC

CRE+NNC

CRE+SRC

Comparisons of recognition rates using 5 methods plus LRC on the YALE-B database with 10 and 20 training samples each class respectively.

20 30 40 50 60 700.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Dimensions

Rec

ogni

tion

rate

s

CRE+LRC

PCA+LRC

LDA+LRCNPE+LRC

SPP+LRC

30 40 50 60 70 80 900.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

DimensionsR

ecog

nitio

n ra

tes

CRE+LRC

PCA+LRC

LDA+LRCSPP+LRC

NPE+LRC

The recognition rates of CRE plus LRC, SPP plus SRC and direct LRC on the YALE-B databases with 20 training samples of each class.

30 40 50 60 70 80 900.4

0.5

0.6

0.7

0.8

0.9

1

Dimensions

Rec

ogni

tion

Rat

es

CRE+LRC

SPP+SRCLRC

单击此处编辑母版标题样式单击此处编辑母版标题样式Experiments on FERETMethod 3 Train 4 Train 5 Train 6 Train

PCA+NNC 29.4(203) 33.0(242) 38.6(253) 42.6(286)

LDA+NNC 61.9(33) 65.8(199) 69.9(199) 75.3(199)

NPE+NNC 58.6(22) 62.3(51) 66.2(46) 70.1(72)

SPP+NNC 36.9(146) 43.2(151) 48.6(176) 50.2(181)

CRE+NNC 64.2(53) 69.4(55) 73.0(71) 77.6(82)

Method 3 Train 4 Train 5 Train 6 Train

PCA+ SRC 53.8(122) 62.8(118) 68.7(121) 73.4(134)

LDA+ SRC 66.7(33) 74.6(26) 80.1(36) 86.4(38)

NPE+ SRC 64.3(42) 70.7(54) 76.4(60) 82.6(68)

SPP+ SRC 52.9(151) 63.7(172) 69.8(185) 74.6(198)

CRE+ SRC 75.6(32) 81.4(37) 86.3(43) 91.6(46)

Method 3 Train 4 Train 5 Train 6 Train

PCA+LRC 40.7(298) 48.6(312) 52.0(335) 54.5(352)

LDA+LRC 65.4(39) 73.4(30) 78.6(51) 84.1(62)

NPE+LRC 61.3(40) 68.7(65) 72.4(92) 77.4(77)

SPP+LRC 50.2(146) 58.7(151) 64.2(176) 68.0(181)

CRE+LRC 85.4(53) 90.2(55) 94.1(71) 97.9(82)

LRC 42.0 50.6 55.4 61.2

Comparisons of recognition rates using CRE plus NNC/LRC/SRC on the FERET database with 5 and 6 training samples each class respectively.

10 20 30 40 50 60

0.4

0.5

0.6

0.7

0.8

0.9

1

Dimensions

Rec

ogni

tion

rate

s

CRE+LRC

CRE+NNC

CRE+SRC

10 20 30 40 50 600.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Rec

ogni

tion

rate

sDimensions

CRE+LRC

CRE+NNC

CRE+SRC

A comparison of recognition rates using 5 methods plus LRC on the FERET database with 6 training samples each class respectively.

10 20 30 40 50 60 70 800

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Dimensions

Rec

ogni

tion

rate

s

CRE+LRC

PCA+LRC

LDA+LRCSPP+LRC

NPE+LRC

The recognition rates of CRE plus LRC, SPP plus SRC and direct LRC on the FERET databases with 6 training samples of each class.

10 20 30 40 50 60 70 800

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Dimensions

Rec

ogni

tion

rate

s

CRE+LRC

SPP+SRCLRC

单击此处编辑母版标题样式单击此处编辑母版标题样式Experiments on Cenparmi Method

Classifier

PCA LDA NPE SPP CRE

NNC 87.6(30) 88.2(9) 85.8(19) 86.9(33) 87.6(33)

SRC 90.0(41) 82.6(9) 89.6(21) 92.1(31) 93.6(31)

RRC 92.1(32) 84.8(9) 92.4(23) 88.1(33) 95.6(38)

The recognition rate curves of PCA, LDA, NPE, SPP and LSPP plus RRC on the CENPARMI handwritten numeral database.

The recognition rate curves of CRE plus RRC/SRC/NNC versus the dimensions on the CENPARMI handwritten numeral database.

0 5 10 15 20 25 30 35 400.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Dimensions

Rec

ogni

tion

rate

s

CRE+RRC

PCA+RRC

LDA+RRCNPE+RRC

SPP+RRC

0 5 10 15 20 25 30 35 400.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Dimensions

Rec

ogni

tion

rate

s

CRE+RRC

CRE+SRC

CRE+NNC

单击此处编辑母版标题样式单击此处编辑母版标题样式Comparisons

0 5 10 15 20 25 30 35 400

100

200

300

400

500

600

700

800

900

0 5 10 15 20 25 30 35 400

100

200

300

400

500

600

700

800

0 5 10 15 20 25 30 35 400

200

400

600

800

1000

1200

1400

0 5 10 15 20 25 30 35 400

200

400

600

800

1000

1200

1400

1600


0 5 10 15 20 25 30 35 400

50

100

150

200

250

300

350

400

450

500

0 5 10 15 20 25 30 35 400

50

100

150

200

250

300

350

400

0 5 10 15 20 25 30 35 400

200

400

600

800

1000

1200

0 5 10 15 20 25 30 35 400

100

200

300

400

500

600

700

800

900


谢谢！

报告人：陈燚

2011 年 8 月 25日

单击此处编辑母版标题样式 class-oriented regression embedding 报告人：陈 燚 2011...

Documents

单击此处编辑母版标题样式 class-oriented regression embedding 报告人：陈燚 2011...