chin-hsien fang( 方競賢 ), ju-chin chen( 陳洳瑾 ), chien-chung tseng( 曾建中 ),and...
TRANSCRIPT
![Page 1: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/1.jpg)
Chin-Hsien Fang(方競賢 ), Ju-Chin Chen(陳洳瑾 ), Chien-Chung Tseng(曾建中 ),and Jenn-Jier James Lien(連震杰 )
Department of Computer Science and Information Engineering,National Cheng Kung University
HUMAN ACTION RECOGNITION IN TEMPORAL-VECTOR TRAJECTORY LEARNING FRAMEWORK
1
![Page 2: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/2.jpg)
+ Motivation+ System flowchart+ Training Process+ Testing Process+ Experimental Results + Conclusions
2
![Page 3: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/3.jpg)
+ Traditional Manifold classification (ex: LDA , LSDA…) *Only spatial information *The input data are continuous sequences *Temporal information should be considered
3
![Page 4: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/4.jpg)
ASM
h*w
d
d*(2t+1)
d*(2t+1)
h*w
d
d*(2t+1)
d*(2t+1)
4
![Page 5: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/5.jpg)
LPP
Temporal data
Metric Learning
5
![Page 6: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/6.jpg)
+ Why dimension reduction?– To reduce the calculation cost
+ Why LPP (Locality Preserving Projections)?– Can handle non-linear data with linear transformation matrix
– Local structure is preserved
6
![Page 7: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/7.jpg)
Try to keep the local structure while reducing the dimension
7
![Page 8: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/8.jpg)
ijij
jT
iT Wxaxa
2)(minarg
ij
TTijj
Ti
T aXLXaWxaxa 2)(2
1
1aXDXa TTSubject to
Where L = (D - W)
Objective function:
L : Laplacian matrixD : Diagonal matrixW : Weight matrix
8
![Page 9: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/9.jpg)
+ Three kinds of temporal information1. LTM(Locations temporal motion of Mahalanobis
distance)2. DTM(Difference temporal motion of Mahalanobis
distance)3. TTM(Trajectory temporal motion of Mahalanobis
distance)
9
![Page 10: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/10.jpg)
LTM
An input sequence: },......,{ 21 nxxxX LPP
},......,{ 21 nyyyY
Temporal
}',......','{' 21 nyyyY
],...,,,,...,[' 11 tiiiitii yyyyyy where
10
![Page 11: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/11.jpg)
DTM
}',......','{' 21 nyyyY
],...,,,,...,[' 11 tiiiiiiitiii yyyyyyyyyy
where
1iy
1iy
iy1 ii yy1 ii yy
11
![Page 12: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/12.jpg)
TTM
}',......','{' 21 nyyyY
],...,,,,...,[' 1111 titiiiiiititii yyyyyyyyyy
where
1iy 1iy
iy
ii yy 1
1 ii yy
2iy21 ii yy
12 ii yy
2iy 12
![Page 13: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/13.jpg)
+ Mahalanobis distance1. Preserving the relation of the data
2. Doesn’t depend on the scale of the data
13
![Page 14: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/14.jpg)
yiyi
yj
yl
yj
yi
yl
yi
yj
yl
LME Space
LMNN
LPP+Temporal Space
)12*( tdiy
ii yy E
iy
)12*( tdiy
14
ijk
ijkikijij
jiT
jiij YYMYY )1()()( ''''
Minimize :
Subject to :
ijkjiT
jikiT
ki YYMYYYYMYY 1)()()()( ''''''''
0ijk
(i)
(ii)
(iii) M has to be positive semi-definite
![Page 15: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/15.jpg)
LPP
Metric Learning
Temporal data
K-NN
15
![Page 16: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/16.jpg)
Test data
Training data
3
1
1
K=5
The winner takes all~~
Labeled as
16
The number of nearest neighbor
![Page 17: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/17.jpg)
17
![Page 18: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/18.jpg)
18
![Page 19: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/19.jpg)
+ Our TVTL framework makes impressive progress compared to other traditional methods such as LSDA
+ Temporal information do have positive influence
+ DTM , TTM are better than LTM because they consider the correlation of the data
19
![Page 20: Chin-Hsien Fang( 方競賢 ), Ju-Chin Chen( 陳洳瑾 ), Chien-Chung Tseng( 曾建中 ),and Jenn-Jier James Lien( 連震杰 ) Department of Computer Science and Information Engineering,](https://reader033.vdocuments.pub/reader033/viewer/2022061602/56649f1d5503460f94c3473b/html5/thumbnails/20.jpg)