manifold learning on probabilistic graphical models 概率图上的流形学习
DESCRIPTION
Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习. 答辩人 : 邵元龙 导师 : 鲍虎军 教授 & 何晓飞 教授 浙江大学 CAD&CG 国家重点实验室 2010 年 3 月 5 日. Outline. Background & Motivation Function Learning v.s. Statistical Modeling Manifold Regularized Variational Inference Algorithm Design & Examples - PowerPoint PPT PresentationTRANSCRIPT
Manifold Learning on Probabilistic Graphical Models概率图上的流形学习
答辩人 : 邵元龙导师 : 鲍虎军 教授 & 何晓飞 教授浙江大学 CAD&CG 国家重点实验室2010 年 3 月 5 日
Outline
Background & Motivation Function Learning v.s. Statistical Modeling
Manifold Regularized Variational Inference Algorithm Design & Examples In Depth Analysis Implementation Experimental Results
Function Learning Given data points , and a function space , find the optimal function , such that
Regularization is Important!!
N 1,
N
i i ix y
2*
1
arg min ,N
i if i
f V y f x f
F*f F
Statistical Modeling All quantities, no matter given or to be
estimated, are random variables. Then we model the joint distribution.
, ,,|
, ,
p dpp
p p d d
H V Θ θ θH V
H VV H h V Θ θ h θ
e.g. Gaussian Mixture Model
Difficulties
How many components are there? Should there be any “components” ?
Difficulties (continued)
What if data reside on a non-trivial manifold
Efforts towards Non-Parametric, but …
What we want…
Review GMM
Function Learning embedded. :f X Z
Problem Formulation
What to regularize? Where to regularize?
Manifold Learning
Manifold Assumption Y changes smoothly with X, and we have
so should be small over manifold Minimizing it over the manifold,
f x x f x
f xx
f x
2minx
f dx
MM
Manifold Regularization
2
2
2,
min
1x
ij i ji j
f dx
S f x f xN
S
MM
Manifold Regularization
2
2
2,
2
2,
min
1
1
x
iji
i
j
ij j
i
j
j
i
f dx
f x f xSN
S yN
y
S
S
MM
Transductive Learning
Problem Formulation
What to regularize? Where to regularize?
Variational Inference For , define , a var. dist.
Approximate the true posterior with it
by minimizing the KL divergence
H H q H
|H
q q H p
H
H H V
* arg min || |q
q KL q pH
H H H V
Manifold Regularized Variational Inference
* arg minq
q H
H F
2
,
|| |
,ij i ji j
KL q p
S d q Z q Z
H H V
S
F
How to Optimize?
* arg minH
H q Hq H
H
HF
2
,
|| |
,ij i ji j
KL q p
S d q Z q Z
H H V
S
F
Optimization Algorithm
* arg mini
iii
q ZZ Zq Z F
2
,: ,
|| | , 2 ,ii iZ Zi i Z ij i j
j i j j i
KL q Z p Z S d q Z q Z
MB S
F
1
*
for ,...,
arg min || |N
Hq H
H Z Z
q H KL q H p H
MB
An Illustration
0.2 2 200
Works Done
Example Distribution Types Convergence Proof Convexity Analysis (More TODO) Computational Complexity Numerical Stability A Flexible Inference Engine
YASIE (Yet Another Statistical Inference Engine)
Interface Design Inference Scheduling Type-Free Mixture Model Design Issues (e.g. Balance of Memory & Comp.
Time)
Experiments
Data Clustering Gaussian Mixture Model
Image Annotation Link Mixture of Unigram
Image Annotation Model Link Mixture of Unigram
Image Similarity Graph
“?” should be something like “Barcelona”
?
Image Annotation Performances
Image Annotation Examples
Any Question? 实验室的老师们:鲍虎军老师,何晓飞老师,蔡登老师,刘新国老师,章国锋老师,黄劲老师…… 师兄师弟师妹们:董子龙,姜翰青,周源,张驰原,林斌斌,薛维,瞿新泉,姚冠红……感谢你们一直以来给我的帮助!