geometry on positive definite matrices induced from v...
TRANSCRIPT
Geometry on
Positive Definite Matrices
Induced from V-Potentials and
Its Application
Atsumi Ohara
University of Fukui
Joint work with Shinto Eguchi (Institute of Statistical Mathematics)
量子系の統計的推測とその幾何学的構造, RIMS 2014 Nov. 12 1
1. Introduction
: the set of Positive Definite
real symmetric matrices
● logarithmic characteristic func. on [Vinberg 63], [Faraut & Koranyi 94]
R);(,detlog)( nPDPPP
2 2
φ(P) = -log det P appears in
Semidefinite Programming (SDP)
self-concordant barrier function
Multivariate Analysis (Gaussian dist.)
log-likelihood function
(structured covariance matrix estimation)
Symmetric cone: log characteristic function
Information geometry on
a potential function in standard case
3
standard IG on
[O,Suda & Amari LAA96]
- : Riemannian metric is the Hesse matrix
of (Fisher for Gaussian) )(P
- , :related to the third derivatives of )(P
Nice properties: -invariant (unique),
KL-divergence, Pythagorean theorem, etc
plays a role of potential function )(P
4
Purpose of this presentation
The other convex potentials
V-potential functions
Their different and/or common geometric
structures
Structure of submanifolds
Decomposition properties of divergences
Application to non-Gaussian pdf’s 5
Outline
Preliminaries
Dualistic geometry induced by V-potentials
Riemannian metric, dual affine connections
Foliated structures
Submanifold of constant determinant
Decomposition of divergence
Applications to multivariate non-Gaussian pdf’s
GL(n)-invariance of geometry of q-Gaussian family
induced by beta-divergence (Quasi-Newton update formulas [Kanamori & O13])
6
7
Preliminaries and Notation
the set of n by n real symmetric matrix
:arbitrary set of basis matrices
(primal) affine coordinate system
Identification
8
PD(n)の情報幾何 [O Suda Amari 96]
PD(n)上に
Riemann計量・・・微少な二点間の距離や角度
接続・・・空間の曲がり方
を定める.
定め方はいろいろあるが,様々な双対性を反映した幾何構造(情報幾何)を考える.
以下,微分幾何速習コース(4枚)
9
接空間
10
Riemann計量 ー接空間の内積ー
11
接続(または平行移動) (1) ー接空間同士の関係ー
12
接続(または平行移動) (2) ー接空間同士の関係ー
13
PD(n)の情報幾何(2)
対称行列集合(ベクトル空間)の基底行列
座標
ポテンシャル関数
Riemann計量
α接続
Information geometry on
Dualistic geometrical structure
: arbitrary vector fields on
:Riemannian metric
a pair of dually flat affine connections 14
V-potential function
-The standard case:
PPssV detlog)(log)(
, RR :)(sV
15
Def.
2. Geometry induced by V-potentials
Def.
Rem. The standard case V= -log:
2,0)(,1)(1 kss k
Prop.1 (convexity conditions)
The Hessian matrix of the V-potential is positive
definite on if and only if
16
Assumption: the convexity conditions hold.
- Riemannian metric is
=
Here,
X, Y in sym(n,R) ~ tangent vectors at P
Rem. The standard case V= -log:
=
17
Prop. (affine connections)
Let be the canonical flat connection on .
Then the V-potential defines the following dual
connection with respect to :
18
divergence function
- a variant of relative entropy,
- Pythagorean type decomposition
19
19
: Dually flat structure on
induced by the V-potential
group invariance of
the structure on
Linear transformation on
congruent transformation:
the differential:
TG
TG
GXGX
nGLGGPGP
*)(
),,(,
R
20 20
group invariance of
the structure on
Linear transformation on
congruent transformation:
the differential:
Invariance
metric:
connections:
and the same for
where
TG
TG
GXGX
nGLGGPGP
*)(
),,(,
R
21 21
Prop.
The largest group that preserves the dualistic
structure invariant is
except in the standard case.
),( RnSLGG
),( RnGLGG Rem. the standard case:
Rem. The power potential of the form:
has a special property.
22
with
with
3. Foliated structures
The following foliated structure features the dualistic geometry induced
from every V-potential.
V-potential.
23
leaf
ray
Prop.
Each leaf and ray are orthogonal
to each other with respect to .
Prop.
Every is simultaneously a - and -
geodesic for an arbitrary V-potential.
24
Prop. For any V, the followings hold:
i) Riemannian metric:
ii) Divergence:
iii) Dual connection:
25
Submanifolds of const det
Induced geometry on from
Prop.
Each leaf is a Riemannian sym. sp.
26
Submanifolds of const det
: Involutive isometry of
Submanifolds of const det
Level surface of both and ADG
Normal vector field N
Centro-affine immersion
27
Submanifolds of const det
Lemma [UOF 00]
The submfd is 1-conformally flat.
Assume
.
Then
where
28
Submanifolds of const det
+ Illustration: decomposition of divergence (1)
If and with ,
then
where
29
Prop.
Each leaf is a homogeneous space of
constant negative curvature
Shown by Gauss eq. and
Lemma (modified Pythagorean) [Kurose94]
30
Submanifolds of const det
Decomposition of divergences (2)
Submanifolds of const det
31
+ Combining the two decomposition results
Prop.
If and are orthogonal at ,
then
>0
4. Application to
multivariate statistics
Non Gaussian distribution
(generalized exponential family)
Robust statistics
beta-divergence,
Machine learning, and so on
Nonextensive statistical physics
Power distribution,
generalized (Tsallis) entropy, and so on
32
U-model and U-divergence
U-model
Def.
Given a convex function U on R and set
u=U’, U-model is a family of elliptic pdf’s
specified by P:
:normalizing const. 33
Rem. When U=exp, the U-model is the
family of Gaussian distributions.
U-divergence:
Natural closeness measure on the U-model
,
Rem. When U=exp, the U-divergence is the
Kullback-Leibler divergence (relative entropy). 34
Example: beta-model and beta-
divergence (1)
Beta-model
For and
q-exponential and q-logarithmic functions 35
Example: beta-model and beta-
divergence (2)
Beta-divergence
36
IG induced from divergences
Divergence induces stat mfd structure.
where
37
Prop.
IG on induced from coincides with
derived from the following V-
potential function:
38
Relation between the U- and V-
geometries
Group invariance for the
power potentials
Prop.
V is of the power form
1) Orthogonality is GL(n)-invariant.
2) The dual affine connections derived from the
power potentials are GL(n)-invariant.
Hence,
Both - and -projections
are GL(n) -invariant. 39
Thm [O & Eguchi 13]
IG on induced from coincides with
on induced from
Implication: statistical inference on using
is GL(n)-invariant.
40
Conclusions
Derived dualistic geometry is invariant under the
SL(n,R)-group actions
For power funcion, dual connections and orthogonality are
GL(n,R)-invariant
Each leaf is a homogeneous manifold with a negative
constant curvature
Decomposition of the divergence function
Relation with the U-model with the U-divergence
41
Main References
A. Ohara, N. Suda and S. Amari, Dualistic Differential Geometry of Positive Definite
Matrices and Its Applications to Related Problems, Linear Algebra and its
Applications, Vol.247, 31-53 (1996).
A. Ohara, Geodesics for Dual Connections and Means on Symmetric Cones,
Integral Equations and Operator Theory, Vol.50, 537-548 (2004).
A. Ohara and S. Eguchi, Geometry on Positive Definite Matrices Deformed by V-
Potential and its Submanifold Structure, ISM Research Memorandom No. 950
(2005) and F. Nielsen eds. Geometric Theory of Information, Chap. 2, 31-55,
Springer (2014).
T. Kanamori and A. Ohara,
A Bregman Extension of quasi-Newton updates I: An Information Geometrical
Framework, Optimization Methods and Software, Vol. 28, No. 1, 96-123 (2013).
A. Ohara and S. Eguchi, Group Invariance of Information Geometry on q-Gaussian
Distributions Induced by Beta-Divergence, Entropy, Vol. 15, 4732-4747 (2013).
42