geometry on positive definite matrices induced from v...

Geometry on

Positive Definite Matrices

Induced from V-Potentials and

Its Application

Atsumi Ohara

University of Fukui

Joint work with Shinto Eguchi (Institute of Statistical Mathematics)

量子系の統計的推測とその幾何学的構造, RIMS 2014 Nov. 12 1

1. Introduction

: the set of Positive Definite

real symmetric matrices

● logarithmic characteristic func. on [Vinberg 63], [Faraut & Koranyi 94]

R);(,detlog)( nPDPPP

2 2

φ(P) = -log det P appears in

Semidefinite Programming (SDP)

self-concordant barrier function

Multivariate Analysis (Gaussian dist.)

log-likelihood function

(structured covariance matrix estimation)

Symmetric cone: log characteristic function

Information geometry on

a potential function in standard case

3

standard IG on

[O,Suda & Amari LAA96]

- : Riemannian metric is the Hesse matrix

of (Fisher for Gaussian) )(P

- , :related to the third derivatives of )(P

Nice properties: -invariant (unique),

KL-divergence, Pythagorean theorem, etc

plays a role of potential function )(P

4

Purpose of this presentation

The other convex potentials

V-potential functions

Their different and/or common geometric

structures

Structure of submanifolds

Decomposition properties of divergences

Application to non-Gaussian pdf’s 5

Outline

Preliminaries

Dualistic geometry induced by V-potentials

Riemannian metric, dual affine connections

Foliated structures

Submanifold of constant determinant

Decomposition of divergence

Applications to multivariate non-Gaussian pdf’s

GL(n)-invariance of geometry of q-Gaussian family

induced by beta-divergence (Quasi-Newton update formulas [Kanamori & O13])

6

7

Preliminaries and Notation

the set of n by n real symmetric matrix

:arbitrary set of basis matrices

(primal) affine coordinate system

Identification

8

PD(n)の情報幾何 [O Suda Amari 96]

PD（n）上に

Ｒｉｅｍａｎｎ計量・・・微少な二点間の距離や角度

接続・・・空間の曲がり方

を定める．

定め方はいろいろあるが，様々な双対性を反映した幾何構造（情報幾何）を考える．

以下，微分幾何速習コース（４枚）

9

接空間

10

Riemann計量 ー接空間の内積ー

11

接続（または平行移動） （１） ー接空間同士の関係ー

12

接続（または平行移動） （２） ー接空間同士の関係ー

13

PD(n)の情報幾何（２）

対称行列集合（ベクトル空間）の基底行列

座標

ポテンシャル関数

Ｒｉｅｍａｎｎ計量

α接続

Information geometry on

Dualistic geometrical structure

: arbitrary vector fields on

:Riemannian metric

a pair of dually flat affine connections 14

V-potential function

-The standard case:

PPssV detlog)(log)(

, RR :)(sV

15

Def.

2. Geometry induced by V-potentials

Def.

Rem. The standard case V= -log:

2,0)(,1)(1 kss k

Prop.1 (convexity conditions)

The Hessian matrix of the V-potential is positive

definite on if and only if

16

Assumption: the convexity conditions hold.

- Riemannian metric is

=

Here,

X, Y in sym(n,R) ~ tangent vectors at P

Rem. The standard case V= -log:

=

17

Prop. (affine connections)

Let be the canonical flat connection on .

Then the V-potential defines the following dual

connection with respect to :

18

divergence function

- a variant of relative entropy,

- Pythagorean type decomposition

19

19

: Dually flat structure on

induced by the V-potential

group invariance of

the structure on

Linear transformation on

congruent transformation:

the differential:

TG

TG

GXGX

nGLGGPGP

*)(

),,(,

R

20 20

group invariance of

the structure on

Linear transformation on

congruent transformation:

the differential:

Invariance

metric:

connections:

and the same for

where

TG

TG

GXGX

nGLGGPGP

*)(

),,(,

R

21 21

Prop.

The largest group that preserves the dualistic

structure invariant is

except in the standard case.

),( RnSLGG

),( RnGLGG Rem. the standard case:

Rem. The power potential of the form:

has a special property.

22

with

with

3. Foliated structures

The following foliated structure features the dualistic geometry induced

from every V-potential.

V-potential.

23

leaf

ray

Prop.

Each leaf and ray are orthogonal

to each other with respect to .

Prop.

Every is simultaneously a - and -

geodesic for an arbitrary V-potential.

24

Prop. For any V, the followings hold:

i) Riemannian metric:

ii) Divergence:

iii) Dual connection:

25

Submanifolds of const det

Induced geometry on from

Prop.

Each leaf is a Riemannian sym. sp.

26

Submanifolds of const det

: Involutive isometry of

Submanifolds of const det

Level surface of both and ADG

Normal vector field N

Centro-affine immersion

27

Submanifolds of const det

Lemma [UOF 00]

The submfd is 1-conformally flat.

Assume

.

Then

where

28

Submanifolds of const det

+ Illustration: decomposition of divergence (1)

If and with ,

then

where

29

Prop.

Each leaf is a homogeneous space of

constant negative curvature

Shown by Gauss eq. and

Lemma (modified Pythagorean) [Kurose94]

30

Submanifolds of const det

Decomposition of divergences (2)

Submanifolds of const det

31

+ Combining the two decomposition results

Prop.

If and are orthogonal at ,

then

>0

4. Application to

multivariate statistics

Non Gaussian distribution

(generalized exponential family)

Robust statistics

beta-divergence,

Machine learning, and so on

Nonextensive statistical physics

Power distribution,

generalized (Tsallis) entropy, and so on

32

U-model and U-divergence

U-model

Def.

Given a convex function U on R and set

u=U’, U-model is a family of elliptic pdf’s

specified by P:

:normalizing const. 33

Rem. When U=exp, the U-model is the

family of Gaussian distributions.

U-divergence:

Natural closeness measure on the U-model

,

Rem. When U=exp, the U-divergence is the

Kullback-Leibler divergence (relative entropy). 34

Example: beta-model and beta-

divergence (1)

Beta-model

For and

q-exponential and q-logarithmic functions 35

Example: beta-model and beta-

divergence (2)

Beta-divergence

36

IG induced from divergences

Divergence induces stat mfd structure.

where

37

Prop.

IG on induced from coincides with

derived from the following V-

potential function:

38

Relation between the U- and V-

geometries

Group invariance for the

power potentials

Prop.

V is of the power form

1) Orthogonality is GL(n)-invariant.

2) The dual affine connections derived from the

power potentials are GL(n)-invariant.

Hence,

Both - and -projections

are GL(n) -invariant. 39

Thm [O & Eguchi 13]

IG on induced from coincides with

on induced from

Implication: statistical inference on using

is GL(n)-invariant.

40

Conclusions

Derived dualistic geometry is invariant under the

SL(n,R)-group actions

For power funcion, dual connections and orthogonality are

GL(n,R)-invariant

Each leaf is a homogeneous manifold with a negative

constant curvature

Decomposition of the divergence function

Relation with the U-model with the U-divergence

41

Main References

A. Ohara, N. Suda and S. Amari, Dualistic Differential Geometry of Positive Definite

Matrices and Its Applications to Related Problems, Linear Algebra and its

Applications, Vol.247, 31-53 (1996).

A. Ohara, Geodesics for Dual Connections and Means on Symmetric Cones,

Integral Equations and Operator Theory, Vol.50, 537-548 (2004).

A. Ohara and S. Eguchi, Geometry on Positive Definite Matrices Deformed by V-

Potential and its Submanifold Structure, ISM Research Memorandom No. 950

(2005) and F. Nielsen eds. Geometric Theory of Information, Chap. 2, 31-55,

Springer (2014).

T. Kanamori and A. Ohara,

A Bregman Extension of quasi-Newton updates I: An Information Geometrical

Framework, Optimization Methods and Software, Vol. 28, No. 1, 96-123 (2013).

A. Ohara and S. Eguchi, Group Invariance of Information Geometry on q-Gaussian

Distributions Induced by Beta-Divergence, Entropy, Vol. 15, 4732-4747 (2013).

42