张量的低秩逼近白敏茹湖南大学数学与计量经济学院 2014-11-15. 目录 1....

张量的低秩逼近

白敏茹湖南大学数学与计量经济学院

2014-11-15

目录

1. 张量的基本概念

2. 张量特征值的计算

3. 张量秩 1 逼近和低秩逼近

4. 张量计算软件

5. 复张量的最佳秩 1 逼近和特征值


1 2

1 2[ ] N

N

n n ni i iX x R

张量：多维数组

1 阶张量：向量

2阶张量：矩阵 A=(aij)

3阶张量：长方体 A=(aijk)

)()2()1()()2()1(

2121 , N

iiiiiiN

NNaaaxaaaX 其中

(1) (2) ( )

1

( ) min{ | }R

Ni i i

i

rank X R X a a a

张量的秩

张量的秩： 1927 年 Hitchcock

NP-Hard

(1) ( )( ) ( ( ), , ( ))n Nrank X rank X rank X n-rank

秩 1 张量：

可计算

其中表示张量 X 的 mode-k mode )(kX

秩 1矩阵： A=a bT = (aibj)


(1) (2) ( )

1

min || ||R

Ni i i i

i

A a a a

(1) ( )

1 2

(1) (2) ( )1 2 3

, , ,

( ) ( ) ( )

min || ||

. . ,

X , , 1, ,

N

N

i i

i

NN

S X X

R R R

n Ri i T iR

A S X X X

s t S R

R X X I i N

张量的低秩逼近：用一个低秩的张量 X 近似表示张量 A

最佳秩 R 逼近

Tucker 逼近

1 2

1 2 1 2 1 1 2 2

1 2

(1) (2) ( )

1 1 1

( )N

N N N N

N

RR RN

i i i r r r i r i r i rr r r

x g a a a

最佳秩 1 逼近： R=1



min rank( )

s.t.

X

X M

M

张量的完备化

低秩张量 M 部分元素被观察到 , 其中是被观察到的元数的指标集 . 张量完备化是指 : 从所观察到的部分元素来恢复逼近低秩张量 M

nm

mm

CxCBx

BxAx

, ,1'

1'1

]1[ mm xAx

* 1 *

* 1

* 1, ,

m

m

T n

S x x

Sx x

x x C x C

Z （ E ） - 特征值

H- 特征值

US- 特征值

20052005, Qi, Qi

B- 特征值 20142014 ，， Cui,Cui, Dai, Nie Dai, Nie

20142014 ，， Ni,Ni, Qi, Bai Qi, Bai

1 [ 1].m mAx x

张量的特征值



对称非负张量的最大 H- 特征值的计算： 1. Ng, Qi, Zhou 2009 ， 2. Chang, Pearson, Zhang 2011 ， 3. L. Zhang, L. Qi 2012 ， 4. Qi, Q. Yang, Y. Yang 2013

Perron-Frobenius 理论

对称张量的最大 Z- 特征值的计算：

1.1. The sequential SDPs method [Hu, Huang, Qi 2013]

2. Sequential subspace projection method[Hao, Cui, Dai. 2014]

3. Shifted symmetric higher-order power method [Kolda,Mayo 2011]

4. Jacobian semidefinite relaxations 计算对称张量所有实的 B- 特征值 [Cui, Dai, Nie 2014]

对称张量的 US- 特征值的计算：

1. Geometric measure of entanglement and U-eigenvalues of tensors, SIAM Journal on Matrix Analysis and Applications， [Ni，Qi， Bai 2014]

2. Complex Shifted Symmetric higher-order power method [Ni, Bai 2014]


3. 张量的秩 1 逼近和低秩逼近张量的秩 1 逼近•最佳实秩 1 逼近的计算方法：交替方向法 (ADM) 、截断高阶奇异值分解 (T-HOSVD) 、高阶幂法 (HOPM) 和拟牛顿方法等。 ---- 局部解，或稳定点Nie, Wang[2013] ：半正定松弛方法 ---- 全局最优解

•最佳复秩 1 逼近的计算方法：Ni, Qi ， Bai[2014] ：代数方程方法 ---- 全局最优解

3. 张量的秩 1 逼近和低秩逼近张量的低秩逼近

•最佳秩 R 逼近的计算方法：交替最小平方法 (ALS)

•最佳 Tucker 逼近的计算方法：

高阶奇异值（ HOSVD ）， TUCKALS3 ， t-SVD

4. 张量计算软件

• Matlab, Mathematica, Maple 都支持张量计算

• Matlab 仅支持简单运算，而对于更一般的运算以及稀疏和结构张量，需要添加软件包（如： N-wayToolbox, CuBatc

h, PLS Toolbox, Tensor Toolbox ）才能支持，其中除 PL

S Toolbox 外，都是免费软件。 Tensor Toolbox 是支持稀疏张量。

• C++ 语言软件： HUJI Tensor Library (HTL) ， FTensor,

Boost Multidimensional Array Library (Boost.MultiArray)

• FORTAN 语言软件： The Multilinear Engine

[A] Guyan Ni, Liqun Qi and Minru Bai, Geometric measure of ent

anglement and U-eigenvalues of tensors, SIAM Journal on Matrix

Analysis and Applications 2014, 35(1): 73-87

[B] Guyan Ni, Minru Bai, Shifted Power Method for computing

symmetric complex tensor US-eigenpairs, 2014, submitted.

5. 复张量的最佳秩 1 逼近和特征值

Basic DefinitionsBasic Definitions

1. A tensor 1. A tensor SS is called is called symmetricsymmetric as its entries as its entries s_s_{{ii1···i···id}}

are invariant under any permutation of their indices.are invariant under any permutation of their indices.2. A 2. A Z-eigenpairZ-eigenpair ((, u, u) to a real symmetric tensor ) to a real symmetric tensor S S is defined byis defined by

3. An 3. An eigenpaireigenpair ((, u, u) to a real symmetric tensor ) to a real symmetric tensor S S is defined byis defined by

20052005, Qi, Qi

20112011, Kolda and Mayo, Kolda and Mayo

[7] T.G. Kolda and J.R. Mayo, Shifted power method for computing tensor eigenpairs, SIAM Journal on Matrix Analysis and Applications, 32(2011), pp. 1095-1124.

uuTuu

uu*Tuu

4. The best rank-one tensor approximation problems4. The best rank-one tensor approximation problems

Assume that Assume that TT a a dd-order real tensor. Denote a rank-one tensor-order real tensor. Denote a rank-one tensor

is to minimizes the least-squares cost functionis to minimizes the least-squares cost function

. Then the rank-one approximation problem. Then the rank-one approximation problem

The rank-one tensorThe rank-one tensor

rank-one approximationrank-one approximation to tensor T. to tensor T.

is said to be is said to be the best realthe best real

If T is a symmetric real tensor, If T is a symmetric real tensor,

the best real symmetric rank-one approximation.the best real symmetric rank-one approximation.

is said to beis said to be

Basic results

• Friedland [2013] and Zhang et al [2012] showed that the Friedland [2013] and Zhang et al [2012] showed that the

best real rank one approximation to a real symmetric tensor,best real rank one approximation to a real symmetric tensor,

which in principle can be nonsymmetric, can be chosen sy which in principle can be nonsymmetric, can be chosen sy

mmetric.mmetric.

• uudd is is the best real rank-one approximation of the best real rank-one approximation of T T if and onlif and onl

y if y if is a Z-eigenvalue of is a Z-eigenvalue of T T with the largest absolute value, with the largest absolute value,

((,u,u) is a Z-eigenpair. [Qi 2011, Friedland2013, ) is a Z-eigenpair. [Qi 2011, Friedland2013, ZhangZhang et al et al

2012]2012]

[8] S. Friedland, Best rank one approximation of real symmetric tensors can be chosen symmetric, Frontiers of Mathematics in China, 8(2013), pp. 19-40.

[9] X. Zhang, C. Ling and L. Qi, The best rank-1 approximation of a symmetric tensor and related spherical optimization problems, SIAM Journal on Matrix Analysis and Applications 33(2012), pp. 806-821.

complex tensors and unitary eigenvalues

A A dd-order complex tensor will be denoted by-order complex tensor will be denoted by

inner productinner product

normnorm

[10] G. Ni, L. Qi and M. Bai, Geometric measure of entanglement and U-eigenvalues of tensors, to appear in SIAM Journal on Matrix Analysis and Applications

The superscript * denotes the complex conjugate. The superscript T denotes transposition.

For For A,B H∈A,B H∈ , define the inner product and norm as, define the inner product and norm as

inner productinner product

normnorm

A rank-one tensorA rank-one tensor

unitary eigenvalue (U-eigenvalue) of T

Denote by Denote by SymSym((d, nd, n)) all symmetric all symmetric dd-order -order nn-dimensional tensors-dimensional tensors

Let Let x x ∈∈ CCnn. Simply denote the rank-one tensor. Simply denote the rank-one tensor

DefineDefine

We call a number We call a number ∈∈ C a C a unitary symmetric eigenvalue (US-eiunitary symmetric eigenvalue (US-ei

genvalue)genvalue) of of S S if if and a nonzero vectorand a nonzero vector

The largest The largest |λ| |λ| is the entanglement eigenvalue. The corresponding ris the entanglement eigenvalue. The corresponding r

ank-one tensor ank-one tensor di ⊗di ⊗ =1=1x x is the closest symmetric separable state.is the closest symmetric separable state.

Theorem 1. Theorem 1. Assume that complex Assume that complex dd-order tensors-order tensors

ThenThen

b) all U-eigenvalues are real numbers;b) all U-eigenvalues are real numbers;

c) the c) the US-eigenpairUS-eigenpair ( (, x, x) to a symmetric ) to a symmetric dd-order complex te-order complex te

nsor nsor S S can also be defined by the following equation systemcan also be defined by the following equation system

oror (1)(1)

3.1. US-eigenpairs of symmetric tensors

Theorem 3. (Takagi’s factorization) Let Theorem 3. (Takagi’s factorization) Let A A ∈∈ CCn×nn×n be a complex sybe a complex symmetric tensor. Then there exists a unitary matrix mmetric tensor. Then there exists a unitary matrix U U ∈∈ CCn×nn×n such tsuch thathat

Case d=2:Case d=2:

Theorem 4. Let Theorem 4. Let A A ∈∈ CCn×nn×n be a complex symmetric tensor. Let be a complex symmetric tensor. Let U U ∈∈ CCn×nn×n be a unitary matrix such thatbe a unitary matrix such that

Let eLet eii = (0= (0, · · · , , · · · , 00, , 11, , 00, · · · , , · · · , 0)0)TT , i , i = 1= 1, · · · , n, · · · , n.. Then bothThen both

andand are US-eigenpairs of are US-eigenpairs of AA..

The number of distinct US-eigenvalues is at most 2The number of distinct US-eigenvalues is at most 2nn..

Theorem 5. Theorem 5. If If 11 = = · · · · · · = = kk > > kk+1+1, , 1 1 ≤ k ≤ n≤ k ≤ n, then the set of all , then the set of all

US-eigenvectors with respect to US-eigenvectors with respect to 11 is is

the set of all US-eigenvectors with respect to the set of all US-eigenvectors with respect to −λ−λ11 is is


The problem of finding eigenpairs is equivalent to solving The problem of finding eigenpairs is equivalent to solving

a polynomial systema polynomial system

Case d Case d 3 3

[8] S. Friedland, Best rank one approximation of real symmetric tensors can be chosen symmetric, Frontiers of Mathematics in China, 8(2013), pp. 19-40.

Theorem 2. Theorem 2. Assume that a complex Assume that a complex dd-order -order nn-dimension symmetric -dimension symmetric tensor tensor S S ∈∈ Sym Sym((d, nd, n). Then). Then

a) if a) if d d ≥≥ 3, 3, d d is an odd integer, and is an odd integer, and 0, then the system (1) is 0, then the system (1) is equivalent to equivalent to

(2)(2)

and the number of US-eigenpairs of (1) is the double of the numband the number of US-eigenpairs of (1) is the double of the number of solutions of (2);er of solutions of (2);

b) if b) if d d ≥≥ 3, 3, d d is an even integer, and is an even integer, and 0, then the system (1) is 0, then the system (1) is equivalent toequivalent to

(3)(3)

and the number of US-eigenpairs of (1) is equal to the number of and the number of US-eigenpairs of (1) is equal to the number of solutions of (3).solutions of (3).

Case d Case d 3 33.2. US-eigenpairs of symmetric tensors

Case d Case d 3 3

Theorem 6. Theorem 6. Let Let d d ≥ 3, ≥ 3, n n ≥≥ 2 be integers, 2 be integers, S S ∈∈ Sym Sym((d, nd, n). If (2) has fi). If (2) has finitely many solutions, thennitely many solutions, then

a) if a) if d d is odd , the number of non-zero solutions of (2) is at mostis odd , the number of non-zero solutions of (2) is at most

b) if b) if d d is even, the number of non-zero solutions of (3) is at mostis even, the number of non-zero solutions of (3) is at most

c) c) S S has at mosthas at most distinct nonzero US-eigenvalues;distinct nonzero US-eigenvalues;

d) for nonzero US-eigenvalues, all the US-eigenpairs of d) for nonzero US-eigenvalues, all the US-eigenpairs of S S are as followsare as follows

where where x x is a solution of (2).is a solution of (2).


Note. Note. 1. Let 1. Let S S be the symmetric 2 ×be the symmetric 2 × 2 ×2 × 2 ×2 × 2 tensor whose non-2 tensor whose non-

zero entries arezero entries are

SS11111111 = 2 = 2, S, S11121112 = = −−11, S, S11221122 = = −−11, S, S12221222 = = −−22, S, S22222222 = 1 = 1..

The number of non-zero solutions of the equation system (2) is 40 The number of non-zero solutions of the equation system (2) is 40

which shows that the bound is tight.which shows that the bound is tight.Note. Note. 2. Cartwright and Sturmfels (2013) showed that every symmetric 2. Cartwright and Sturmfels (2013) showed that every symmetric

tensor has finite E-eigenvalues. At the same time, they indicated that thtensor has finite E-eigenvalues. At the same time, they indicated that th

e magnitudes of the eigenvalues with ||e magnitudes of the eigenvalues with ||x|| x|| = 1 may still be an infinite set = 1 may still be an infinite set

(See Example 5.8 of [Cartwright and Sturmfels (2013) ]), which implie(See Example 5.8 of [Cartwright and Sturmfels (2013) ]), which implie

s that the system s that the system S x S x d−d−11 = = x x has infinite non-zero solutions, where has infinite non-zero solutions, where S S is a is a

symmetric 3 ×symmetric 3 × 3 ×3 × 3 tensor whose non-zero entries are 3 tensor whose non-zero entries are

SS111111 = 2 = 2, S, S122122 = = SS212212 = = SS221221 = = SS133133 = = SS313313 = = SS331331 = 1 = 1..

[11] D. Cartwright and B. Sturmfels, The number of eigenvalues of a tensor, Linear Algebra and its Applications, 438(2013), pp. 942-952

Note. Note. 3. Let 3. Let S S be the symmetric 3×3×3 tensor as in Note 2. Then be the symmetric 3×3×3 tensor as in Note 2. Then x x

= for all 0 = for all 0 < a < < a < 1 are non-zer1 are non-zer

o solutions of o solutions of Sx Sx d−d−11 = = x*x*. It implies that (2) may have infinite non-. It implies that (2) may have infinite non-

zero solutions.zero solutions.

4. Best symmetric rank-one approximation of symmetric tensors

Theorem 7. Theorem 7. Let Let S S be a symmetric complex tensor. Let be a symmetric complex tensor. Let be a US-be a US-

eigenvalue of eigenvalue of SS. Then. Then

a) a) − − is also a US-eigenvalue of is also a US-eigenvalue of SS ; ;

b) b) GG((SS) = ) = maxmax..

Case d=2Case d=2

Theorem 8. Theorem 8. Let Let A A ∈∈ CCn×nn×n be a complex symmetric matrix.be a complex symmetric matrix.

Then for all Then for all x x ∈∈ UEV UEV ((A, A, 11) ∪) ∪ UEV UEV ((A,− A,− 11) and ) and ∈∈ C C

with with ||| | = 1, (= 1, ( xx) ⊗) ⊗ (( xx) are best symmetric rank-one ) are best symmetric rank-one

approximation of approximation of AA..

Case d ≥ 3

The best symmetric rank-one approximation problem is to find a

unit-norm vector x ∈ Cn, such that

By Theorem 7, introducing the US-eigenvalue method, Q1 is equ

ivalent to the following problem

Theorem 9. Theorem 9. Let Let S S ∈∈ Sym Sym((d, nd, n). Then). Then

a) the best symmetric rank-one approximation problem is equivala) the best symmetric rank-one approximation problem is equival

ent to the following optimization problement to the following optimization problem

approximation of approximation of S S for eachfor each

rank-onerank-one

The problem of finding eigenpairs is equivalent to solving The problem of finding eigenpairs is equivalent to solving

a polynomial systema polynomial system

Let Let x x = = y y + + z−z−1, 1, y, z ∈y, z ∈ RRnn. Then Q3 is equivalent to the f. Then Q3 is equivalent to the f

ollowing problemollowing problem

Example 1. Assume that S is a real symmetric tensor with d = 3 and n

= 2. Then Q4 is equivalent to the following optimization problem

Table1. US-eigenpairs of S with S111 = 2, S112 = 1, S122 = −1, S222 = 1.

The best real rank-one approximation is also the best The best real rank-one approximation is also the best complex rank-one approximation.complex rank-one approximation.

The absolute-value largest of Z-eigenvalues is not itThe absolute-value largest of Z-eigenvalues is not its largest US-eigenvalue.s largest US-eigenvalue.

The best real rank-one approximation is sometimes also tThe best real rank-one approximation is sometimes also t

he best complex rank-one approximation even if the tenshe best complex rank-one approximation even if the tens

or is not a symmetric nonnegative real tensor, see Table 1.or is not a symmetric nonnegative real tensor, see Table 1.

The absolute-value largest of Z-eigenvalues is sometimes The absolute-value largest of Z-eigenvalues is sometimes

not its largest US-eigenvalue, see Table 2.not its largest US-eigenvalue, see Table 2.

By observing numerical examples, we find the following results:By observing numerical examples, we find the following results:

Question 1Question 1: What is the necessary and sufficient condition for : What is the necessary and sufficient condition for

the equality of the largest absolute Z-eigenvalue and the equality of the largest absolute Z-eigenvalue and

the largest US-eigenvalue to a real symmetric tensor?the largest US-eigenvalue to a real symmetric tensor?

谢谢大家！

张量的低秩逼近 白敏茹 湖南大学数学与计量经济学院 2014-11-15. 目 录 1....

Documents

张量的低秩逼近白敏茹湖南大学数学与计量经济学院 2014-11-15. 目录 1....