das verfahren von cholesky ein geod¨atischer beitrag zur ... filedas verfahren von cholesky ......

Schuh GW2014, Berlin, 7. Okt. 2014 – 1

Das Verfahren von Cholesky—

ein geodatischer Beitrag zur numerischen Mathematik

Wolf-Dieter Schuh

Institut fur Geodasie und Geoinformation

Universitat Bonn

Geodatische Woche 2014Session 6: Theoretische Geodasie

Berlin, 7.-9. Oktober 2014

Contents

Choleskyfactorization

Choleskyapplications

Example: GOCEprocessing

Resume


Cholesky factorization

Cholesky applications

Example: GOCE processing

Resume

Andre-Louis Cholesky


A.-L. Cholesky

Cholesky approach


Cholesky inversion



Resume


Andre-Louis Cholesky(1875 - 1918)

Commandant Benoit (1924):

NOTE SUR UNE METHODE DE RESOLUTION DESEQUATIONS NORMALES PROVENANT DE L’APPLICATIONDE LA METHODE DES MOINDRES CARRES A UN SYSTEMED’EQUATIONS LINEAIRES EN NOMBRE INFERIEUR A CELUIDES INCONNUES. — APPLICATION DE LA METHODE ALA RESOLUTION D’UN SYSTEME DEFINI D’EQUATIONSLINEAIRES.Bulletin geodesique, 1924, 2, 67-77



A.-L. Cholesky

Cholesky approach


Cholesky inversion



Resume


Brezinski (2006?)

unpublished manuscript:

Cholesky, A. (2. Dec. 1910):

Sur la resolution numerique des

systemes d’ equations lineaires

(On the numerical solution of

systems of linear equations)



A.-L. Cholesky

Cholesky approach


Cholesky inversion



Resume


Brezinski (2006?)

unpublished manuscript:

Cholesky, A. (2. Dec. 1910):

Sur la resolution numerique des

systemes d’ equations lineaires

(On the numerical solution of

systems of linear equations)

(Sept. 2014)

Cholesky approach


A.-L. Cholesky

Cholesky approach


Cholesky inversion



Resume


Nx = n =

RTR x︸︷︷︸z

= n =

RTz = n R x = z

= =

forward substitution backward substitution

⇒ z ⇒ x



A.-L. Cholesky

Cholesky approach


Cholesky inversion



Resume


N = RT

R

n11 n12 n13

n12 n22 n23

n13 n23 n33

=

r11r12 r22r13 r23 r33

r11 r12 r13r22 r23

r33

rii =

√√√√ nii −

i−1∑

k=1

r2ki , i=1, . . . ,m

rij =

(

nij −i−1∑

k=1

rkirkj

)

/rii ,i=1, . . . ,mj=i+1, . . . ,m

Cholesky (forward) reduction

Cholesky reduction


A.-L. Cholesky

Cholesky approach


Cholesky inversion



Resume


Equation interpretation/reading:

rii =

√√√√ nii −

i−1∑

k=1

r2ki

Cholesky reduction


A.-L. Cholesky

Cholesky approach


Cholesky inversion



Resume



rii =

√√√√ nii −

i−1∑

k=1

r2ki

i

i i

i

N R

=

√√√√ −

⟨

,

⟩

Cholesky reduction


A.-L. Cholesky

Cholesky approach


Cholesky inversion



Resume



rij =

(

nij −i−1∑

k=1

rkirkj

)

/rii

i

j i

i

N R

j

=

(

−⟨

,

⟩)

/

Cholesky inversion


A.-L. Cholesky

Cholesky approach


Cholesky inversion



Resume


N ⇒ R ⇒ N−1

r11 r12 r13r22 r23

r33

⇒

n(−1)11 n

(−1)12 n

(−1)13

n(−1)12 n

(−1)22 n

(−1)23

n(−1)13 n

(−1)23 n

(−1)33

n(−1)ii =

1

r2ii− 1

rii

m∑

k=i+1

rikn(−1)ik , i=1, . . . ,m

n(−1)ij = −

m∑

k=i+1

rikn(−1)kj ,

i=1, . . . ,mj=i+1, . . . ,m

Cholesky inversion by backward substitution

Cholesky applications




Resume


Cholesky application




Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block





Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

Choleskysolver





Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

CholeskysolverCholeskyinversion





Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

Choleskyparallelsolver





Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

Choleskysparsesolver





Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

Choleskysparsesolver

Choleskypartialinverse





Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

Choleskycomplexsolver





Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

partialCholeskysolver





Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

partialCholeskysolver

CholeskyMoore-Penrose

inverse





Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

⇒decorrelation

Example: GOCE processing




GOCE-TIM

GMM decorrelation

Cholesky revisited

Toeplitz-Cholesky

Adaptive filter

Resume


GOCE-TIM gravity field model


GOCE-TIM data segments440 mio observations, eff. 1273 days, 88 segments

Special focus on LOOC (13-Jun-2012 — 21-Oct-2013)4 × 40,578,861 observations, eff. 463 days, 48 segments

LOOC-only models have the same accuracy level as the TIM RL4 model

Gauss-Markov model — complete decorrelation




GOCE-TIM

GMM decorrelation

Cholesky revisited

Toeplitz-Cholesky

Adaptive filter

Resume


input: ℓ, Σ ℓ= Hℓ, Σ= I

model: ℓ+ v = Ax ℓ+v=Ax

principle: vTΣ

−1v vTv

(R

−1)T

= H

Σ = RTR ℓ=(R

−1)T

ℓ

A=(R

−1)T

A

Σ=(R

−1)T

ΣR−1 = I





GOCE-TIM

GMM decorrelation

Cholesky revisited

Toeplitz-Cholesky

Adaptive filter

Resume


ℓ = Hℓ

ℓ = (R−1)Tℓ

RTℓ = ℓ

RT ℓt =

1

rtt

(

ℓt −t−1∑

i=1

ritℓi

)

=⇒ linear - time variant - recursive - causal - filter





GOCE-TIM

GMM decorrelation

Cholesky revisited

Toeplitz-Cholesky

Adaptive filter

Resume


pros and cons: recursive decorrelation filters

+ finite filter — finite time series

+ flexible with respect to data distribution (data gaps)

+ utilization of sparse structures — finite covariance functions

– recursive procedures are time-consuming

– time variant filter - attended memory O{n2}– computational complexity O{n3} 1

3(n3 + 8n)

– parallel implementation is very elaborately

– scalability O{√cores}

Decorrelation — Σ =⇒ H


crucial issue: numerical implementation of the factorization Σ =⇒ H

Decomposition of Computation of R orR Decorrelationmatrix

Decorrelation of ℓ

Σ = RTR Forward Cholesky reduction H = (RT )−1 L= (RT )−1L ⇐⇒ R

TL=L

∗

∗ RT

∗ ∗ ∗

L

=

L

Causal recursive filter

Σ =RRT

Backward Cholesky reduc-tion

H =R−1

L=R−1

L ⇐⇒RL= L

∗ ∗ ∗

R ∗

∗

L

=

L

Anti-causal recursive filter

Σ = (R−1)TR−1 Recursive backward edging H = R L= RL

L

=

∗ ∗ ∗

R ∗

∗

L

Anti-causal non-recursive filter

Σ =R−1

(R−1

)T Recursive forward edging H =RT

L=RT

L

L

=

∗

∗ RT

∗ ∗ ∗

L

Causal non-recursive filter

Toeplitz systems — Cholesky reduction


Focus: Causal filter via Cholesky reduction of Toeplitz structured covariancematrices resulting from AR(p) process:

St = α1St−1 + α2St−2 + . . .+ αpSt−p + Et; E ∼ WN(0, Iσ2E)





Yule-Walker equation

γ0 γ1 . . . γp . . . γnγ1 γ0 . . . γp−1 . . . γn−1

......

. . .. . .

...γp γp−1 . . . γ0 . . . γp−1

......

. . .. . .

...γnγn−1 . . . γp−1 . . . γ0

1−α1

...−αp

0...0

=

σ2E

0...00...0






γ0 γ1 . . . γp . . . γnγ1 γ0 . . . γp−1 . . . γn−1

......

. . .. . .

...γp γp−1 . . . γ0 . . . γp−1

......

. . .. . .

...γnγn−1 . . . γp−1 . . . γ0

1−α1

...−αp

0...0

=

σ2E

0...00...0

Causal non-recursive filter(written in reversed order)

γ0 γ1 . . . γjγ1 γ0 . . . γj−1

......

. . ....

γj γj−1 . . . γ0

r(−1)jj r

(−1)jj

r(−1)j−1,jr

(−1)jj

...

r(−1)1j r

(−1)jj

=

10...0






γ0 γ1 . . . γp . . . γnγ1 γ0 . . . γp−1 . . . γn−1

......

. . .. . .

...γp γp−1 . . . γ0 . . . γp−1

......

. . .. . .

...γnγn−1 . . . γp−1 . . . γ0

1−α1

...−αp

0...0

=

σ2E

0...00...0

Causal non-recursive filter(written in reversed order)

γ0 γ1 . . . γjγ1 γ0 . . . γj−1

......

. . ....

γj γj−1 . . . γ0

r(−1)jj r

(−1)jj

r(−1)j−1,jr

(−1)jj

...

r(−1)1j r

(−1)jj

=

10...0

Resume: Efficient computation of the filter coefficients

j < p : r(−1)ij , i=1, . . . , j derived from recursive forward edging (left box)

j ≥ p : r(−1)jj = 1

σE; r

(−1)j−k,j = −αk

σE, k=1, . . . , p fixed by coefficient comparison



Causal non-recursive filters: Rigorous warmup strategy for finite time series

sparse filter withalmost Toeplitz

structure

warmup:computed by Choleskyrecursive forward edging

main part:results direct from theAR coefficients

Adaptive whitening filter


Adaptive causal non-recursive whitening filter: warmup and single gap


structure

main part:results direct from theAR coefficients

warmup:computed by Choleskyrecursive forward edging

data gaps:computed by Choleskyrecursive forward edging

Adaptive whitening filter


Adaptive causal non-recursive whitening filter: multiple gaps

20 40 60 80 100 120

20

40

60

80

100

120


structure

characteristics:#data points : 120#model : AR(7)-process#gaps : 21

42 4559 65 68100108

Resume




Resume


Resume




Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

Resume




Resume



asymmetric

sparse

supernodal

symmetric

regular

Toeplitz cyclic

positive definite

rectangular

singular

infinite

dense

illposed

1D 2D

block

Thank youfor your

attention !

das verfahren von cholesky ein geod¨atischer beitrag zur ... filedas verfahren von cholesky ......

Documents