francis’s algorithmfrancis’s algorithm david s. watkins watkins@math.wsu.edu department of...

Francis’s AlgorithmDavid S. Watkins

watkins@math.wsu.edu

Department of Mathematics

Washington State University

Francis’s Algorithm – p. 1

Eigenvalue Problem: Av = λv

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv

How to solve?

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s algorithm,

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s algorithm, aka

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s algorithm, aka

the implicitly shiftedQR algorithm

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s algorithm, aka

the implicitly shiftedQR algorithm

50 years!

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s algorithm, aka

the implicitly shiftedQR algorithm

50 years!

Top Ten of the century (Dongarra and Sullivan)

Francis’s Algorithm – p. 2

John Francis

Francis’s Algorithm – p. 3

Who is John Francis?

Francis’s Algorithm – p. 4

Who is John Francis?born near London in 1934

Francis’s Algorithm – p. 4

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

Francis’s Algorithm – p. 4

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

Francis’s Algorithm – p. 4

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

primitive computer

Francis’s Algorithm – p. 4

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

primitive computer

no software

Francis’s Algorithm – p. 4

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

primitive computer

no software

experimented with a variety of methods

Francis’s Algorithm – p. 4

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

primitive computer

no software

experimented with a variety of methods

invented His algorithm and programmed it

Francis’s Algorithm – p. 4

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

primitive computer

no software

experimented with a variety of methods

invented His algorithm and programmed it

moved on to other things

Francis’s Algorithm – p. 4

Some History

Francis’s Algorithm – p. 5

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s Algorithm – p. 5

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s first paper (QR)

Francis’s Algorithm – p. 5

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s first paper (QR)

A − ρI = QR, RQ + ρI = A

Francis’s Algorithm – p. 5

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s first paper (QR)

A − ρI = QR, RQ + ρI = A repeat!

Francis’s Algorithm – p. 5

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s first paper (QR)

A − ρI = QR, RQ + ρI = A repeat!

Kublanovskaya

Francis’s Algorithm – p. 5

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s first paper (QR)

A − ρI = QR, RQ + ρI = A repeat!

Kublanovskaya

. . . but this is not “Francis’s Algorithm”

Francis’s Algorithm – p. 5

Francis’s Algorithm

Francis’s Algorithm – p. 6

Francis’s AlgorithmSecond paper of Francis

Francis’s Algorithm – p. 6

Francis’s AlgorithmSecond paper of Francis

real matrices

Francis’s Algorithm – p. 6

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

Francis’s Algorithm – p. 6

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

Francis’s Algorithm – p. 6

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

want to stay in real arithmetic

Francis’s Algorithm – p. 6

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

want to stay in real arithmetic

two steps at once

Francis’s Algorithm – p. 6

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

want to stay in real arithmetic

two steps at once

double-shiftQR algorithm

Francis’s Algorithm – p. 6

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

want to stay in real arithmetic

two steps at once

double-shiftQR algorithm

radically different from basic QR

Francis’s Algorithm – p. 6

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

want to stay in real arithmetic

two steps at once

double-shiftQR algorithm

radically different from basic QR

Usual justification: Francis’s implicit-Q theorem

Francis’s Algorithm – p. 6

Francis’s Algorithm

Francis’s Algorithm – p. 7

Francis’s Algorithmupper Hessenberg form

Francis’s Algorithm – p. 7

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

Francis’s Algorithm – p. 7

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI)

Francis’s Algorithm – p. 7

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

Francis’s Algorithm – p. 7

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

computep(A)e1

Francis’s Algorithm – p. 7

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

computep(A)e1 cheap!

Francis’s Algorithm – p. 7

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

computep(A)e1 cheap!

Build unitaryQ0 with q1 = αp(A)e1.

Francis’s Algorithm – p. 7

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

computep(A)e1 cheap!

Build unitaryQ0 with q1 = αp(A)e1.

Perform similarity transformA → Q−1

0AQ0.

Francis’s Algorithm – p. 7

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

computep(A)e1 cheap!

Build unitaryQ0 with q1 = αp(A)e1.

Perform similarity transformA → Q−1

0AQ0.

Hessenberg form is disturbed.

Francis’s Algorithm – p. 7

An Upper Hessenberg Matrix@

@@

@@

@@

@@

@@

@@

Francis’s Algorithm – p. 8

After the Transformation ( Q−10 AQ0)

@@

@@

@@

@@

@@

Francis’s Algorithm – p. 9

After the Transformation ( Q−10 AQ0)

@@

@@

@@

@@

@@

Now return the matrix to Hessenberg form.

Francis’s Algorithm – p. 9

Chasing the Bulge@

@@@

@@

@@

@@@

Francis’s Algorithm – p. 10

Chasing the Bulge@

@@

@@

@@

@@

@

Francis’s Algorithm – p. 11

Done@

@@

@@

@@

@@

@@

@@

Francis’s Algorithm – p. 12

Done@

@@

@@

@@

@@

@@

@@

The Francis iteration is complete!

Francis’s Algorithm – p. 12

Summary of Francis Iteration

Francis’s Algorithm – p. 13

Summary of Francis IterationPick some shifts.

Francis’s Algorithm – p. 13

Summary of Francis IterationPick some shifts.

Computep(A)e1. (p determined by shifts)

Francis’s Algorithm – p. 13

Summary of Francis IterationPick some shifts.

Computep(A)e1. (p determined by shifts)

Build Q0 with first columnq1 = αp(A)e1.

Francis’s Algorithm – p. 13

Summary of Francis IterationPick some shifts.

Computep(A)e1. (p determined by shifts)

Build Q0 with first columnq1 = αp(A)e1.

Make a bulge. (A → Q−1

0AQ0)

Francis’s Algorithm – p. 13

Summary of Francis IterationPick some shifts.

Computep(A)e1. (p determined by shifts)

Build Q0 with first columnq1 = αp(A)e1.

Make a bulge. (A → Q−1

0AQ0)

Chase the bulge. (return to Hessenberg form)

Francis’s Algorithm – p. 13

Summary of Francis IterationPick some shifts.

Computep(A)e1. (p determined by shifts)

Build Q0 with first columnq1 = αp(A)e1.

Make a bulge. (A → Q−1

0AQ0)

Chase the bulge. (return to Hessenberg form)

A = Q−1AQ

Francis’s Algorithm – p. 13

Quicker Summary

Francis’s Algorithm – p. 14

Quicker SummaryMake a bulge.

Francis’s Algorithm – p. 14

Quicker SummaryMake a bulge.

Chase it.

Francis’s Algorithm – p. 14

Remarks

Francis’s Algorithm – p. 15

RemarksThis is pretty simple.

Francis’s Algorithm – p. 15

RemarksThis is pretty simple.

noQR decomposition in sight!

Francis’s Algorithm – p. 15

RemarksThis is pretty simple.

noQR decomposition in sight!

Why call it theQR algorithm?

Francis’s Algorithm – p. 15

RemarksThis is pretty simple.

noQR decomposition in sight!

Why call it theQR algorithm?

Confusion!

Francis’s Algorithm – p. 15

RemarksThis is pretty simple.

noQR decomposition in sight!

Why call it theQR algorithm?

Confusion!

Can we think of another name?

Francis’s Algorithm – p. 15

RemarksThis is pretty simple.

noQR decomposition in sight!

Why call it theQR algorithm?

Confusion!

Can we think of another name?

I’m calling it Francis’s Algorithm.

Francis’s Algorithm – p. 15

RemarksThis is pretty simple.

noQR decomposition in sight!

Why call it theQR algorithm?

Confusion!

Can we think of another name?

I’m calling it Francis’s Algorithm.

This is not a radical move.

Francis’s Algorithm – p. 15

Question

Francis’s Algorithm – p. 16

QuestionHow should we view Francis’s algorithm?

Francis’s Algorithm – p. 16

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Francis’s Algorithm – p. 16

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly?

Francis’s Algorithm – p. 16

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?

Francis’s Algorithm – p. 16

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?

. . . and the answer is:

Francis’s Algorithm – p. 16

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?

. . . and the answer is:Why not?

Francis’s Algorithm – p. 16

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?

. . . and the answer is:Why not?

This simplifies the presentation.

Francis’s Algorithm – p. 16

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?

. . . and the answer is:Why not?

This simplifies the presentation.

I’m putting my money where my mouth is.

Francis’s Algorithm – p. 16

Francis’s Algorithm – p. 17

I’m putting my money where my mouth is . . .

Francis’s Algorithm – p. 17

I’m putting my money where my mouth is . . .

. . . and saving one entire section!

Francis’s Algorithm – p. 17

Pedagogical Pathway

Francis’s Algorithm – p. 18

Pedagogical Pathwayreduction to Hessenberg form

Francis’s Algorithm – p. 18

Pedagogical Pathwayreduction to Hessenberg form

Francis’s algorithm

Francis’s Algorithm – p. 18

Pedagogical Pathwayreduction to Hessenberg form

Francis’s algorithm

Try it out!

Francis’s Algorithm – p. 18

Pedagogical Pathwayreduction to Hessenberg form

Francis’s algorithm

Try it out!

It works great!

Francis’s Algorithm – p. 18

Pedagogical Pathwayreduction to Hessenberg form

Francis’s algorithm

Try it out!

It works great!

Why does it work?

Francis’s Algorithm – p. 18

Ingredients of Francis’s Algorithm

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithmsubspace iteration (power method)

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithmsubspace iteration (power method)

subspace iterationwith changes of coordinate system

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithmsubspace iteration (power method)

subspace iterationwith changes of coordinate system

Krylov subspaces

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithmsubspace iteration (power method)

subspace iterationwith changes of coordinate system

Krylov subspaces(instead of the implicit-Q theorem)

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithmsubspace iteration (power method)

subspace iterationwith changes of coordinate system

Krylov subspaces(instead of the implicit-Q theorem)

Krylov subspaces and subspace iteration

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithmsubspace iteration (power method)

subspace iterationwith changes of coordinate system

Krylov subspaces(instead of the implicit-Q theorem)

Krylov subspaces and subspace iteration

Krylov subspaces and Hessenberg form

Francis’s Algorithm – p. 19

Power Method, Subspace Iteration

Francis’s Algorithm – p. 20

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

Francis’s Algorithm – p. 20

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

Francis’s Algorithm – p. 20

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

Francis’s Algorithm – p. 20

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj

Francis’s Algorithm – p. 20

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj (|λj+1/λj |)

Francis’s Algorithm – p. 20

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj (|λj+1/λj |)

Substitutep(A) for A

Francis’s Algorithm – p. 20

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj (|λj+1/λj |)

Substitutep(A) for A (shifts, multiple steps)

Francis’s Algorithm – p. 20

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj (|λj+1/λj |)

Substitutep(A) for A (shifts, multiple steps)

S, p(A)S, p(A)2S, p(A)3S, . . .

Francis’s Algorithm – p. 20

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj (|λj+1/λj |)

Substitutep(A) for A (shifts, multiple steps)

S, p(A)S, p(A)2S, p(A)3S, . . .

convergence rate|p(λj+1)/p(λj) |

Francis’s Algorithm – p. 20

Subspace Iterationwith changes of coordinate system

Francis’s Algorithm – p. 21

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

Francis’s Algorithm – p. 21

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

Francis’s Algorithm – p. 21

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

build unitaryQ = [q1 · · · qj · · ·]

Francis’s Algorithm – p. 21

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

build unitaryQ = [q1 · · · qj · · ·]

change coordinate system:A = Q−1AQ

Francis’s Algorithm – p. 21

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

build unitaryQ = [q1 · · · qj · · ·]

change coordinate system:A = Q−1AQ

qk → Q−1qk = Q∗qk = ek

Francis’s Algorithm – p. 21

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

build unitaryQ = [q1 · · · qj · · ·]

change coordinate system:A = Q−1AQ

qk → Q−1qk = Q∗qk = ek

span{q1, . . . , qj} → span{e1, . . . , ej}

Francis’s Algorithm – p. 21

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

build unitaryQ = [q1 · · · qj · · ·]

change coordinate system:A = Q−1AQ

qk → Q−1qk = Q∗qk = ek

span{q1, . . . , qj} → span{e1, . . . , ej}

ready for next iterationFrancis’s Algorithm – p. 21

This version of subspace iteration . . .

Francis’s Algorithm – p. 22

This version of subspace iteration . . .

. . . holds the subspace fixed

Francis’s Algorithm – p. 22

This version of subspace iteration . . .

. . . holds the subspace fixed

while the matrix changes.

Francis’s Algorithm – p. 22

This version of subspace iteration . . .

. . . holds the subspace fixed

while the matrix changes.

. . . moving toward a matrix under which

span{e1, . . . , ej}

is invariant.

Francis’s Algorithm – p. 22

This version of subspace iteration . . .

. . . holds the subspace fixed

while the matrix changes.

. . . moving toward a matrix under which

span{e1, . . . , ej}

is invariant.

A →

[

A11 A12

0 A22

]

(A11 is j × j.)

Francis’s Algorithm – p. 22

Application to Francis’s Iteration(first pass)

Francis’s Algorithm – p. 23

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

Francis’s Algorithm – p. 23

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

power method

Francis’s Algorithm – p. 23

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

power method+ change of coordinates

Francis’s Algorithm – p. 23

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

power method+ change of coordinates

q1 → Q−1q1 = e1

Francis’s Algorithm – p. 23

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

power method+ change of coordinates

q1 → Q−1q1 = e1

casej = 1 of subspace iteration with a change ofcoordinate system

Francis’s Algorithm – p. 23

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

power method+ change of coordinates

q1 → Q−1q1 = e1

casej = 1 of subspace iteration with a change ofcoordinate system

. . . but this is just a small part of the story.

Francis’s Algorithm – p. 23

Krylov Subspaces . . .

Francis’s Algorithm – p. 24

Krylov Subspaces . . .. . . and Subspace Iteration

Francis’s Algorithm – p. 24

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

Francis’s Algorithm – p. 24

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

j = 1, 2, 3, . . . (nested subspaces)

Francis’s Algorithm – p. 24

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

j = 1, 2, 3, . . . (nested subspaces)

Kj(A, q) are “determined byq”.

Francis’s Algorithm – p. 24

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

j = 1, 2, 3, . . . (nested subspaces)

Kj(A, q) are “determined byq”.

p(A)Kj(A, q) = Kj(A, p(A)q)

Francis’s Algorithm – p. 24

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

j = 1, 2, 3, . . . (nested subspaces)

Kj(A, q) are “determined byq”.

p(A)Kj(A, q) = Kj(A, p(A)q)

. . . becausep(A)A = Ap(A)

Francis’s Algorithm – p. 24

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

j = 1, 2, 3, . . . (nested subspaces)

Kj(A, q) are “determined byq”.

p(A)Kj(A, q) = Kj(A, p(A)q)

. . . becausep(A)A = Ap(A)

Conclusion: Power method induces nested subspaceiterations on Krylov subspaces.

Francis’s Algorithm – p. 24

power method: q → p(A)kq

Francis’s Algorithm – p. 25

power method: q → p(A)kq

nested subspace iterations:

p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . .

Francis’s Algorithm – p. 25

power method: q → p(A)kq

nested subspace iterations:

p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . .

convergence rates:

|p(λj+1)/p(λj) |, j = 1, 2, 3, . . . , n − 1

Francis’s Algorithm – p. 25

Krylov Subspaces . . .

Francis’s Algorithm – p. 26

Krylov Subspaces . . .. . . and Hessenberg matrices . . .

Francis’s Algorithm – p. 26

Krylov Subspaces . . .. . . and Hessenberg matrices . . .

. . . go hand in hand.

Francis’s Algorithm – p. 26

Krylov Subspaces . . .. . . and Hessenberg matrices . . .

. . . go hand in hand.

A properly upper Hessenberg=⇒

Kj(A, e1) = span{e1, . . . , ej}.

Francis’s Algorithm – p. 26

Krylov Subspaces . . .. . . and Hessenberg matrices . . .

. . . go hand in hand.

A properly upper Hessenberg=⇒

Kj(A, e1) = span{e1, . . . , ej}.

More generally . . .

Francis’s Algorithm – p. 26

Krylov-Hessenberg Relationship

Francis’s Algorithm – p. 27

Krylov-Hessenberg Relationship

If A = Q−1AQ,

Francis’s Algorithm – p. 27

Krylov-Hessenberg Relationship

If A = Q−1AQ,

andA is properly upper Hessenberg,

Francis’s Algorithm – p. 27

Krylov-Hessenberg Relationship

If A = Q−1AQ,

andA is properly upper Hessenberg,

then forj = 1, 2, 3, . . . ,

Francis’s Algorithm – p. 27

Krylov-Hessenberg Relationship

If A = Q−1AQ,

andA is properly upper Hessenberg,

then forj = 1, 2, 3, . . . ,

span{q1, . . . , qj} = Kj(A, q1).

Francis’s Algorithm – p. 27

Application to Francis’s Iteration

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

p(A)Kj(A, e1) = Kj(A, p(A)e1)

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

p(A)Kj(A, e1) = Kj(A, p(A)e1)

i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

p(A)Kj(A, e1) = Kj(A, p(A)e1)

i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}

subspace iteration with a change of coordinatesystem

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

p(A)Kj(A, e1) = Kj(A, p(A)e1)

i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}

subspace iteration with a change of coordinatesystem forj = 1, 2, 3, . . . ,n − 1

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

p(A)Kj(A, e1) = Kj(A, p(A)e1)

i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}

subspace iteration with a change of coordinatesystem forj = 1, 2, 3, . . . ,n − 1

|p(λj+1)/p(λj) | j = 1, 2, 3, . . . ,n − 1

Francis’s Algorithm – p. 28

Details

Francis’s Algorithm – p. 29

Detailschoice of shifts

Francis’s Algorithm – p. 29

Detailschoice of shifts

We change the shifts at each step.

Francis’s Algorithm – p. 29

Detailschoice of shifts

We change the shifts at each step.

⇒ quadratic or cubic convergence

Francis’s Algorithm – p. 29

Detailschoice of shifts

We change the shifts at each step.

⇒ quadratic or cubic convergence

Watkins (2007, 2010)

Francis’s Algorithm – p. 29

Detailschoice of shifts

We change the shifts at each step.

⇒ quadratic or cubic convergence

Watkins (2007, 2010)

Francis’s Algorithm – p. 29

Where is John Francis?

Francis’s Algorithm – p. 30

Where is John Francis?question asked frequently by Gene Golub

Francis’s Algorithm – p. 30

Where is John Francis?question asked frequently by Gene Golub

inquiries by Golub and Uhlig

Francis’s Algorithm – p. 30

Where is John Francis?question asked frequently by Gene Golub

inquiries by Golub and Uhlig

Francis is alive and well,retired in the South of England.

Francis’s Algorithm – p. 30

Where is John Francis?question asked frequently by Gene Golub

inquiries by Golub and Uhlig

Francis is alive and well,retired in the South of England.

was unaware of the impact of his algorithm

Francis’s Algorithm – p. 30

Where is John Francis?question asked frequently by Gene Golub

inquiries by Golub and Uhlig

Francis is alive and well,retired in the South of England.

was unaware of the impact of his algorithm

appearance at the Biennial Numerical AnalysisConference in Glasgow in June of 2009

Francis’s Algorithm – p. 30

John Francis speaking in Glasgow

Francis’s Algorithm – p. 31

A Portion of the Audience

Francis’s Algorithm – p. 32

Afterwards

Photos courtesy of Frank UhligFrancis’s Algorithm – p. 33