Desanka

Radunovic

–N

UM

ER

ICK

EM

ETO

DE

Linearnaalgebra;jednacin

e

1.D

irektnem

etode3

2.Iterativne

metode

93.

Gradijentne

metode

144.

Nelinearne

jednacineisistem

i18

5.S

opstvenevrednostiivektorim

atrica25

1

Matrice

A=

(aij )

,A∗=

(aji ),

A>=

(aji )

Herm

ite-ovaA∗=

A(A>=

Asim

etricna)

unitarnaA∗=

A−1

(A>=

A−1

ortogonalna)

Norm

evektora

iindukovanenorm

em

atrica‖A‖=

sup

x6=0‖A

x‖‖x‖

uniformna

‖x‖∞

=max

1≤i≤n |x

i |,‖A‖∞

=max

1≤i≤n

n∑j=

1 |aij |

apsolutna‖x‖

1=

n∑i=

1 |xi |,

‖A‖1=

max

1≤j≤n

n∑i=

1 |aij |

euklidska‖x‖

2=

√√√√

n∑i=

1 |xi | 2,

‖A‖2=√

max

1≤i≤nλi (A∗A)

2

sferna‖A‖s=

√√√√

n∑i=

1

n∑j=

1 |aij | 2

,‖A

x‖2≤‖A‖s ‖

x‖2

uslovljenostcond

(I)=

1≤

cond(A)=‖A‖·‖A−1‖≤∞,

cond(A)=∞

zadet(

A)=

0

Sopstvene

vrednostiivektori

Ax=

λx

⇐⇒

D(λ)≡

det(A−λI)=

0

|λ|‖x‖

=‖λx‖

=‖A

x‖≤‖A‖‖

x‖−→

|λ|≤‖A‖

A∗=

A,

λ1≥λ

2≥···≥

λn,

−→λ

1=

max

x6=0

(A

x,x)

(x,x),

λn=

min

x6=0

(A

x,x)

(x,x)

3

Direktn

em

etod

e

Gauss-o

vametodaelim

inacije

Ax=

b⇐⇒

Ux=

cU=

∗...∗

...0

∗

(U;c)=

Ln−

1Pn−

1Ln−

2Pn−

2 ···L

1P

1 (A;b),

L=

L−1

1···

L−1

n−1=

10

...∗

1

lij=

a(j−

1)

ij

a(j−

1)

jj

,Lj=

10

...1

−lj

+1,j

1...

0−ln,j

01

,Pj=

01

0...

11

0...

01

xn=

cn

unn

,xi=

1uii

(

ci −

n∑

j=i+

1

uijxj

)

,i=

n−1,...,1.

4

Trougaonadekompozicija

matrice

A=

LU

u1k=

a1k,

uik=

aik −

i−1

∑

j=1

lij ujk,

k=

i,...,n

lk1=

ak1

u11,

lki=

1uii

(

aki −

i−1

∑

j=1

lkj uji

)

,k=

i+1,...,n,

i=

2,...,n.

Choleskydekompozicija

A=

LL∗,

A∗=

A

l11=√a11,

lii=

√√√√

aii −

i−1

∑

k=1 |lik | 2

,

li1=

ai1

l11,

lij=

1ljj

(

aij −

i−1

∑

k=1

likljk

)

,1<

j<

i≤n.

5

Householder-o

vamatrica

H=

I−2

ww∗,

‖w‖2=

1

osobineH=

H∗=

H−1

iliH

2=

I

y=

Hx

−→‖y‖

2=‖x‖

2,

(y,x)=

(x,y)

x=

(x1,...,xm) >,

σ=‖x‖

2,

x1=|x

1 |eıα

y=

ke1=

(k,0,...,0) >

=H

x−→

k=−σeıα

w=

x−ke1

‖x−ke1 ‖

2

QR

dekompozicija

A=

QR

R=

∗...∗

...0

∗

=Tn−

1...T

1A,

Q=

T1...Tn−

1,

Q∗=

Q−1=

Tn−

1...T

1

Tm=

1...

0|

0...

0...

|...

0...

1|

0...

0

0...

0|

......

...|

Hm

0...

0|

dim(Hm)=

n−m+1,

xm=(

a(m−1)

m,m

...a(m−1)

n,m

)>(Am={

a(m

)i,j

}

=TmAm−1,A

0≡A)

6

Singularnadekompozicija

(SVD)

A=

UW

V∗

U∗=

U−1,

V∗=

V−1,

W=

(

D0

00

)

,D=

σ1

0...

0σr

singularnevrednosti

σ2k=

λk (A∗A),

σ1≥σ

2≥···≥

σr>0

dim(A)=

m×n,

r≤min(m,n),

dim(U)=

m×m,

dim(V)=

n×n,

(A∗A)V=

V(W∗W)

−→V=(v

1,...,v

n

)

,(A∗A)vk=

σ2kvk

(AA∗)U=

U(W

W∗)

−→U=(u

1,...,u

m

)

,(AA∗)

uk=

σ2kuk

k=

1,...r,

r=

n=

dim(A)−→

σ1=‖A‖2,

σn=

1

‖A−1‖

2

,cond

(A)=‖A‖2 ‖A−1‖

2=

σ1

σn

7

sistem

LU

Ly=

Pb,

Ux=

y(PA

x=

LU

x=

Pb)

Cholesky

Ly=

b,

L∗x=

y

QR

y=

Q∗b

,R

x=

y

SV

Dx=

VW−1U∗b

determ

inan

ta

LU

det(P

A)=±det(A)=

det(L)·

det(U)=

u11 ···

unn

Cholesky

det(A)=

(l11 ···

lnn)2

inverzna

matrica

AA−1=

I−→

Axi=

ei ,

i=

1,...,n.

A−1=

(x1,...,x

n)

,I=

(e1,...,en)

8

Iterativne

meto

de

F(x)=

0⇐⇒

x=

G(x)

x(k+1)=

G(x(k)),

k=

0,1

,...,

limk→∞

x(k)=

x,

x=

G(x)

kontrakcija‖G(x)−

G(y)‖≤q‖x−y‖,

0≤q<1

x∈Rn

q=

max

z‖J(

G(z))‖,

J(

G(z))

=

{

∂gi

∂xj (z

)

}

ni,j=

1

greska

apriorna‖x−x

(k)‖≤

qk

1−q ‖G(x(0

))−x(0

)‖−→

k≥ln

ε(1−

q)

‖G(x

(0))−

x(0

)‖lnq

aposteriorna‖x−x

(k)‖≤

q

1−q ‖x(k)−

x(k−

1)‖

−→‖x(k)−

x(k−

1)‖≤1−q

qε

9

Geom

etrijskainterpretacija

metode

iteracijex(k+1)=

g(x(k)),

(n=

1)

--

--

•

x0

x1

x2

x3

•

x0

x1

x2

x3

x0

x1

x2

x3

•

x0

x1

x2

x3

6 6

6 6

q>

10

<q

<1

−1

<q

<0

q<−

1

10

”Preco

nd

ition

ing

”(preduslovljavanje)

matricom

P

Ax=

b⇐⇒

Px=

Nx+

bza

A=

P−N

x(0

)proizvoljno

Px

(k+

1)=

Nx

(k)+

b,

k=

0,1

...,

x(k+1)=

P−1N

x(k)+

P−1b

konvergencijaJ(

G(x))

≡P−1N

−→‖P−1N‖<1

zaN=

P−A

x(k+

1)=

x(k)−

P−1(A

x(k)−

b)

vektorgreske

r(k)=

Ax

(k)−

b,

x(k+1)=

x(k)−

P−1r(k)

11

Meto

de,

A=

L+D+U

(donjitrougao+

dijagona+

gornjitrougao),

Jacobi

P=

D,

N=−(L+U)

x(k+

1)

=−D−1(L+U)x

(k)+P−1b

=x

(k)−

D−1r(k)

Gauss-S

eidel

P=

L+D,

N=−U

x(k+

1)

=−(L+D) −

1U

x(k)+(L+D) −

1b

=x

(k)−

(L+D) −

1r(k)

Relaksacija

ωA

x=

ωb,

0<ω<2

(ω=

1G

-S)

P=

ωL+D,

N=−(

ωU+(ω−1)D)

x(k+

1)=−(ωL+D) −

1(

ωU+(ω−1)D)

x(k)+ω(ωL+D) −

1b

=x

(k)−

ω(ωL+D) −

1r(k)

12

Rich

ardson-ovametoda

(uopstenjeuvo

-denjemparam

etraτk )

stacionarnax(k+1)=

x(k)−

τP−1r(k)

(τk ≡

τ)

P−1=

I−→

τopt=

2

M+m,

M=

maxi |λ

i (A)|,

m=

mini |λ

i (A)|

‖x−

x(k)‖

2≤(

M−m

M+m

)

k‖x−

x(0

)‖2

nestacionarnax(k+1)=

x(k)−

τkP−1r(k)

P−1=

I−→

τj,opt

=2

M+m+(M−m)cos

2j−

12kπ,

j=

1,...,k,

‖x−

x(k)‖

2≤2

(√M−√m

√M

+√m

)k‖

x−

x(0

)‖2,

13

Gradije

ntnemetode

A=

A∗,

Ax=

b⇐⇒

F(x)

=min

F(x)

funkcionalF(x)=

12(A

x,x)−

(b,x)

gradijent∇F( x

(k))=

(

∂F

∂x1

...

∂F

∂x

n

)>=

Ax(k)−

b=

r(k)

x(k+1)=

x(k)+

λkd(k),

k=

0,1

,...,

limk→∞

x(k)=

x

popravkaλk=−(∇

F(x

(k)),d

(k))

(

Ad

(k),

d(k))

=−

∑

ni=1∂F(x

(k))

∂x

id(k)

i∑

ni=1

∑

nj=

1∂

2F(x

(k))

∂x

i ∂x

jd(k)

id(k)

j

14

po

koo

rdin

atnisp

ust

d(k)=

e(k),

e(k)=(0···

01

0···

0)>

najstrm

ijispu

std(k)=−∇F(x

(k))=−

r(k),

λk=−δk

x(k+1)=

x(k)+

δkr(k),

δk=

(r(k),

r(k))

(

Ar(k),

r(k)),

k=

0,1

,...,

kon

jug

ovanig

radijen

td(k)=

p(k),

(

Ap

(i),p

(j))

=δ(i−

j)

x(k+1)=

x(k)−

λkp(k),

k=

0,1

,...,

r(k)=

Ax

(k)−

b,

p(k+

1)=

r(k+

1)−

µkp

(k),

p(0

)=

r(0

)

λk=

(r(k),

p(k))

(

Ap

(k),

p(k)),

µk=

(

Ap

(k),

r(k+

1))

(

Ap

(k),

p(k))

15

Pokoordinatniinajbrzispust

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

−4

−2 0 2 4 6 8

10

12

14

16

16

Metodanajm

anjih

kvadrata

min

x ‖r(x

)‖2

‖r(x

)‖22=‖A

x−

b‖22=

(A>A

x,x)−

2(A>

b,x)+(b

,b)

Pridruzeniproblem

sasam

okonjugovanimoperatorom

A>/

Ax=

x−→

A>A

x=

A>

x

F(x)=

12(A>A

x,x)−

(A>

b,x)

Ax=

b⇐⇒

‖r(x)‖2=

min‖r(x

)‖2,

F(x)

=min

F(x)

Om

ogucavaprim

enugradijentnih

metoda

ikadam

atricasistem

anije

Herm

ite-ova

17

Nelin

earne

jedn

acine

f(x)=

0,

x∈R1

Newton-ovametoda

◦◦◦

◦

•

x0

x1

x∗

◦◦◦ ◦

•

x∗

x1

x0

◦

x2

◦

x2

x(k+1)=

x(k)−

f(

x(k))

f ′(

x(k)),

x(k+1)=

x(k)−

f(

x(k))

f ′(

x(0))

18

greska|x−

x(k)|≤

M2

2m1 |x

(k)−

x(k−

1)| 2

−1

−0.8

−0.6

−0.4

−0.2

00.2

0.40.6

0.81

−1

−0.8

−0.6

−0.4

−0.2 0

0.2

0.4

0.6

0.8 1x=

1m

1=

min|f′(x)|

M2=

max|f′′(x)|

Uslovikonvergencije:

•f∈C

1[a,b]

•f(a)f(b)<0

•sign

f′(x)=

const,f′(x)6=

0

•sign

f′′(x)=

const

•f(x0 )f′′(x0 )

>0

19

Regula-fa

lsi

(levo)x(k+1)=

x(k)−

f(

x(k))

f(

xF

)−f(

x(k))(xF−

x(k))

◦ ◦◦◦

◦

◦

◦◦•

x0

x1

x2

x3

◦◦

◦ ◦◦◦

◦◦

•

x0

x1

x2

x3

x∗

x∗

Secica

x(k+1)=

x(k)−

f(

x(k))

f(

x(k−

1))−

f(

x(k))(x(k−

1)−

x(k))

(desno)

20

Iteracijem

etoderegula-falsi

−7

−6

−5

−4

−3

−2

−1

0−

350

−300

−250

−200

−150

−100

−50 0 50

x=−

1.000026526307451

f ′(x)≈

f(

xF

)−f(

x(k))

xF −

x(k)

Greska:

|x−x(k)|≤

M1 −m

1

m1|x

(k)−

x(k−

1)|

m1=

min|f′(x)|

M1=

max|f′(x)|

21

Metodapolovlje

nja

x≈

x(k)=

12(ak+

bk),

greska|x−x

(k)|≤

1

2k+

1(b−

a)≤ε

ilik≥[

lnb−aε

ln2

]

[a,b]≡

[a0,b0 ]⊃

···⊃[ak ,bk ]⊃

···,

f(ak )f(bk )<0,

bk −

ak=

12k(b−

a)

MetodaBairsto

w-a

Pn(x)=

xn+

a1xn−

1+···

+an=

0

Pn(x)=

(x2+

px+

q)(xn−

2+

b1xn−

3+···

+bn−

3x+

bn−

2)+

rx+

s

bm=

am−pbm−1 −

qbm−2,

m=

1,...,n,

b−1=

0,

b0=

1

r=

an−

1 −pbn−

2 −qbn−

3=

bn−

1,

s=

an −

qbn−

2=

bn+pbn−

1

cm=

bm−pcm−1 −

qcm−2,

m=

1,...,n−1,

c−1=

0,

c0=

1

cn−

2 ∆p+cn−

3 ∆q=

bn−

1

c ′n−1 ∆

p+cn−

2 ∆q=

bn

∆p≡∆p(k)=

p(k+

1)−

p(k)

∆p≡∆q(k)=

q(k+

1)−

q(k),

k=

0,1,....

22

Sistem

inelin

earnih

jedn

acina

F(x)=

0,

x∈Rn

F(x)=

f1(x)

...fn(x)

,x=

x1...xn

,F′(x

)=

J(F(x))

=

∂f1(x)

∂x1

···∂f1(x)

∂x

n...

......

∂fn(x)

∂x1

···∂fn(x)

∂x1

Newton-ovametoda

x(k+1)=

x(k)−

[F′(x

(k))]−

1F(x

(k))

‖F′(x

)−F′(y

)‖≤γ‖

x−

y‖

[F′(x

)] −1

postojii‖[F′(x

)] −1‖≤β

−→

‖[F′(x

(0))] −

1F(x

(0))‖≤α

x,y

ukonveksnom

skupuC∈X,

h=

αβγ

2<1

limk→∞

x(k)=

x,

F(x)=

0

greska

‖x−

x(k)‖≤αh

2k−

1

1−h

2k

23

Metodaite

racije

F(x)=

0⇐⇒

x=

G(x)≡{gi (x

)}ni=

1

x(0),

x(k+1)=

G(x

(k))

,k=

0,1

,...,

q=

max

x‖J(G

(x))‖

<1

−→limk→∞

x(k)=

x,

x=

G(

x)

greska‖x−

x(k)‖≤ε

za‖x

(k)−

x(k−

1)‖≤1−q

qε

ilik≥ln

ε(1−

q)

‖G(

x(0

)

)−x

(0)‖

lnq

Gradije

ntnemetode

F(x)

=0

⇐⇒

F(x)

=min

F(x)

x(k+1)=

x(k)+

λkd(k),

λk=−

∇F(x

(k))·

d(k)

(d(k)) >

A(x

(k))

d(k),

k=

0,1

,....

F(x

(k)+λk d

(k))

≈F(x

(k))

+λk ∇

F(x

(k))·

d(k)+

12λ

2k(d

(k)) >

A(x

(k))

d(k)

24

So

pstven

evred

no

stiivektori

Axk=

λkxk

D(λk )=

det(A−λkI)=

(−1)n(λnk+p1λn−

1k

+···+

pn)=

0

Interp

olacija

D(λ)≡

n∑i=

0

(

n∏

j=

0

j6=i

x−µj

µi −

µj

)

D(µi ),

|µi |≤‖A‖

Le

Verrier

pk=−1k(Sk+p1Sk−

1+p2Sk−

2+···

+pk−

1S

1 ),

k=

2,...,n.

p1=−S

1,

Sk=

tr(Ak)=

n∑i=

1

a(k)

i,i=

λk1+···

+λkn

Krilov

b(n−

1)

ip1+b(n−

2)

ip2+···

+b(0

)ipn=−b(n)

i,

i=

1,...,n,

b(k)=

(b(k)

1,b(k)

2,...,b(k)

n) >

=A

b(k−

1)=

Akb

(0),

b(0

)proizvoljno

Dan

ilevskiT−1AT=

P=

p1

...pn−

1pn

10

0...

01

0

,P

yk=

λk y

k ,

xk=

Tyk ,

k=

1,...,n,

25

Givens-o

vametodarotacije

(Um

atricarotacije,

U−1=

U∗)

A∼A′=

U∗n−

1,n ···

U∗24U∗23AU

23U

24 ···

Un−

1,n=

∗......∗

∗...

......

0∗∗

B=

U∗k,l A

Uk,l ,

Ukl=

10

...cosφ

−e −

ıψsin

φ...

eıψsin

φcosφ

...0

1

←k

←l

cosφ=

|ak,k−

1 |√

|ak,k−

1 | 2+|al,k−

1 | 2,

e −ıψsin

φ=

al,k−

1

ak,k−

1

α−→

bl,k−

1=

0

26

A

24

68

10

246810

G**A

24

68

10

246810

A*G

24

68

10

246810

G*A

*G*

24

68

10

24681027

Jacobije

vametoda

A∗=

A

Am=

U>m···

U>1A

U1 ···

Um

limm→∞

Am=

D=

diag(λ1,...,λn)

limm→∞

U1 ···

Um=

U=

(x1

...x

n)

Um≡Ukl=

10

...cosφ

−sin

φ...

sinφ

cosφ

...0

1

←k

←l

tan2φ=

2akl

akk −

all

−→a(m

)k,l

=0

Am={a(m

)i,j},

ν2m=

n∑i,j

=1

i6=j

a(m

)i,j

2,

ν2=

n∑i,j

=1

i6=j

a2i,j ,

σ≥n,

ν2m≤ν2(1−

2σ2

)

m→m→∞0

tacnostνm≤εν

zaa(m

)i,j≤

εnν

ilim≥

2lnε

ln(1−

2σ2 )

28

Householder-o

vametoda

A∼A′=

Q∗AQ=

Tn−

2...T

1AT

1...Tn−

2=

∗......∗

∗...

......

0∗∗

Tm=

1...

0|

0...

0...

|...

0...

1|

0...

0

0...

0|

......

...|

Hm

0...

0|

,Q=

T1...Tn−

2

M=

dim(Hm)=

n−m,

xm=(

a(m−1)

m+

1,m

...a(m−1)

n,m

)

>(

Am={

a(m

)i,j

}

=TmAm−1,A

0≡A)

x=

(x1,...,xM) >,

σ=

√√√√

M∑i=

1 |xi | 2,

x1=|x

1 |eıα,

k=−σeıα,

β=(

σ(σ+|x

1 |))−

1,

u=

x−ke1,

H=

I−βu

u∗

(

H=

H∗=

H−1)

29

LR

metoda

Ai=

LiRi ,

Li=

10

......

∗...1

,Ri=

∗...∗

......

0∗

Ai+

1=

RiLi=

Li+

1Ri+

1,

i=

1,2

,....

Ai+

1=

L−1

iAiLi=

(L

1 ···Li ) −

1A

1(L

1 ···Li ),

Ai≡

Ai1=

TiUi=

(L

1 ···Li )(Ri ···

R1 )

Teorema:

limi→∞Ai=

limi→∞Ri=

λ1

...∗

......

0λn

,limi→∞Li=

Iako

•LR

dekompozicije

Ai=

LiRi

postojeza

svakoi=

1,2,...,

•|λ

1 |>|λ

2 |>···

>|λn |,

•postoje

dekompozicije

X=

LxRx ,

Y=

LyRy ,

gdeje

A=

XDY,

D=

diag(λ

1,...,λn),

X=(x

1,...,x

n

)>,

Y=

X−1

30

QR

metoda

Ai=

QiRi ,

Q−1

i=

Q∗i,

Ri=

∗...∗

......

0∗

Ai+

1=

RiQi=

Qi+

1Ri+

1,

i=

1,2

,....

Ai+

1=

Q∗iAiQi=

(Q

1 ···Qi ) ∗

A1(Q

1 ···Qi ),

Ai≡

Ai1=

PiUi=

(Q

1 ···Qi )(Ri ···

R1 )

Teorema:

limk→∞S∗k−

1QkSk=

I,

limk→∞S∗k−

1AkSk−

1=

limk→∞S∗kRkSk−

1=

λ1

...∗

......

0λn

Sk=

diag(

eıφ

(k)

1,...,eıφ

(k)

n

)

,limk→∞a(k)

jj=

λj ,

j=

1,...,n,

(

Ak={a(k)

jj})

akoje|λ

1 |>|λ

2 |>···

>|λn |

ipostojidekom

pozicijaY=

LyRy ,

A=

XDY,

D=

diag(λ

1,...,λn),

Y=

X−1,

Ly=

10

......

∗...

1

,Ry=

∗...∗

......

0∗

31

Delim

icanp

rob

lemso

pstven

ihvred

no

stiλ1,x1

|λ1 |>|λ

2 |≥···≥

|λn |

Metodaproizv

oljn

ogvektora

limk→∞

v(k+1)

jv(k)

j

=λ1,

limk→∞

v(k)=

x1

v(k)=(

v(k)

1...

v(k)

n

)>,

v(k)=

Av

(k−

1)=

Akv

(0),

v(0

)proizvoljno

Metodaskalarnogproizv

oda

limk→∞

(v(k),

w(k))

(v(k−

1),

w(k))=

λ1,

limk→∞

v(k)=

x1

v(k)=

Av

(k−

1)=

Akv

(0),

w(k)=

A∗w

(k−

1)=

(A∗)kw

(0),

v(0

),w

(0)

proizvoljno

32

Metodatra

gova

|λ1 |=

limk→∞

k √

|tr(Ak)|,

λ1=

limk→∞

tr(Ak+1)

tr(Ak)

,tr(

Ak)=

n∑

i=1

a(k)

i,i

x1≈

Akv(0),

v(0

)proizvoljno

Metodaiscrp

ljivanja

A1=

A−

λ1x1y∗1

A1x1=

0,

A1xk=

λkxk,

k=

2,...,n.

A∗yk=

λkyk,

(x1,y1

)

=y∗1x1=

1

33