고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

73
VisCom TR Series - Copyright c Jonathan Richard Shewchuk and VisCom Commune. (1 1 4 ) , , †† , ‡‡ , , †† , ‡‡ (conjugate gradient) . . . . . . 2 , (steepest descent), (conjugate directions), (conjugate gradient) . (Jacobi) , , . (preconditioning) . . 66 , . . 1 1 - Jonathan Richard Shewchuk. - An Introduction to the Conjugate Gradient Method without the Agonizing Pain. (Natrual Science and Engineering Research Council of Canada) 1967 (National Science Foundation) (Grant ASC-9318163) . (NSERC), (NSF), . 1

Upload: donggyu-park

Post on 14-Apr-2017

219 views

Category:

Engineering


8 download

TRANSCRIPT

Page 1: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VisCom TR Series - Copyrightc©Jonathan Richard Shewchuk and VisCom Commune.

LLLsss���LLL������{{{������ªªªEEEPPPêêê���ªªª���ÁÁÁ (114:::)

������{{{†,���ôôô���‡,îîî���|||††,���êêêõõõ‡‡ bbb

†ôôô<<<¡¡¡ppp, ‡ûûûÙÙÙ¡¡¡ppp, ††ÙÙÙSSS½½½vvvqqq¡¡¡ppp, ‡‡ÙÙÙSSS¡¡¡ppp

ÅÅÅccc

�ªEPê(conjugate gradient)�ªù �±µ¿vÇü�xÍèßð�©õu �Ú��$lq

& �Ī�°.� # ݱ 3ê �ù þ�� � sCõ ���# �[: z<ê {� °ØL �°.�¢

þ�� f��ù��ê$����#�êt[uu�t�y{��ú�q4L�°.� ¢�î

¿��ª�¢îê��� :�©��(¡���Í]$�úL�ߥ3©ë¢�Ù<u¢�ý

!��;î]�}°. ���ªEPê�ª���,ù�³ 8t�u#�©£½��ÍI¢PL

¿�ØqK�°.� Û��¤[[¢ë����8¾�&IX¢L�ê{��н��,�°.

2òÍé��]ú��¢�,�õ�Ì �/ �(steepest descent),�ª�³(conjugate directions),

�ª EPê(conjugate gradient)�ªú îê£ ,�°.Lî¯â� b�ü(Jacobi)�ª,/ �,�ª E

Pê�ª#�½¶úz< �ÚPÌþ,�°.�ýL;�ý(preconditioning)YüxÍ�ªEPê�

ª#�j¥üq�°.���� Û���úÖ3�3 ²L�ùd³ú �°. 66����úCW

ü6,��¢S[Í×�z<ù�°°.�]�ù=��t¿°ô�ª÷¿z<üL�÷6ÂÙÛ��

Aéù�[: z<Y¥ÙCWý°.1

1Ù9� - Jonathan Richard Shewchuk.Ù[C@ - An Introduction to the Conjugate Gradient Method without the Agonizing Pain.Ùf[��u�j#°�W�uÊç(Natrual Science and Engineering Research Council of Canada)� 1967[�W$¡PzYyY¡&³(National Science Foundation)��Ù(Grant ASC-9318163)ú�I�ØqP°.�[t�[>Y@Áù9��>©�6j#°�W�uÊç(NSERC),yY¡&³(NSF),ÓùyvAÙ�>©¿�I��Kt�Ký°.

1

Page 2: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

���[[[ttt���©©©

c©1994 by Jonathan Richard Shewchuk.9���î�:S#³ý��Ù .�[t�9��vè��¿

+I�L,[t�Â�¿IX¢Ã\�{�¢�îÄ3ÄPüq�jþ½�°.

� [t� �ª EPê �ªú � ¢ Ö3 WÙ L òù ¡f�ú ò� æ � ��qP°. #�3

�:P¨, ½A©b£ 4Ì, �ýL 4� �rÿ qE [ê� �[: z<��ê ^÷¦ N² s� ��°

([email protected]);���ÏÚ�Ù��¥«:�j¥�þ,�°.#�"&�[tõ½zè��ßÌ

¢P»�[î��°.

�Ä�ª�©Ì�ù,úWÙ L� �P���3#�éýyõþß(William L. Briggs)� “A

multigrid Tutorial” [2]ú8�¢°.�þù��«�4��ù½¡t:�ÏÚ�$�5��þ�°.

½X �ª� [© 4� NL �� ,� \¾ÙÛú �ò�sL � [t� #L� ª� �� üeú CW

© s}Ï ¡�ò �Íß(Omar Ghattas)�3 "»¢ �Põ �ÿ°. üeú © s}Ï C�ß �Ná(James

Epperson),Ú�ü�¡¬ ¢(David O’Hallaron),C�ßß/Xdß(James Stichnoth),§!ªWî(Nick Tre-

fethen),�ýL°¦����(Daniel Tunkelang)�3ê�P¢°.

�Å¢ÙÛ�ú�ú½�êÀ�<�\Ò�U���õà °:

copyright c©Jonathan Richard Shewchuk and VisCom Commune 2

Page 3: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

1 ���qqq������

����ªEPê(conjugate gradient method)ú�ͲL°ú:, 4 �t¿°ô[¶ú�}�Ú,�þ

��qEþ� ����\�&�M9°.����þ�ú #ê�© �G°°.�þ�ùÂÙÛ�

³ 3�ªEPêõ�Á°÷6,qC¢�[:z<ê{}LqN3�ªEPê�ª��ü�LKü}

���¢(!(hint)ê{���ª�"�úz< L�}°.��ù�¢���_<ÍÛ�Ïf 3

ü}÷6,$ò�ªEPê�ªú�Ͳ�¡f��Ô�ß Ï�Aé×qý�I¦��Ù LÍI¢

NLý�ú�Í�õ��8t5�P°.

�ªEPê�ªùxÍ�Aé�$¢èßðú��æ©�$��PÌü��Ī�°.�ªEP

ê�ªù°üÍ×�èßð�IsîY:�°.

Ax = b (1)

�: x�?ò�¯â�L, b�NL��¯â�6 A�NL��A�(square),Âa(symmetric),j�A

ÙÒ(positive-define)±µ�° -Óùj�ÙAÙÒ(positive-indefinite)±µ�½ê�°. “j�AÙÒ”�X

|ú�y ���q£¹°L£��ê#|�°èU_Ç,�¦%A£�Å�{°.�èßðù`yÛ

�Aé,uSÛu,ç¿Ûu,½¡¾C#ú©@£:[[#Í#�î¢òÛª(finite difference)Yî¢Å

�ª(finite element method)#Y�ù|Å¢��ª�t�s#Í&°.

�ªEPê�ªY�ù�Ä�ªù �±µúPÌ£:�:¦ °.A�S}¢EÍ��$^ù�

ªù AõÛ© L�XÞ(backsubstitution)��©�Aéú��,�°.S}¢ AúÛ© �Ú(ý�è

�ù Â? èßðú �Ä©t ©@ �Ú (ý� è�Y üæ °.�ýL ¢¥ A� Û©ü8 èßðù b�

� �úæ©t�ò3Ãô¿�½�°.�S}¢±µú�ù-?ý¾�õ���ÌÀ¾�� �

±µY üp©Ã�. �±µ A� X�Å�(triangular factors)�� 0� I¨ Ù�� ��:÷¿ Ù�� AÃ

°ýCÌ�°.Û©���-?ý�¢G:[�Ý� £½ê�÷6è����(�½ê�°.î�

q�XÞúYA��Ä�ªÃ°Ì��½ê�°.�8�ÂÙÛ��Ä:�ªù-?ýîñ�mL

�±µ�©�ò3ô�¢°.

���� Û�xͽ¡ú�½°°L�A 6,Lî¯â¦LîY�ù,�©t��?ô°

L£��ê±µU�YxÍë�#�©t��NL�°L�A¢°.�¢�Æ: �Aú�Ö÷¿

����ªEPê�ª��Gõ� ¢¢<� 3u9©#�,�°.

2 vvv���ªªª

=��A�®v��©U_Ã8tè� �.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 3

Page 4: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

±µú vè � æ© Â[�õ PÌ£ ,�6,�[�� ¯âõ #Í4�Ú PÌý°.ße�� �ýß

[�¿v袰.A� n× n±µ�L x® b�¯â,�, n× 1±µ�°.½é 1ù°üY��vÇ¢°.

A11 A12 · · · A1n

A21 A22 A2n

......

...

An1 An1 · · · Ann

x1

x2

...

xn

=

b1

b2...

bn

.

� ¯â� 4:(inner product)ù xT y�L ße� ¦∑n

i=1 xiyiú #Í6°. xT y = yTx�°. x® y� �

p 8 xT y = 0�°.��:÷¿ xT y® yTAx®�� 1× 1±µ�ü�vÇéùße�÷¿D�ý°.

�c?� 0�I¨¯â x�©°üÙ#éú�T£:,±µ A�j�AÙÒ(positive-definite)�°.

xTAx > 0 (2)

��tIX¢�yõ���G¢°Lì� ��.��]ù�°��[: �]�I¦qtj�

AÙÒ ±µ��©�Mù,YqN3°ò3Ã����© �,�ÑÏ��I¨°.�õ�© �

æ©t�j�AÙÒ(positive-definite)��,� 2òÍé�qN3�³úyX��U_Ç�Å��°.

���÷¿ (AB)T = BTATY (AB)−1 = B−1A−1��|Å¢�ƨ#éú��L�r �.

3 2òòòÍÍÍééé

2òÍé(quadratic form)ù�³&©°üY�ùÍ×�ße�ú � 2ò¥½�°.

f(x) =12xTAx− bTx+ c (3)

A�±µ�L x® b�¯â�°.�ýL c�ße�\½�°.�����t A�Âa:�L(symmetric)

j�AÙÒ EÍ,Ax = b�©õ�Ì � f(x)õ/�Ü£½�üú�³&Ã�²¢°.

���;��t���°üY���³¢ìC�C[Cõ��L°j¢�]úz<£,�°.

A =

3 2

2 6

, b =

3

−8

, c = 0. (4)

copyright c©Jonathan Richard Shewchuk and VisCom Commune 4

Page 5: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

èßð Ax = b��� 1�#Í#�°.��:÷¿© x� n��#e8(Ú��)�pò>�° (�:�

�� #e8ù �� n − 1 òÙú ��°).� [C� ©� x = [2,−2]T�°.Âÿü� �òé f(x)� ��

2�#Í&°. f(x)�#Lx�?jù�� 3Y�°. A�j�AÙÒ��:[� f(x)��©A�ü�v

8ùj]xÍ×�P�Y�ù?j�°. (��© è��Ì��&z< 9°)

[�� 1] 2òÙxÍèßð�. ©�xÛ��pò>�°

2òé��Ð�(gradient)�°üY��A�ý°.

f ′(x) =

∂∂x1

f(x)

∂∂x2

f(x)...

∂∂xn

f(x)

. (5)

�Ð��sq�> x�©, f(x)��$À���³ú�òÅ�¯â���°.�� 4ù½é 4Y�

ù\½õ½é 3�Â� �uq��Ð�¯âõz< L�°.P�?j�j]8�±�t��Ð��

0�°. f ′(x)� 0�üêÀ¥÷¿� f(x)ù/�Ü£½�°.

°��Ø¢�zú�©½é 5õ½é 3�Â� �°üúîꣽ�°.

f ′(x) =12ATx+

12Ax− b. (6)

copyright c©Jonathan Richard Shewchuk and VisCom Commune 5

Page 6: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 2] 2òÍé f(x)����. �v8�/9>ù Ax = b�©�°.

[�� 3] 2òÍé�#Lx. �ÍÙMxùô�¢ f(x)ú �°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 6

Page 7: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 4] 2òÍé� �Ð� f ′(x). ?� x�  �, �Ð� ¯â� f(x)� �$ ¾3 \ç � �³ú �ýÅ6 #Lx�

�p¢°.

A�Âa�8��Aéù°üY����Üý°.

f ′(x) = Ax− b. (7)

�Ð�õ 0÷¿ 8Íý�uL� �xÍèßð½é 1úu�°.� t¿ Ax = b�©� f(x)��

G>(critical point)�°. A�Âa�<�I¦�j�AÙÒ(positive-definite)�8©� f(x)�/��°.

0�t Ax = b�©� f(x)ú/�Ü � xúüü÷¿�u£½�°. (A�Âa:��M÷8½é 6��

ªEPê�ª�èßð 12 (AT + A)x = b�©õü3ý°�,úÃ�sL�°.�: 1

2 (AT + A)�Âa

±µ�°.)

Âa j�AÙÒ±µù·�©3^ù��ú L�ú«?qE���> p�t� f®xÍèßð�

© x = A−1bP��[GõU_Ã�.½é 3ú�Ö÷¿ A�Âa EÍ (�±µ�j�AÙÒ(positivie-

definite) ��[G{�)°ü���¥úÃ�½�°. (ÙÀ C1).

f(p) = f(x) +12(p− x)TA(p− x). (8)

A�j�AÙÒ(positive-definite)�L 8,Ù#é 2��©ó��¨�?� p 6= x�©j½�°.�

� x� f�;�/��úy¢°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 7

Page 8: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

j�AÙÒ(positive-definite)±µ�[¢�$�[: �©��±µ� 2òÍé f(x)�j]8�ý°

�,�°.A��cj�AÙÒ�I¦�8°ô� ���°.A�ü�AÙÒ�þ½��Ú��j�

AÙÒ±µ�©ÙÒõ�ö@Y,��� 2�I�æõ��qnù?j�ý°. A�"�±µ EÍ

��î�¢©�{�EÍ�°;©��¦ù #��x,Óù�A¢ f ú���#e8(hyperplane)�

ý°. A� æ�t z<¢ qE EÍê I¦�8, x� K$>(saddle point)� ü6 / ��# �ª EPê

�ª#�ìC£,�°.�� 5�� ¢� ��úÃ�sL�°. b® c�ùj]Í�/�>�q%

�õ@A¢°.� #j]Í?j���³úyX�M�°.

xÍèßðú · Ì q²× Ã�� [C¿ ºÍ£«?/ �Y �ª EPê �ª #� �ª�ù ��

1® �� #e8� pòõ �©t� I¦� �� 2® �� ¥½� /�Üõ �© �[: �©� �  �

:[�°.

4 ///   ������ªªª

/ ��ª�t�,qE��> x(0)�tè� �j]x���±÷¿y� K4²�°.© x�?Û

&�«3ü�S&ú�T£:«��³G x(0), x(1) · · ·úD¢°.

 #�³G�t f��$�ò3�� ��³úxØ¢°.��³ù f ′(x(i))��Â�³�°.½é 7�

0ò8,��³ù −f ′(x(i)) = b−Ax(i)�°.

��të����r©b£=��A�õ��¢°.¡ò(error) e(i) = x(i) − x�©¿Ùâv�#%

ýFqK�#õ#Í4�¯â�°.#"�(residual) r(i) = b−Ax(i)ùAÝ¢ b÷¿Ùâv�#ò��

�#õ #Í6°. r(i) = −Ae(i)�ù Ö3 N ½ �L,� #"�� @v ¡òõ ±µ Aõ �Ì � b® �ù

W�÷¿ ºÞ¢ ,÷¿ Ç ½ �°.Ì |Å¢ ,ù r(i) = −f ′(x(i))��� ,�6,#"�(residula)õ /Â

 ��ª��±�³Y�ù,÷¿f� �b¢°.üxÍ[C�t�³�ó��A�ú:Ì � 14<

�t f�¢°.� t¿ “#"�(residual)”��� Ìqõ �3 ü8 “/  � �³”��� �yõ C¤ý

êÀ�r©��.

x(0) = [−2,−2]T�t ;�¢°L �A �.�ü ½± � �zù / �(steepest descent)�³ú 0

�t�� 6 (a)���ìx\�q%�Fq�,�°.�,°üY��^¿ÏqE>úD£,�°.

x(1) = x(0) + αr(0). (9)

[C� � �³ú 0� v�# �±¢ Vú xØ£ , ��°. (�,qE αõ xØ£ , �õ @A �

,�°.)

copyright c©Jonathan Richard Shewchuk and VisCom Commune 8

Page 9: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 5] (a)j� AÙÒ ±µú æ¢ 2òé. (b) ü� AÙÒ ±µú æ¢ 2òé. (c) "�±µ(�ýL j� ÙAÙÒ).

GM� �±ú �#� xÛ� ©� �¦�°. (d)ÙAÙÒ ±µ� :. ©� K$ >�� :[� / �Y �ª EPê

�ª�:Ìü�M�°. 3òÙ�#��\�t�"�±µ�èK$?jú��½�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 9

Page 10: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 6]/ ��ª. (a)x(0) = [−2,−2]T�t;�©t f�/ ���³÷¿¢³GõD¢°. (b)�8�pòx

\� �� >� |�t fõ /�Ü � >ú ü�°. (c)� j]xù � 8(2òÍé j]8Y Ò_x �³� ½� e8)�

pò �x�°. (d)/9>�t��Ð���;³G��Ð�®�p¢°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 10

Page 11: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 7] Ò_ �x(ìx ÜUv)ú 0� � V�t �Ð� f ′õ Ã�sL �°. xÛæ¿ � �Ð�� � �è Ã�L

�°(>x ÜUv). �Ð� ¯â� f� �Ð�� �� �³ú #Í4L �ù Ò_xú 0� � :� ��ñú #Í6°.

Ò_x\�t�Ð��Ò_xY�p �V�t f�/�ú �°.

[�� 8]��,/ ���ªù [−2,−2]T �t;� L [2,−2]T�t½¶¢°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 11

Page 12: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

�xÒ_(line search)ù�xú0� fú/�Ü � αõxØ �YA�°.�� 6 (b)��YAúz<

¢°.½�e8Yj]8��#�pòxæ�¢>úxØ êÀCc¢°.�� 6 (c)���e8�pò

��©A�ýj]x�°.j]x�/9>�t αùv� �?

�Æ: y:Ûú �©, α� �³ ꥽(directional derivative) ddαf(x(1))� 0�: fõ /�Ü¢°. �

©ñ(chain rule)� �©, ddαf(x(1)) = f ′(x(1))T d

dαx(1) = f ′(x(1))T r(0) �°. � ½éú 0÷¿  L, r(0)®

f ′(x(1))���pü3 � α�xØüqb¢°(�� 6 (d)õÃ�).

� � ¯â� /�>�t t¿ �p� üqb  � �[: �î� �°.�� 7� Ò_xú 0�t °

j¢ >��t �Ð� ¯âõ Ã�sL �°. j]x(�� 6 (c))æ� ��� >�t ��� j]x �Ð�

(slope)��>�t� 2òÍé�����Ð�(gradient)õÒ_�xæ��°ú: (�� 7,>xÜU

v)f��¯â�£�®�°.�ý�¯â�ùÒ_xú0��: f���íúvÇ¢°. f��ý

�Ð�¯â�£�� 0 V-�Ð��Ò_x��p �V-�t/�ú �°.

αú@A �æ©, f ′(x(1)) = −r(1)��î� 8°üúuú½�°.

rT(1)r(0) = 0

(b−Ax(1))T r(0) = 0

(b−A(x(0) + αr(0)))T r(0) = 0

(b−A(x(0))T r(0) − α(Ar(0))T r(0) = 0

(b−Ax(0))T r(0) = α(Ar(0))T r(0)

rT(0)r(0) = αrT

(0)(Ar(0))

α =rT(0)r(0)

rT(0)Ar(0)

�õ?�Â�©Ã8,/ �(Steepest Descent)�ªù°üY�°.

r(i) = b−Ax(i), (10)

α(i) =rT(i)r(i)

rT(i)Ar(i)

, (11)

x(i+1) = x(i) + α(i)r(i). (12)

copyright c©Jonathan Richard Shewchuk and VisCom Commune 12

Page 13: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

��C��� 8�t®��½¶£:«�½±ý°.��&�E¿��î� �.������Ð�

��;��Ð��©�p �:[�#Í#�Ç\�°.

æ�s�¢NLý�ù¢¥��Ä:�°���±µ-¯âU�úÅu¢°./ �(Steepest De-

scent)�ª�GSüÌù±µ-¯âU��©_Íü�Ú,°±&ê��ÏÚ #�{\½��°.

½é 12�jº�X� −AúU L búÌ 8°üúu�°.

r(i+1) = r(i) − αAr(i). (13)

r(0)úGS �橽é 10ú¢¥GS �,ù��è�Å ��,��ó��Ä�t�½é 13õ

G�©t PÌ£ ½ �°.½é 11Y ½é 13?��t #Í#� ±µ-¯â U Arù ¢ ¥� GS 8 ý°.

�é�ú�Ì ��Äú£:f��[C>ù½é 13ú�©A�ü�½�� x(i) ÷¿ÙâqEü

#�(feedback)ê{�f�ý°�>�°.0�tÙô�½>�¤�¡ê��:÷¿ ¢AÝ¢ x�I¦

�����qE>�½¶£½�°�,�°.� ¢³>ù½é 10ú�Ì �s�:÷¿AÝ¢#"

�(residual)õ°èGS¥÷¿�©@£½�°.

/ �(Steepest Descent)�ª�½¶�¢Ûu�;�,� Û��Lî¯â�¢Ýì¢�©õ

£½�êÀ ®�£¿��²L¢°.

5 LLLîîî¯âââ(eigenvector)®®®LLLîîî(eigenvalue)���¢¢¢LLLööö

���xͽ¡¢Y@ú½�¢ó�,Lî¯â(eigenvector)®Lî(eigenvalue)�X| �êX�N

½�{}°.� Û�ú�òZ�P���õ�òZ�P®üæ °8,Lî �g� �,ú��L

[Cõ{��ªùC¡ò9��@��,��êÂ�X| � “�A÷¿” �© L��Mú,�°.Ý

± 3ê�Lî¯â®Lî�¢�[: �©�{°8,�ªEPê�ª�è�© �G£,�

°.� Û��cLî¯â¦Lî�¦ �,��©ÍL&& ú��L�°8�<ú&Clq

êý°.

Lî¯â�ù�Æ:÷¿Ûuêu¿tPÌý°;/ �(Steepest Descent)�ª�#�ªEPê�

ªùNLý�ú½± �æ©Lî¯âõGS£�Å�{°.

5.1 LLLîîî���gggLLLîîî?

±µ B� Lî¯â v� Bõ :Ì � ºÞú  �ê �Í� ºÜ� �q#� M� 0� I¨ ¯â�°(AÝ

 3�Â�³÷¿ºÞü�,ùC½). v�£��º $#��³��Â�þ½����,�÷¿î�

copyright c©Jonathan Richard Shewchuk and VisCom Commune 13

Page 14: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

�Mú,�°.°è©, Bv = λv qEße�\½ λ�U&¢°.�: λ� B�Lî�°.���

\½ α�©,¯â αvê�èô�¢Lî λõ �Lî¯â�°.�� B(αv) = αBv = λαv��:[

�°.°è©,Lî¯âõ¾�õ��$#v�ê�;&Lî¯â�°.

�(·êE�b£«?�Īù[[ BõqE¯â�G�©t�Ä:Ì �©õu¢°. B� #

� Lî¯â� �Ä:÷¿ :Ìþ :,°ü � �� �ÏÚ  #� \á� �f£ ½ �°. |λ| < 1 EÍ�

8, i�X¢Â¿�: Biv = λiv� 0¯â¿½¶£,�°(�� 9). |λ| > 1�8,Biv�X¢Â¿v�,�

°(10).�¥ B�:Ìþ:�°,¯â� |λ|��0�v�$#�I�3ý°.

[�� 9] v� −0.5�Lî�Âÿü� B�Lî¯â. i��� 8, Biv� 0�½¶¢°.

[�� 10]���t v�Lî 2õ���Lî¯â�°. i��� 8, Biv�X¢Â¿�S¢°.

B� Âa±µ EÍ([[ �©� Mù EÍ�ê �  °), B� xÍ: ë� n �� Lî¯âõ �

�°. �õ v1, v2, · · · , vn�� v� �. � �¦ù î� � M°. �� � Lî¯â� 0� I¨ :¾¢ \

½� ¾�� ºEþ ½ �� :[�°. ��� Lî¯â� �ê�3 Âÿü� Lîú ���Ú, �õ

λ1, λ2, · · · , λn¿ vè �.� Lî�ù sq� ±µ� © î� 3 A�ý°.Lîù t¿ �ù ú

��½ê�Lt¿°õ½ê�°.�õ�q,¨#±µ I�Lîù?� 1�L, 0¯â�I¨���¯

â� I�Lî¯â�°.

B� Lî¯â� I¨ ¯â� :Ìü8 qE �� �q(«?xͽú �©¥� �q �Í |Å¢ �

Á-�<�t�òXL� ���Á-ù #�¯âõ±ô�N²�°ô¯â�¦÷¿�s �,�°.

Lî¯â���¦ {vi}� Rn��9õ�Ø�\áúL²©Ã�(Âa±µ B�xÍë� n��Lî¯

copyright c©Jonathan Richard Shewchuk and VisCom Commune 14

Page 15: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

â�ú��°).��� n-òÙ¯â���Lî¯â��xÍS¦÷¿vÇþ½�°.Z¢±µY¯â�

UùÛ�ªY�:Ìüt¿���Lî¯â�U©�� B�îY�ë�:÷¿SP£½�°.

�� 11�t¯â x��Lî¯â v1Y v2�¦÷¿#Í#�°. Bõ x�U �,ù BõLî¯â�

� �� U � ¦ú u � ,Y �°. Bõ �Ä©t U© #�8, Bix = Biv1 + Biv2 = λi2v2� ý°.?

�Lî�¾�� 1ð�÷8, Bix� 0�½¶ú£,�°. (�� xúu� L��Lî¯â�?��

Bõ�Ä:÷¿U£EÍ 0�½¶ �:[�°).Lî�| #�ê 1ðÀú �°8, x�X¢

Â� �S£ ,�°.�,� �¿ ½X©u ¡��� ±µ� ßX!¤ �E(spectral radius)� |Å�ú Ù�

 ��î�°.±µ�ßX!¤�Eù°üY�°.

ρ(B) = max |λi|, λi ù B�Lî¯â

x� 0��ò3½¶ �õÙ 8 ρ(B)ù 1ð�Ib L,�  8�ù,�^°.

[�� 11]¯â v(ìx ÜUv)� Lî¯â�(>x ÜUv)� xÍ S¦÷¿ vÇþ ½ �°. �® �[ý Lî� λ1 =

0.7Y λ2 = −2 �°. B� �Ä:÷¿ :Ìü8 ¢ Lî¯â� 0� ½¶ L °ô  #� �S¢°. 0�t Bix �è

�S¢°.

xÍ ë� n �� Lî¯âõ ° ��� G � üÂa ±µw( )� U&¢°� Pìú s�£ �Å

� �°. ��ù ±µ�ú ÙT ±µ(defective matrix)�L ¢°. � �öù _<¢ xÍ Â½¡��� � ±

µ� © Ã�� ¾�¢ :�îú � #Í4L �°. \�¢ z<ù � [t�t °Ø�� CX Ä! �

�,ÙT ±µ(defective matrix)� ±ô "�ù ��Üý Lî¯â(generalized eigenvectors)® ��Üý Lî

(generalized eigenvalue)õ�Ì �Ûu£½�°.Bix� 0�½¶ �æ¢�Å?ÛS&÷¿?���

ÜýLî(generalized eigenvalues)�� 1ð�ùú�Kb¢°�ªYù�;&îî ��,�õ�

< �,ùÌÎq²×�°.

��t îÌ¢ Pìù °üY �°: j� AÙÒ ±µ� Lîù ?� j�°.� Pìù Lî� A

�õ�Ì �°üY���<£½�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 15

Page 16: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

Bv = λv

vTBv = λvT v.

j�AÙÒ(positive-definite)A��� �, vTBv�j�° (��I¨ v� �).0�t λ�èj�

½��{°.

5.2 bbb���üüü(Jacobi)���ÄÄĪªª

0��\½¶ �YAúI�,�Zu�P��t �õ��Ú�êÓ�ü�,ù]ÁI¨°.XÌ

îÌ¢ YAú U_Ã�: Ax = bú �� æ¢ b�ü(Jacobi)�ª�°.±µ A� � ÙÛ÷¿ Ûý(split)ý

°: A�Â��Ûú�¿ L°ô�Û�ù?� 0 D®Â�x�Ûù?� 0�L°ô�Û�ù A®

�ù E.0�t A = D + E�°.��ü�ªù°üY�°:

Ax = b

Dx = −Ex+ b

x = −D−1Ex+D−1b

x = Bx+ x, whereB = −D−1E, z = D−1b. (14)

D�Â�±µ�t¿Ö3�±µúu£½�°.�¨#éù°üY�ù>Üéú�©Ö3�Ä�ª

÷¿ºÞ£½�°.

x(i+1) = Bx(i) + z. (15)

sq� è� ¯â x0� ©,� ½éù �´� ¯â�ú f�¢°.Íý� � �Ä�ªú �© ��:÷

¿#Í#�¯â� # #��;�Yüp°ú:© x�Ì�«×��õ��°. x�½é 15�L

A>��LÙò�Ú,�� x(i) = x�: x(i+1)ê�è x®�ù�ü�:[�°.

��t��½éîêYA��Í��:÷¿Ã�,�°.Pì�îê���: ,�°.½é 14Âê

� x�¢¨#éúv�����q7½�°.PìAõ� ��¿Ûý �� 8 -�,æ��®°ô

D® Eú xØ ��  8 - �Íß-��Ý(Gauss-Seidel)�ª,9ò��°Üª(Successive Over-Relaxation,

SOR)#ú îê£ ½ �°.�: ��� �� Íý� xØ¢ Ûý �ª� ©¾ � B� �ù � ßX!¤

�E(spectral radius)õ �,�°.��t���`�\b��Ûý�ªúxØ �°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 16

Page 17: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

���qE¯â x(0)�tè�¢°L �.��Ä�©,Bú�¯â�:Ì¢�� zú�@Y�Ì

¢°.��ijG�tqE��§q���?

 #� ¯âõ Íý� �y � �© L �� °ô ¯â�� ¦÷¿ ��� ÙYú °è ¢ ¥ :Ì© Ã

�.�Äú�©#Í#� x(i)úAÝ¢© x®¡ò¨ e(i)�¦÷¿vÇ©Ã�.� 8½é 15�°üY�

�ý°.

x(i+1) = Bx(i) + z

= B(x+ e(i)) + z

= Bx+ z +Be(i)

= x+Be(i) ½é 14��©,

∴ e(i+1) = Be(i). (16)

��Äù x(i)� “AÝ¢ÙÛ”��³ús�M��(x�LA>��:[�°);�¥�Ä:�°¡ê

³ù�³ú��°.½é 16õ�©<�&N½��>ù ρ(B) < 1 EÍ� i�X¢Â¿?� 8¡ò

¨ e(i)� 0�½¶¢°�,�°.0�t#�¯â x(0)õqN3xØ �=�/[: ©��³úyX�

G¢°.

]Á x(0)õqN3xØ �=�0�sq�´Ì¡ò4�t x�½¶ �Ú��Å¢�Äì½��

�³úyX3ý°. ����³ùßX!¤�E ρ(B)��³�ü©ýCÑ|Å 6,ßX!¤�Eù

½¶��êõ@A¢°. vj� B�Lî¯â��ÏÚ�$ÀLîú��� (�, ρ(B) = λj )Lî¯â�

L �.Lî¯â��xÍS¦÷¿vÇý#�¡ò e(0)� vj��³÷¿�Ûú��°8,��Û��

$�ý3½¶£,�°.

B���:÷¿ÂaêI¦L(üÀ A�Âa���ê)Ý°;£�ê?ô°. ��b�ü(Jacobi)�

ª� ½¶�ê� ρ(B)� ¾3 _Íü6, � ù Z¢ Z¢ A� µ²�°. ݱ 3ê b�ü �ªù ?�

A�©½¶ �,ùI¦6,î�qj�AÙÒ(positivie-definite) A�©t꽶 �Mú½�

°.

5.3 uuu���:::���

�¢�]�úu�:÷¿Ã��æ©,���½é 4�#Í&�õ�qòL¢°.$9,LîYLî

¯âõü��ª��Å °.A���©Lî λõ������Lî¯â v�©°ü���¢°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 17

Page 18: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 12] A�Lî¯â� 2òÍé f(x)��©A�ýj]8�9�³Y�X¢°. ���Lî¯â��[ýLî�

��:Å�°. ���Lîù©¾ü�EP��7ôAê�ü»¢°.

Av = λv = λIv

(λI −A)v = 0.

Lî¯â� 0¯â�I¦°.0�t λI −A�"�±µ�qb¢°.0�t°ü���¢°.

det(λI −A) = 0.

λI − A�±µéú"�°¨é(characteristic polynomial)��Ùô°.�,ù λ�¢ nò°¨é÷¿,

�°¨é��(Y)�±µ A�Lî�ý°.½é 4��� A�"�°¨éù°üY�°.

det

λ− 3 −2

−2 λ− 6

= λ2 − 9λ+ 14 = (λ− 7)(λ− 2),

°¨é���Lî�t¿±µ�Lîù 7Y 2�°. λ = 7�:Lî¯â�°üY��u£½�

°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 18

Page 19: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

(λI −A)v =

4 −2

−2 1

v1v2

= 0

∴ 4v1 − 2v2 = 0.

��Aéú�T ����©��¿Lî¯â�°.�õ�q, v = [1, 2]T�Lî¯â�°.�ù�ª

÷¿Lî 2�©¾ �Lî¯â¿ [−2, 1]Tõu£½�°.�� 12�tǽ���®��,�Lî¯

â���¾¢ÍÙ�9Y�X¢°�,úǽ�°.Z¢Lî�¾8Ì�¢EPõ��°�,êÝ

£½�°. (ü½Lîù�� 5(b)® 5(d)�Ã��,Y��9ú0��ô£: f����¥ú�

y¢°.)

�Cb�ü(Jacobi)�ª��ô �?åúÃ�.½é 4���\½�ú�Ì �°üY�ù�Äé

úu�°.

x(i+1) = −

13 0

0 16

0 2

2 0

x(i) +

13 0

0 16

2

−8

=

0 − 23

− 13 0

x(i) +

23

− 43

B�Lî¯âõu 8Lî� −

√2/3�: [

√2, 1]T�L,Lî�

√2/3�: [−

√2, 1]T�°.��ù

�� 13(a)�t ��÷¿ #Í# �°.��ù A� Lî¯â® �X � M÷6,j]8� 9Yê \[� {

°.

�� 13(b)�b�ü(Jacobi)�ª�½¶�úÃ�sL�°.�NLý��½±ü8t0���ê�¢

E¿��Ä�ªú�©��:÷¿#Í#�¡ò¨������Lî¯â�Û��qN3ºÜ ��

U_È÷¿��©£½�°.(�� 13(c), (d), (e)).�� 13(f)�ÜU$vèõ�©Lî¯â�Û�ú�ý

L�°.���Ûù�� 11�tÃ��®�����Lî��©@Aý�꿽¶¢°.

����<ú�©� Û�� “Lî¯â”��,�¾ê��L�úÃ6���æ©p½����q

6h\¢L[êu�I¦�,�ÍîÌ¢êu��PìúÝì&NU÷8^9°.

6 ///   ���(Steepest Descent)���ªªª���½½½¶¶¶ÛÛÛuuu

6.1 ¢¢¢¥¥¥������©©©üüü���

/  � �ª� ½¶ú �© � æ© Íx e(i)� Lî λeõ ��� Lî¯â�L  �. � 8 #"

copyright c©Jonathan Richard Shewchuk and VisCom Commune 19

Page 20: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 13] b�ü �ª� ½¶�. (a) B� Lî¯â�� ��� LîY ¥Í #Í#�°. A� Lî¯â® µý ��

Lî¯â� Ã]8� 9� I¦°. (b)b�ü �ª� [−2,−2]T�t è� � [2,−2]T�t ½¶¢°. (c,d,e)¡ò ¯â

e(0), e(1), e(2)(ìx ÜUv)® ��� Lî¯â �Û(>x ÜUv). (f) ÜU$ù �ü 4� ¡ò ¯â� Lî¯â �Ûú

vÇ¢°. ¡òõvÇ ����Lî¯â�Ûù���Lî�0��\£½���ê¿ 0�½¶¢°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 20

Page 21: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

�(residual)r(i) = −Ae(i) = −λee(i)ê�èLî¯â�°.é 12õ�Ì �°üúuú½�°.

e(i+1) = e(i) +rT(i)r(i)

rT(i)Ar(i)

r(i)

= e(i) +rT(i)r(i)

λerT(i)Ar(i)

(−λee(i))

= 0.

�� 14� � �ª� · ¢ ¥�� ©� êµ ��õ Ã�u°.> x(i)� ÍÙ� 9 �ÏÚ  #� n�

�°.0�t#"��ÍÙ�|îú�¿�ýÅ3ý°. α(i) = λ−1e õxØ¥÷¿���: ½¶úuú

½�°.

[�� 14] /  �(Steepest Descent)�ªù ¡ò¨�  #� Lî¯â¿ vÇþ EÍ ¢¥� �Ä�÷¿ AÝ¢ ©�

½¶¢°.

ÌÎ��: Ûuúæ©, e(i)õLî¯â�xÍS¦÷¿vÇ©b 6,Z¢�Lî¯â��A�

�p(orthonormal)�qb¢°.ÙÀ C2�t±µ A�Âa�EÍ n���pLî¯â�U&¥ú�<¢

°.Lî¯â����¾�¿ºE£½��:[����Lî¯âõA�Ü£½�°.�©3Lî¯â

õxØ 8°üY�ùîÌ¢"�ú �°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 21

Page 22: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

vTj vk =

1, j = k,

0, j 6= k.(17)

¡ò¨úLî¯â�xÍS¦÷¿vÇ �.

e(i) =n∑

j=1

ξjvj , (18)

�: ξj� e(i)��Û»£��°.é 17Yé 18¿Ùâ°ü�¨#éúu�°:

r(i) = −Ae(i) = −∑

j

ξjvj , (19)

||e(i)||2 = eT(i)e(i) =

∑j

ξ2j , (20)

eT(i)Ae(i) = (

∑j

ξjvTj )(∑

j

ξjλjvj)

=∑

j

ξ2jλj , (21)

||r(i)||2 = rT(i)r(i) =

∑j

ξ2jλ2j , (22)

rT(i)Ar(i) =

∑j

ξ2jλ3j , (23)

é 19� r(i) �èLî¯â�Û�¦÷¿vÇþ½�üúÃ�s6,���Û�£�� −ξjλj�°.é

20Y 22��¿�ÍL�ß(Pythagoras)�ªY�°.

�CÛuú©Ã�.é 12�t°üúuú½�°.

e(i+1) = e(i) +rT(i)r(i)

rT(i)Ar(i)

r(i)

= e(i) +

∑j ξ

2jλ

2j∑

j ξ2jλ

3j

r(i) (24)

�����tÍý� e(i)� #�Lî¯â�Û�ú��EÍ α(i) = λ−1e õxØ¥÷¿�¢¥��

©�½¶¥úÃU°.�C e(i)�������?�Lî¯â�ô�¢LîX λõ���EÍõU_

Ã�.é 24õ�Ì �°üúu�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 22

Page 23: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 15]/ �ùLî�?��°8�¥«�Ä�tAÝ¢©�½¶¢°.

e(i+1) = e(i) +λ2∑

j ξ2j

λ3∑

j ξ2j

(−λe(i))

= 0

�� 15���ª�è��:÷¿©�½¶¥úÃ�L�°.?�Lî��ô� �:[�ÍÙù

uÍ(spherical)�üqqEæX�tè� �[G{�#"�(residual)�u�|îú�ýÅ3ý°. αõ

xØ �,ùX�t®�ô��¿ α(i) = λ−1¿xØ¢°.

� #,=��t¿°ôLî�U&£EÍqEé÷¿ α(i)õxØ Ì�êLî¯â�Û�ú?

�C$£½�{°.0�t�:xØ£½���ªù�[�ÍÊ�ý°.Pì,é 24�#Í#��Û½

� λ−1j �©�|Xe�÷¿Ç½�°.�|X ξ2j� e(i)��Û��ÏÚÀ�Û�Íx¿æõ�3

¢°.@Y:÷¿��Ä:�° e(i)�¥ù�Û��ÏÚ�Ù�ìC¿£���q#�ꢰ (üÀ�

Ù&�I¦��).�¢�î¿/ ��ªY�ªEPê�ªù N(rougher)�LÙô°.�¿��

ü�ªùßXÌ(smoother) Ú,��î�?�Lî¯â�Û����Ä:�°vq��:[�°./Â

 �Y�ªEPê�½¡[¶��Ù�t[[ßXâ¿�G éüL���ìC¿�ßXÌ�I¦°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 23

Page 24: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 16]����¯â�ô�¢�C�i(norm)ú �°.

6.2 ������½½½¶¶¶���

��: EÍ�©/ ��ª�½¶�ú�b� ²8�C�i(energy norm)||e||A = (eTAe)1/2ú

A�©b¢° (�� 16úÃ�).�i(norm)ùîÁý�i(Euclidean norm)ð°Ø��ÑÍ6qE8�

t�ÌÎ��ß Ïi(norm)�°;é 8úU_Ã8 ||e||Aõ/�Ü �,�@v f(x(i))õ/� �,�

úN½�°.�i(norm)ú�Ì �°üúuú½�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 24

Page 25: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

||e(i+1)||2A = eT(i+1)Aei+1

= (eT(i) + α(i)r

T(i))A(e(i) + α(i)r(i)) (½é 12��©)

= eT(i)Ae(i) + 2α(i)r

T(i)Ae(i) + α2

(i)rT(i)Ar(i) (½é 14��©)

= ||e(i)||2A + 2rT(i)r(i)

rT(i)Ar(i)

(−rT

(i)r(i)

)+

(rT(i)r(i)

rT(i)Ar(i)

)2

rT(i)Ar(i)

= ||e(i)||2A −(rT

(i)r(i))2

rT(i)Ar(i)

= ||e(i)||2A

(1−

(rT(i)r(i))

2

(rT(i)Ar(i))(e

T(i)Ae(i))

)

= ||e(i)||2A

(1−

(∑

j ξ2jλ

2j )

2

(∑

j ξ2jλ

3j )(∑

j ξ2jλj)

)(¨#é 21, 22, 23��©)

= ||e(i)||2Aω2, ω2 = 1−(∑

j ξ2jλ

2j )

2

(∑

j ξ2jλ

3j )(∑

j ξ2jλj) (25)

Ûuù ω�\¢úü�,�µ²�°.�|X®Lî�qN3½¶��³úyX��õÃ��æ©

n = 2 EÍ�©@Yõîê£,�°. λ1 ≥ λ2�L �.±µ A�ßX!¤S&½(spectral condition

number)� κ = λ1/λ2 ≥ 1¿A�ý°. e(i)��Ð� (Lî¯â��©A�ü�_vG�©)�è�>�

�U:�6 µ = ξ2/ξ1�ý°.�C°üéúu�°.

ω2 = 1− (ξ21λ21 + ξ22λ

22)

2

(ξ21λ1 + ξ22λ2)(ξ21λ31 + ξ22λ

32)

= 1− (κ2 + µ2)2

(κ+ µ2)(κ3 + µ2) (26)

ω�ù/ ��ª�½¶�êõ@A 6 µ® κ�¢¥½¿�� 17Y���²�°.���

�¿Xtà ����õÝ £½�°.�c e(0)�Lî¯â�8�Ð� µ� 0�ü6 (ÓùX¢),�:

����t ω� 0�ÿúǽ�°.0�t½¶ù��:÷¿�Øq�°.Z¢Lî�?�ô� °8

S&½ κ� 1�ü6,�:�è ω� 0�ý°.

�� 18ù �� 17� P ��� ��� ��� ©¾ � �õ Ã�L �°.�� �ò Íé�ù Lî¯â

��©A�ü�_vG�t�²P°.�� 18(a)® 18(b)�ÀS&½õ����°./ �ùè�>�

Ï� ^3 xØü}°8 �ò3 ½¶¢° (�� 18(a)).� # ��:÷¿ κ� À ú ��8 �Í #D �

 úà ° (�� 18(b)).�¥«��ùÀS&½�·#D�õ�$�[:÷¿Ã�L�°: f(x)�u

îÍ×õ��6/ ��ªù�uî�jÅúµ°�° 6$��± �G �\×�ý°.��

copyright c©Jonathan Richard Shewchuk and VisCom Commune 25

Page 26: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

18(c)® 18(d)�t S& ½� �ù �t¿ �ò Íéù $� uÍ� ý°.�: ½¶ù è�>� [G {�

�ò3�Øq�°.

[�� 17] /  �(Steepest Descent)�ª� ½¶ê ω. ½¶ê ω� µ (¡ò¨ e(i)� �Ð�)Y κ (A� S& ½)�

¥½�°. ½¶ù µ® κ��ù�:�ò°. LAý±µ�t µ = ±κ�:/J�½¶úà °.

κõ\½¿£:(A�LAüq�÷t¿),�³¢y:Ûú�©é 26� µ = ±κ�:/Â�ÿúN½

�°.�� 17�t�x��©��q�� y¢õ�xúǽ�°.�� 19�Íý�PÌ©µÏ±µ

A�©/J�è�>ú�ýL�°.�è�>�ù ξ2/ξ1 = ±κ¿A�ü�xæ�n °. ω�\¢(/

J�è�>�©¾)ù µ2 = κ2¿zA¥÷¿�üú½�°.

ω2 ≤ 1− 4κ4

κ5 + 2κ4 + κ3

=κ5 − 2κ4 + κ3

κ5 + 2κ4 + κ3

=(κ− 1)2

(κ+ 1)2

ω ≤ κ− 1κ+ 1

.(27)

é 27�Ù#é��� 20��²K�°.±µ�#D\×��ú½À(�,S&½ κ�Á½À),/ 

��½¶�ê�ÌÎ�²�°.< 9.2�tÂa�6j�AÙÒ ±µ�°üY�ùS&½õ��°8

é 27� n > 2 EÍ�ê�;&��¢°�,ú�<¢°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 26

Page 27: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 18]� P �C� �� 17� P � ���� ©¾ � �> ��õ vÇ¢°. (a)� À κ,�ù µ. (b)#D ½¶� �.

κ® µ?�¾°. (c)�ù κ® µ. (d)�ù κ®À µ.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 27

Page 28: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 19] ìx�ù /  � �ª�t /J� ½¶ú Ã�� è�>ú vÇ L �°. >xù ½¶ YA�t $X�

³G�úÃ�L�°. �è��/J��>�t�Øq�EÍ,�°ü³G�è/J��y�t�Øq�°. ���

³G�tj]89(ç_ÜUv)õAÝ& 45ê¿pò 3ý°. ��t κ� 3.5�°.

κ = λmax/λmin,

��O/ÂLîY/�Lî�ü(ratio)�°./ ��ª�½¶@Y�°üY�°.

||e(i)||2 ≤(κ− 1κ+ 1

)i

||e(0)||A, �ýL (28)

f(x(i))− f(x)f(x(0))− f(x)

=12e

T(i)Ae(i)

12e

T(0)Ae(0)

(½é 8��©)

=(κ− 1κ+ 1

)2i

.

7 ���ªªª���³³³���ªªª

7.1 ���ªªª���

/  �(Steepest Descent)�ªù [[ �� 8Y �� �;� �y Ò_°Ï �³ú °è Ò_¢°. �¢

copyright c©Jonathan Richard Shewchuk and VisCom Commune 28

Page 29: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 20]/ ��½¶�ù±µ�S&½(condition number)���¥�0�JÜý°.

Ú �©3 �ù �³� Ò_ú #�qt �  ¥ ½±£ ,� I¦�  #� Ò_ �³� © �b£ $

ýõ ¢ ¥� �¿ � ½ �°8 Ì -� Mú«?I�%q� °üY �°: Íx t¿ �p � Ò_ �³

d(0), d(1), · · · , d(n−1)��¦úP�Ã�.�Ò_�³�©tÍý�¢¥�³G¿O$ x®�$�«Ï

³G«��êÀ �,�°.�:�Å¢�±$ýõN½�°8,?�Ò_�³�© 1¥�Ò_�½

± 8üL,@v n³Gó�©�êµ£,�°.

�� 21ù_v9úÒ_�³÷¿PÌ �z< L�°.� (½e)³G�t�AÝ¢ x1-_v®�X

£:«�Ò_ú�± 6,�¥« (½�)³G�t�Ù �©úu3ý°. e(1)� d(0)®½��ús@©

Ã�.�õ��Ü 8,�³G�t°üY��æXõxØ �,�°.

x(i+1) = x(i) + α(i)d(i) (29)

¢¥Ò_¢ d(i) �³÷¿�X÷¿Ì�\Ò_��Å{êÀ �æ©, e(e+1)� d(i)�t¿�p©b

¢°�Pìú�Ì � α(i) úu¢°.�¢S&ú�Ì �°üéúuú½�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 29

Page 30: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 21]�p�³�ª. IÖ3ê��ªùÍý��y»úNL�úEÍ���Ì£½�°

dT(i)e(i+1) = 0

dT(i)(e(i) + α(i)d(i)) = 0 (½é 29�� �)

α(i) = −dT(i)e(i)

dT(i)d(i)

(30)

ݱ 3ê��ªùPì8?{��ª�°.��ªúPÌ � α(i)õGS ²8 e(i)õNIb� 

�Ú, e(i)õK°�,ù�y©õNL�°�,�t¿[C����y�²��\× ,�°.

�¢[C>ú©@ ²8,�p �Ò_�³�I¦� A-�p Ò_�³úPÌ �b¢°.�¯â

d(i) ® d(j)�t¿ A-�pZ��ª(conjugate)�ü²8°üS&ú�T©b¢°.

dT(i)Ad(i) = 0

�� 22 (a)� A-�p¯â�qN3Ã���õÃ�sL�°.����xÈæ�;³üq�°L\

\©Ã�.�� � Û� �� 22 (a)� j�ú ä!It ÍÙ� Ù�¤ Ã�:«� !I¾D°L f�© Ã

�.� 8¯â���� 22 (b)�¤�p �,�¤Ã�,�°

^¿ÏÅuP¨ù e(i+1)® d(i)�t¿ A-�p(�� 23 (b))õ�Øqb¢°�,�°.��pS&ù/

copyright c©Jonathan Richard Shewchuk and VisCom Commune 30

Page 31: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 22]�� (b)¯â�û���p t¿�� (a)¯â�ûù A-�p�°.

 ��ªY�ô��¿*_�³ d(i)ú0�t/�>úü�,Yô� °.�õÝ  �æ©,�³

꥽õ 0÷¿�LI�®���qÃ�.

d

dαf(x(i+1)) = 0

f ′(x(i+1))T d

dαx(i+1) = 0

−rT(i+1)d(i) = 0

dT(i)Ae(i) = 0

é 30�îê�ªú�Ì �,*_�³��t¿ A-�p�üêÀ � α(i)�°üY��vÇý°.

α(i) = −dT(i)Ae(i)

dT(i)Ad(i)

(31)

=dT(i)r(i)

dT(i)Ad(i)

(32)

½é 30Y�µý�éùGS��  °.�é�t*_¯â�#"�(residual) â®ô� °8,�

éù@v/ ��ª�éY�ù,�ý°�>�s@ �(½é 11ú÷L è¡).

��ª� n��³G4� xõGS£½�°�,ú�< �æ©,¡ò¨(error term)ú*_�³�

�xÍS¦÷¿°üY��vÇ©Ã�.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 31

Page 32: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 23]�ª �³ �ªù n�� �Ä 4� ½¶¢°. (a)� ¥« ³G�t� qE �³ d(0)õ 0� ½± ý°. e(1)�

��è d(0)� A-�p©b¢°�C¢S&ú�Ì �/�> x(i)�xØý°. (b)#�¡ò e(0)�ç_ÜUv¿vèý

A-�p �Û� ¦÷¿ vÇ£ ½ �°. �ª �³ �ª� � �Ä ³G� ¢ ¥ ½±þ :�° �� �Û |�t  #�

�Û�ò»¿C$ý°.

e(0) =n−1∑j=0

δjd(j) (33)

� : δj ù �³¢ ½¡: �põ �© uú ½ �°.*_ �³�ù \Ò A-�p�t¿, dT(k)Aõ ½é

33�jº����X�U 8 #�Ò_�³úC½¢°ô?�Ò_�³�Âÿ � δj �úC$

£½�°:

dT(k)Ae(0) =

∑j

δjdT(k)Ad(j)

dT(k)Ae(0) = δ(k)d

T(k)Ad(k) (d¯â� A-�p��� �uq�)

δ(k) =dT(k)Ae(0)

dT(k)Ad(k)

=dT(k)A(e(0) +

∑k−1i=0 α(i)d(i))

dT(k)Ad(k)

(d¯â� A �p��� �uq�)

=dT(k)Ae(k)

dT(k)Ad(k)

(½é 29�� �îê ) (34)

copyright c©Jonathan Richard Shewchuk and VisCom Commune 32

Page 33: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

½é 31Y 34�� �,Íý� α(i) = −δ(i)�úN½�°.�Pìù¡ò¨�¢^¿Ï[>úCW

¢°.°ü½é�tÃ��®��, xõu� ��Û # #õüI#��YAù@v¡ò¨ú�Û

»¿ #BC$©#��,÷¿Ç½�°.(�� 23(b)õÃ�)

e(i) = e(0) +i−1∑j=0

α(j)d(j)

=n−1∑j=0

δ(j)d(j) −i−1∑j=0

δ(j)d(j)

=n−1∑j=i

δ(j)d(j) (35)

n¥��Ä��#8,?��Û�C$üq e(n) = 0�ý°;�<�.

7.2 ������-ÙÙÙyyy!!!(Gram-Schmidt)���ªªª���

�C A-�p*_�³÷¿�Øq��¦ {d(j)}úu �� 8ý°.°±ß¥3ê�õ�³ 3f� 

��ª���Ú��ª��¿�ª��-Ùy!YA(conjugate Gram-Schmidt process)�°.

n��xÍë�¯â u0, u1, · · · , un−1õ��L�°L �.üÀð� : xØ��  �� 

�� _v 9ú � n �� xÍ ë� ¯â¿ 8 ½ �ú ,�°.Ò_ �³ d(i)õ ��q 4� æ© Íx uiõ

��L£�,��t�;�?�Ò_�³¯â(�� 24õÇ,)® A-�p �M�?��Ûú�°.°

�L#t+I��¯â��¿�;�?�Ò_�³Y A-�p Ò_�³ d(i)�ý°.°ô¿ �8

d(0) = u0¿�L, i > 0 ?�EÍ�©°üY��Ò_�³¯âõu£½�°.

d(i) = ui +i−1∑k=0

βikd(k) (36)

�: βik� i > k EÍ� �A�ý°.��úu �æ �, δjõu£:PÌ¢,Yô�¢�ª

úPÌ£½�°.

dT(i)Ad(j) = uT

i Ad(j) +i−1∑k=0

βikdT(k)Ad(j)

0 = uTi Ad(j) + βijd

T(j)Ad(j), i > j (d¯â�P�� A-�p��� �)

βij = −uT

i Ad(j)

dT(j)Ad(j)

(37)

copyright c©Jonathan Richard Shewchuk and VisCom Commune 33

Page 34: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

�ª�³ (conjugate direction)�ª���-Ùy!�ªÜ (Gram-Schmidt conjugation)úPÌ£:�[

C>ù ^¿Ï Ò_ �³ú @A � æ© �;� ?� Ò_ �³ ¯â�ú -?ý� ��L �qb ¢°�

,�6,Ìu#;�Ò_�³ú@A �æ©t�?� O(n3)��S��Å °�,�°.Pì³æ£

��9¯âõ�Ö÷¿�ªÜ(conjugation)õ½± �Ò_�³�ú@A �°8,�ª�³�ªù@

v�Íß�$ª(�� 25)Yô�¢,�ý°.@Y:÷¿�ª�³�ªù�ªEPê�ª� -�ªEP

ê�ª�è�ª�³�ª�¢[ê�°-#Í#�¢�¢[Cõ©@£:«�$�PÌü�MU°.

�ª�³�ªú�© �Ú�ªî: Å�� (�ªEPê�ªê�ô���°)�� 25��� 21õ

� ,� ÝY °� ,ú NIòý� ,�°.�ª �³ �ªú ½±£ :(�ª EPê �ªú j¥ �),

��ª�Pì�q& (Óù¾��ºÍý)W��tÇ:�p�³ (orthogonal direction)�ª÷¿[Cõ

�L��,Y�°�,ú�r �.

[�� 24] � ¯â� ��-Ùy! �ªÜ(Gram-Schmidt conjugate).� �� xÍ ë� ¯â u0® u1 ¯â�t è�¢°.

d(0) = u(0)¿ zA¢°. ¯â u1ù d0� A-�p(Z� conjugate) u∗® d0� e± u+� �Û÷¿ Û©þ ½ �°.

�ªÜ�ó A-�pÙÛ��+3üq d1 = u∗�ý°

7.3 ¡¡¡òòò¨���///:::���

�ª�³�ªù°üY�ù&y��"���°:��ªù�¥�³G�tÒ_� Ìý©æ4�t

/x�©õu¢°.Ò_�´Ìý���q%�«? span{d(0), d(1), · · · , d(i−1)}�Ò_¯â d(0), d(1), · · · ,

d(i−1)��©f�ü� i-òÙÙÛW��6�õ Di�LvÇ �. e(i) ù e(0) + Di¿ÙâxØþ½�

°.���t � “/x�©”��3Xáy�«?�ª�³�ª�/x�©õü�°�,ù��ª

� e(0) +Di�ÙÛW�4�túxØ£: ||e(i)||A�/��ü�úxØ¢°�,�°(�� 26ú

Ã�).Pì qE ¡��ù ÙÛ W� e(0) + Di 4�t ||e(i)||A ú /�Ü¥÷¿� �ª EPê (conjugate

gradient)îê �ꢰ.

ô�¢ �ª÷¿ ¡ò¨ �è *_ �³� xÍ S¦÷¿ vÇþ ½ �÷6(½é 35),¡ò¨� �C�

i(norm)ù°üY�ù¦SvÇ÷¿#Í7½�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 34

Page 35: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 25]9�³³æ¯âõPÌ¢�ª�³�ªù�Íß�$ª(Gauss Elimination)÷¿êN²K�°.

||e(i)||A =n−1∑j=i

n−1∑k=i

δ(j)δ(k)dT(j)Ad(k) (½é 35��©îê)

=n−1∑j=i

δ2(j)dT(j)Ad(j) (d¯â� A-�p��� �îê)

�¦S(summation)vÇé���?�¨ùI�PÌü�MùÒ_�³���[üq�°. e(0) +Di

ÙÛW��txØý���°ô¯â e��¦SvÇéúÝ$°ú:#Í#�,Yô�¢¨�ú�K

b 6,��@v e(i)���è/��C�i(norm)ú��°�,ú�<¢°.

½éú �© /:�ú �< �÷¦, �E �[:÷¿ �õ �©£ ½ �êÀ ©Ã�. �ª �³ �ª

� �z YAú è�:÷¿ vÇ£ ½ �� /x� �ªù �� 22® �� Íý� �z � W�Y “�q

&(stretched)”W�út¿üp �,�°.�� 27 (a)® 27 (c)��ª�³�ª� R2 W�Y R3W��t

qN3 ô� ��õ Ã�sL �°; � ���t t¿ ½�÷¿ Ã�� x�ù t¿ �p(orthogonal)¢°.

�8�,�� 27(b)® 27(d)�t�ô�¢��úLî¯â9ú0��²ÍÙÍ#Lx�uÍ(spherical)�

üêÀ¢,�°.����t½�÷¿Ã��x�ùt¿ A-�p�°.

�� 27(a)úÃ8�ª�³�ª�#�8AX x(0)�tè� 6,Ò_�³ d(0)õ�Ì �¢³G�

ô �°ü8AX x(1)>�t'8L�°.�:^¿Ï¡ò¯â e(1)�Ò_�³ d(0)Y A-�p�°.�

copyright c©Jonathan Richard Shewchuk and VisCom Commune 35

Page 36: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 26]����tq�3\©���� e(0) + D2 = e(0) + span{d(0), d(1)}�°. ÍÙù�C�i(norm)��A¢

ú �#Lx�°. �³G�½±ý�,�ª�³�ªù e(0) + D2 \�t ||e(i)||A �/��ü�> e(2)õü3

ý°.

©3üù x(1)�qN3ÙÛW� x(0) +D1�t/�>���,úÃ$£½�ú«?»ù�� 27(b)�#

Í#�°: e(1)® d(0)�t¿ A-�p��:[���Ã���q&W�(stretched space)�t�¯â�½

�ú�ØL�°.� ¯â e(1)ùô�¢�C�i(norm)�¡òõ � (�,ô�¢ ||e||Aõ���)#L

x�úvÇ �ôîÙ����ö�°.0�tÙÛW� x(0) +D1��¿ x1�n���ôîÙ-�, x(1)Y

ô�¢ �C� i(norm)� ¡òõ � #Lx-� ? � ?e8� ü6 �: � ?e8Y #Lx� ? �

�>��¿ x(1)�°.

�,ù Pì �ý g��¢ �ù I¦°; �y 7.1<�t U_Æ �® ��, Ò_ �³Y ¡ò¨� t¿

A-�ª��Pì�@vÒ_�³ú0� fõ/�ÜèÅ�,(0�t ||e||Aõ/�ÜèÅ�,)Yô� 

°.� #�ª�³�ª��¥«³Gõ½± ��¥«Ò_�³ d(1) �³÷¿ ||e||Aú½±¢��

ê�;& ||e||A��;�PÌ°ÏÒ_�³ d(0) �³�t/���î�ý°�,úqN3Ã$£½�

ú«? i¥��Äú½± L#8,qN3 f(x(i))�ÙÛW� x(0) +Di ;��t/�Üþ½��,�«?

d(0)® d(1)�t¿ A-�p��:[��� 27(b)�t½�÷¿�&°.��tÒ_�³ d(1)�© xõ³

¢°,ù<�¢Ú,��î� d(0) ¯â� xõ|î÷¿ L x(1)õ�#�Ù�© x(1)�t? �:[

�°.XòÙ�C��õÌÎ�Ã�u°.�� 27(c)® 27(d)����|î��ù���ÍÙý��²

K �°. x(1)� ½Ù ÍÙ� æ� n� �L, x2)� 4Ù ÍÙ�� n� �°.�� ��ú �î& U_ Ã�:

e8 x(0) +D2�ÀÍÙú�Y 6�òL�÷6,�e8ù4ÙÍÙY x(2)�t? �e8�ý°.©

x��e8I��ÍÙ��|î��°.

�� 27(c)õÃ8Xt��[ú°èL\½�°.��� ÛY4� x(1) æX�t��ÚÙÛW�

copyright c©Jonathan Richard Shewchuk and VisCom Commune 36

Page 37: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 27] �ª �³ �ª� /:�. (a) �òÙ [C. ½�÷¿ �#� x�ù t¿ �p(orthogonal)¢°. (b)

“�q&(stretched)”W��t vÇý ô�¢ [C. ��t ½�÷¿ �#� x�ù t¿ A-�p�°. (c) XòÙ [C,

xõ |î÷¿  � � �� ôî ÍÙ�� Ã�L �°. �x x(0) + D1ù ½Ù ÍÙ�� © x(1)>�t ?¢°. e8

x(0) +D2�4ÙÍÙ�© x(2)>�t?¢°. (d)XòÙ[Cõ�q&W�(stretched space)÷¿¦Dú:?å.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 37

Page 38: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

x(0) +D2\ú(q°¦8t¡ò�i(norm)||e||õ/�¿£½��V÷¿�ô ²L¢°L�A �;

�¢ÚÍý�°©½��VùÒ_�³ d(1)�t«q(½�{°.�c�Ò_�³ d(1)�/�>ú�

¿�ýÅL�°8Íý�@võµ�£½�°.�Ò_¯â d(1)�/�>úO$�ýÅL�°LÃ$

£½���î����?

�� 27(d)��»úÃ�u°. d(1)� d(0)� � A-�p�t¿,����tt¿½�÷¿�#L�

°.�C� Û�e8 x(0) + D2��e8�¢$�[� ,�¤f� LI�õ�°Ã�;�:Ã3

ü�$8ù�� 27(b)®ô�£,�°.> x(2)ù[��|î�þ,�L,> x�[��I�Å��3ü

�Ú,�æX� x(2)�t½�÷¿O$I�¿4²�æX�þ,�°. d(1)Y d(0)�½�÷¿�#L�÷

t¿,Ò_�³ d(1)�AÝ& x(2)õ³ 3ü6,� x(2)�e8 x(0) + D2 \���>��ÏÚ© x��

$�«Ï>�þ,�°.e8 x(0) +D2� x(2)�n���u�? �e8�°.��t�¥«³Gõ½

± 3ü8,D2® A-�p ��³÷¿Ó�� x(2)�t© x¿O$4²�3þ,�°.

�� 27(d)�t Xá �� §q���õ �© � Z °ô �ªù � Û �ê� © x� tt ÙÛ W�

x(0) +Di�t�Ó��êÀC¢üq��uâú�÷¿!I¾�L�°L\\ �,�°.�³G�°

Ý$ �ÙÛW�(expanding subspace)D�òÙ� #B�q#L,�:�°uâùS�BÌ� Û�

«�°�¤½�êÀ�î¿×�,�°.�CW�úé� r²�� 27(b)�¤Ã�3��8�,��

¿�ª�³�ª�ý°.

�ª �³ �ª� Z°ô |Å¢ "�� �� ��� #Í# �°. Íý� � ¥� ½± ³G�t #e

8(hyperplane)x(0) +Di� x(i)�n���ÍÙ�? 3ý°�,úU_ÃU°. 4<�z<¢�®��

���>�t���#"�(residual)���>ú�#�ÍÙ�v8Y�p¢°�Pìú�r �.�,

ù@v r(i)� Di®�p¢°�,ú�y¢°.�õ½¡:÷¿Ã��æ©é 35� −dT(i)Aõ���jº

X�U©Ã�.

−dT(i)Ae(i) = −

n−1∑j=i

δ(j)dT(i)Ad(j) (38)

dT(i)r(j) = 0, i < j (d- â� A-�p��� �.) (39)

�¨#éù°ô�ª÷¿îꣽê�°.Ò_�³�©¢¥�³Gõ½± L#8X÷¿Ì�

\ù��³÷¿Ò_£�Å�{°�Pìú\� �;¡ò¨ù�;�?�Ò_�³�©¨\ A-�

p�6, r(i) = −Ae(i)�t¿#"�(residual)��;�Ò_�³YsC#�p�ý°.

Ò_�³� u¯â�ú�Ì ���qP÷t¿, u0, · · · , ui−1�f� �ÙÛW���¿ Di�6,#

"�(residual)r(i)�� ¢�; u¯â®�p¢° (�� 28úÃ�).�,ù½é 36Y r(j)�4:úD �

�<£½�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 38

Page 39: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

dT(i)r(j) = uT

i r(j) +i−1∑k=0

βijdT(k)r(j) (40)

0 = uTi r(j), i < j (½é 39�� �) (41)

X÷¿ PÌþ ¨#é�  # Ì ��Ú.� ¨#éù ½é 40(�ýL �� 28)÷¿ Ùâ uú ½ �� °

üY�ùé�°.

dT(i)r(i) = uT

i r(i). (42)

[�� 28] Ò_ �³ d(0)® d(1)� ¯â u0, u1õ �Ì � ��qP÷t¿, ��ù ô�¢ ÙÛW� D2(ç_ ��)õ

f�¢°. ¡ò¨ e(2)� D2� A-�p�6, #"�(residual)r(2)� D2® �p�°. °ü �Äú æ« ^¿Ï Ò_ �³

d(2)ù��«��Ò_�³¯â�¿f�ü�ÙÛW�D2�A-�p êÀ (u2õ�Ì �)��q�°. d(2)õf�£

: u2õ��L��-Ùy!�ªÜõ:Ì �f� �:[�, d(2)® u2��>�ù D2�e±¢e8�n��°.

� <ú ��÷8t ¢ �� Ì s�£ ,ù /  � �ªú PÌ£ :® �ô��¿ �ª �³ �ª

�è ±µ-¯â U � �Sú � �Ä è��° ¢ ¥�  êÀ £ ½ �°� >�°. �õ æ©t� #"

�(residual)úGS£:°üY�ù>ÜéúPÌ �b¢°.

r(i+1) = −Ae(i+1)

= −A(e(i) + α(i)d(i)

= r(i) − α(i)Ad(i) (43)

8 ���ªªªEEEPPPêêê(conjugate gradient)���ªªª

�ª EPê �ªú °Ø� ��t �ª EPê �ª� ¢ 4Ì� ��«�ê s�ü� M� ,� �\£

,���,�Å¢?�$X��C @P°.Pì�ªEPê�ª���,ù�ª�³�ªYô�¢Ú,

³�Ò_�³ú��:#"�(residual)õ�ªÜ ���°�,�° (�, ui = r(i)¿zA¢°).

copyright c©Jonathan Richard Shewchuk and VisCom Commune 39

Page 40: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

� ¢ xØ �ªù �² �� �î¿ ¦ý:�°.Íx /  � �ª�t #"�(residual)ú PÌ �

,� � ô�°÷¦ �ª EPê �ª�t� PÌ � G£ �î� {°.�«,#"�(residual)ù �;� Ò

_ ¯â® �p¢°� ^ù "�ú ��L �°(½é 39). 0�t #"�(residual)¯â� 0¯â� I¦�8

¨\�;�Ò_�³�xÍë� ^Ò_�³úf�£½��°.#"�¯â� 0¯â EÍ���

�¿©�uq�EÍ�t¿[C�ü�M�°.#|�Ã3ü9��,#"�(residual)úPÌ �Ú��

ÌΦý: �î�U&¢°.

#"� ¯âõ PÌ � ,� qE �yõ ���� f�© Ã�.Ò_ ¯â�� #"�(residual)ú �Ì

 ���q��:[�,#"�¯â��f� �ÙÛW�,� span{r(0), r(1), · · · , r(i−1)}ù Di®ô� 

°.Z¢ ��� #"�(residual)� �; *_ �³Y �p�t¿ �;� #"� ¯â ?�®ê �è �p¢

°((�� 29÷S).0�t½é 41ù°üY��°èvÇý°.

rT(i)r(j) = 0, i 6= j (44)

43�t�y¿ÏPìù���^#"�(residual)r(i)��;#"�(residual)® Ad(i−1)�xÍS¦�

��,�°. d(i−1) ∈ Di�ú\� 8,�Pìù@v���^¿ÏÙÛW� Di+1��;ÙÛW� Di®

ÙÛW� ADi�¦÷¿ÙâÍ�ý°�,�°.

D(i) = span{d(0),Ad(0),A2d(0), · · · ,Ai−1d(0)}

= span{r(0),Ar(0),A2r(0), · · · ,Ai−1r(0)}.

�ÙÛW�ú¾�¿�ÙÛW�(Krylov subspace)��L �Ú,�,ù #�¯â�qE±µú�

Ä:÷¿:Ì �f�£½�°.�W�ù�Í^ù"�ú��L�°: ADi� Di+1�j¥üq�L,

°ü#"� ri+1Y Di+1Y�p t¿(½é 39), ri+1� DiY A-�põ�Ú°�PìúN½�°. ri+1�

d(i)õC½¢�;�?�Ò_�³Y�y A-�põ�ØL��:[���-Ùy!@ªÜ�z��³©

��,�°!

½é 37�#Í#���-Ùy!\½� βij = −uTi Ad(j)/d

T(j)Ad(j)�ú\� �;�éú�� 3��

qÃ�.Íx½é 43Y riú4: 8°üY�ù@Yõu�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 40

Page 41: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 29]�ªEPê�ª�t,�³G�t^¿u©��#"�(residual)��;�³G�t#Í.Ï#"�(residual)

X�� ��p�6,^Ä3xØü�Ò_�³ù�;�?�#"�®Ò_�³�©A-�p�üêÀ��q�°.

(Ò_ �³ù #"� ¯âõ �Ì � ��°). r(2)® d(2)� �>ù D(2)(q�3 �²� ÙÛW�)® e±¢ e8� n�

�°. �ªEPê�ª�t d(2)� r(2)® d(1)�xÍS¦�°.

rT(i)r(j+1) = rT

(i)r(j) − α(j)rT(i)Ad(j)

α(j)rT(i)Ad(j) = rT

(i)r(j) − rT(i)r(j+1)

rT(i)Ad(j) =

1

α(i)rT(i)r(i), i = j,

−1α(i−1)

rT(i)r(i), i = j + 1, (é 44�� �)

0, �½�EÍ.

∴ βij =

1

α(i−1)

rT(i)r(i)

dT(i−1)Ad(i−1)

, i = j + 1

0, i > j + 1, (é 37�� �.)

�Á�¤ÂÙÛ� βij ¨ùP�K£¹°.�C^¿Ï¯â� A-�p�üêÀ �æ©Ì�\�;

�PÌ°ÏÒ_¯âú?�õ9$£�Å�{°.� ¢|Å¢$>��ªEPê�ª�|Å¢>�

6,�ÄèW�Ä!ê®è�Ä!ê8�t O(n2)�t O(m)¿��ý°.�: mù A� 0�I¨�Û�

½�°.�CÙâ#� β(i) = βi,i−1�9cÍ÷¿v�£,�°.XÌ��&��8°üY�°.

β(i) =rT(i)r(i)

dT(i−1)r(i−1)

(½é 32�� �)

=rT(i)r(i)

rT(i−1)r(i−1)

(½é 42�� �)

�C��«�z<¢4Ì?�?I #¿Aý©Ã�.�ªEPê�ª��°üY�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 41

Page 42: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 30]�ªEPê�ª

d(0) = r(0) = b−Ax(0), (45)

α(i) =rT(i)r(i)

dT(i)Ad(i)

(½é 32® 42�� �) (46)

x(i+1) = x(i) + α(i)d(i), (47)

r(i+1) = r(i) − α(i)d(i),

β(i+1) =rT(i+1)r(i+1)

rT(i)r(i)

(48)

d(i+1) = r(i+1) + β(i+1)d(i) (49)

�� 30ù �ª EPê �ªú Íý� PÌ L �� �C� :Ì°ú :� u3 ü� � ú Ã�s

L �°. Pì � �ª�t #Í#� �Ð�(gradient)� t¿ �ª� I¦� :[� “�ª EPê(conjugate

gradient)”��Ìq�°��Gý,��L£½�°.��ªù�Ð�õ�ªÜ �Ò_�³úu�:

[� “�ªÜý�Ð�(conjugated gradient)”�ª��L �,�ÌAÝ¢vÇ�°.

9 ���ªªªEEEPPPêêê���½½½¶¶¶ÛÛÛuuu

�ª EPê� n ¥� �Ä ó� °Òý°. �©°8 �ª EPê �ª� ½¶ Ûu� · [îú �qb £

copyright c©Jonathan Richard Shewchuk and VisCom Commune 42

Page 43: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

«?ìC¿ � �ªú :Ì 8 Ùô�½> �¤� ¡ò :[� #"�(residual)� >> ÙAÝ©�6,£�

¡ò(cancellation error):[�Ò_¯â�P�� A-�p��>>\ìý°.�¤�¡ò��¢[C�/

 ��ª�t©@¢�®��°Û½���,£�¡ò��¢[C�Ö3©@£½��[C�I

¦°.��¤¡ò��©Ò_¯â�P���ª��·q��[C:[� 1960[Â��(:«�½¡

G���ªEPê�ª�[îú��MU÷6, 1970[Â��qt�Äú�©©õu ��ª÷¿�

�ªEPê�ª�îY:���@Y��vý�ó�bü¿t°è[îú�3ý°.

è��� �ªEPê�¢>©êº°°.¡�(�ªEPê�ìC¿:Ìü�[C��¾��

CX#vt n¥��Ľ±�Ý� ¢EÍ�ÂÙÛ��:[�,½¶Ûu�|Å¢�yõ��°.½

¶ÛuùÙô�½>¡òõ��æ¢,�I#�,AÝ¢NLý�÷¿©@£½{�[C��©�

ªEPê�ª�îÌ °�>ú�< �æ¢,�°.

�ªEPê�ª��¥«�ijG�/ ��ª��¥«�ijG®ô� °.0�t 6.1<

�4Ìú�¿�Ì ��ªEPê�ª�³¢¥��Ä��½¶ �S&úz<£½�°.

9.1 °°°¹¹¹¢¢¢°°°¨éééLLLòòò���

�ªEPê�ª��³G�t, e(i)� e(0) +Di¿ÙâxØüL,�: Di�°üY�ùÙÛW��ú�

yU_ÃU°.

Di = span{r(0), Ar(0), A2r(0), · · · , Ai−1r(0)}

= span{Ae(0), A2e(0), A3e(0), · · · , Aie(0)}.

�®�ù¾�¿�(Krylov)ÙÛW�ùZ¢���Û^ù"�ú��°.LAý i�©,¡ò¨ù°

üY��vÇý°.

e(0) =

I +i∑

j=1

ψjAj

e(0)

G½ ψj� α(i), β(i)® [Gü6,AÝ¢ [G�ù ��t |Å � M°.|Å¢ ,ù 7.3� #® ��

�<,�,�ªEPê�ª� ||e(i)||Aõ/�Ü �G½ ψjõxØ¢°�Pì�°.

æ é�t \ÒK� vÇù °¨é÷¿ vÇþ ½ �°. Pi(λ)õ ò½ i� °¨é��  �. Più �

¿ �# ±µ� ¤ ½ �÷6, ��ù ô� 3 GSý°. �õ �q, �� P2(λ) = 2λ2 + 1��L  8,

P2(A) = 2A2 + I�ý°.� ¢õ����v�ªù Pi(A)v = Pi(λ)v(Av = λv,A2v = λ2v, · · · .)®��

Lî¯â�tîÌ °.

�� Pi(0) = 1�üêÀ ²L 8,¡ò¨ù°üY��vÇþ½�°.

e(i) = Pi(A)e(0),

copyright c©Jonathan Richard Shewchuk and VisCom Commune 43

Page 44: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

�ªEPê�ªù ψj G½õxØ£:,�°¨éúxØ¢°.�°¨éú e(0)�:Ì£:�îY�©

SP �./ ��ª�Ûu�t®�ô��¿, e(0)õ�p³æLî¯â��xÍS¦÷¿vÇ¢°.

e(0) =n∑

j=1

ξjvj ,

�ýL°ü�é�ú��q7½�°.

e(i) =∑

j

ξjPi(λj)vj

Ae(i) =∑

j

ξjPi(λj)λjvj

‖e(i)‖2A =∑

j

ξ2j [Pi(λj)]2λj .

�ª EPê �ªù � vÇéú /�Ü � °¨éú ü÷6, � �ª� ½¶ �ê� IXý ^Iê /J

� Lî¯â� ½¶ � Aêð ^ú ½� {°. Λ(A)� A� Lî�� �¦��  8 ¡ò� �C�

i(norm)ù°üY�°.

‖e(i)‖2A ≤ minPi

maxλ∈Λ(A)

[Pi(λ)]2∑

j

ξ2jλj

= minPi

maxλ∈Λ(A)

[Pi(λ)]2‖e(0)‖2A (50)

��31ù Lî 2® 7ú ��� �C� © æ� vÇéú /�Ü � °¨éú = �� [ê� ò

½� © Ã�sL �°. P0(0) = 1ú �T � ò½ 0� °¨éù  # �� {÷6, �� 31(a)�t Ã

� �® �� P0(λ) = 1�°. ò½ 1� /: °¨éù P1(λ) = 1 − 2x/9�L, ��31(b)�t Ç ½ �°.

P1(2) = 5/9, P1(7) = −5/9�L,�ª EPê� �¥« �Ä ó ¡ò¨� �C�i� #�X 5/9ð ¾

�MüúN½�°.��31(c)��¥��Äóé50�@Y� 0�ÿúÃ�u°.�,ù 2ò°¨é��

>(P2(0) = 1, P2(2) = 0, P2(7) = 0)ú�#êÀ£½��:[�°.��:÷¿ò½ n�°¨éù n+1�

>�ú�#¿À��½�÷t¿ n��t¿°ôLî�ú½Ì£½�°.

��«�� f�õ �© �ªEPê�ª� n¥� �Äó� AÝ¢ @Yõ u�°� ,ú Ì � N ½ �

°: ÌÍ� �ù Lî�� U&¢°8 �ªEPê�ª� Ì �ò3 ½¶¢°� ,� ¢ �<�°.X¢

½u�Ùô�½>A}êõ��°L£:,AÝ¢©õGS �æ©Åuü��Äì½��Ê©bt¿

°ôLî��½�ý°. (�ô[Ò�¢¢��°ô� ���°: x(0)� A��ÙLî¯â�Y�

y A−�põ�ØL��EÍ�°.�c x(0)õ;�£:#Í#�M�Lî¯â�ù��©¾ �Lî

��èé 50�#Í#�Mú,�°.� #XtEL¢�®��,� ¢Lî¯â��èÙô�½

>�¤�¡ò��©°è#Í#3þ,�°.)

copyright c©Jonathan Richard Shewchuk and VisCom Commune 44

Page 45: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 31] i¥� �Äú ½±¢ �ó �ª EPê �ª� v�# ½¶°��� Pi(0) = 1�� sq� C¢ S&�t i ò

°¨é Pi���Lî�tv�# 0��«Ï��µ²�°.

Lî�� λminY λmax P�� Ý�Y 3 Ûjüq �� ,ð,��31(d)® �� ��� ?� ��

EÍ�ªEPê�ª�ÌÎý½·£½�üúN½�°.�,ù�ªEPê�ª�té50�ú�

3���°¨éúxØ �,�ÌÎÑq��:[�°.

��±µA�Lî��"�úNL�°8�ô½¶ú��q4�°¨éúCK£½ê���,�

�t��$��: EÍ,�Lî�� λminY λmaxP��Lò3Ûj L,|Äü�Lî�½�:

q�ù½�t¿°ôLî�#Í#6,Ùô�½>�¤�¡ò��f �EÍõ�A¢°.

9.2 ���üüüËËË���(Chebyshev)°°°¨ééé

¢ �� îÌ¢ ?��ªù î¢ �� >� ©t� I¦� ©æ[λmin, λmax]�t é50ú /�Ü � ,�

°.�õ½±£½��°¨é�ù�üË�(Chebyshev)°¨éú��÷¿¢°.

ò½ i��üË�(Chebyshev)°¨éù°üY�°.

Ti(ω) =12[(ω +

√ω2 − 1)i + (ω −

√ω2 − 1)i].

(�é�°¨é�¤Ã��M�°8, i� 1�# 2�EÍ�©�qÃ�.)=���üË�(Chebyshev)°

¨é�� ��32� �²K �°.�üË�(Chebyshev)°¨é�ù A�� ω ∈ [−1, 1]�t |Ti(ω) ≤ 1| "

�ú��6 (Pì -1Y 1�P��t�ô¢°),?�°¨é��qt |Ti(ω)|�ù ω /∈ [−1, 1] A���

copyright c©Jonathan Richard Shewchuk and VisCom Commune 45

Page 46: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 32]ò½ 2, 5, 10, 49��üË�(Chebyshev)°¨é�

t/ÂXõ 3ý°.y} �MùvÇ÷¿�b� 8, |Ti(ω)|�����#Í#��\����

t� ¢¢�$�ò3��¢°.

½é 50�°üY�ù Pi(λ)õxØ¥��©/�Üü�,ùÙÀ C3�tz<ý°.

Pi(λ) =Ti(λmax+λmin−2λ

λmax−λmin)

Ti(λmax+λminλmax−λmin )

� °¨éù �� λmin ≤ λ ≤ λmax�t �üË�(Chebyshev)°¨é� �ô "�ú ��°(��33÷S).

Û?� Pi(0) = 1��� ÅuP¨ú ?T 3  6,Û�� λminY λmax P�� u��t / 1ú ��

°.0�té50÷¿Ùâ°ü�é���¢°.

‖e(i)‖A ≤ Ti

(λmax + λmin

λmax − λmin

)−1

‖e(0)‖A

= Ti

(κ+ 1κ− 1

)−1

‖e(0)‖A

= 2

[(√κ+ 1√κ− 1

)i

+(√

κ− 1√κ+ 1

)i]−1

‖e(0)‖A. (51)

P�Í\ÒK��¥«�½(�Õ, addend)� i���¥�0� 0÷¿½¶¢°.0�t�ªEPê�½¶

ùXÌc¢Ù#é÷¿°üY��vÇ �,���:�°.

‖e(i)‖A ≤ 2(√

κ− 1√κ+ 1

)i

‖e(0)‖A (52)

copyright c©Jonathan Richard Shewchuk and VisCom Commune 46

Page 47: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 33] ��: EÍ�t �� λmin = 2® λmax = 7�t é 50ú /�Ü � °¨é P2(λ). � Mxù ò½ 2�

Chebyshev°¨é� ¾�õ ºÍ¢ ,�°. � ¥� �Ä �ó� ¡ò¨� �C� i(norm)ù #�� 0.183�õ J�

M�°. 2��Lî��U& �,÷¿�yN²K�Ï�� 31(c)®üp �.

�ªEPê��¥«³G�/ ���¥«³G®ô� °.é51�t i = 1¿�8,/ ��[

¢½¶@Y é28úu�°.�,ù�� 31(b)��²�xÍ°¨é�EÍ°.

�� 34��ªEPê��Ä�°�½¶�¢êv�°.ìC¿:Ì°ú:,�ªEPê�^ùLî

Ûj#^ùè�æX#��î:[�Â��EÍé 52�èP �½¶�êðÌ�ò3½¶¢°.

é 52®é 28úüp 8,�ªEPê(conjugate gradient)�ª�/ � (steepest descent)�ªÃ°Ì�

ò3½¶ �°�,�<Ý ° (�� 35÷S).� #�ªEPê�?��ijG�/ ��ªÃ°

Ì �ò3 ½¶ � ,ù I¦°; �õ �8,�ª EPê �ª� �¥« ³G� /  � �ª� �¥« ³

G®ô� °.é52�tU©K��½ 2��©t�ªEPê�ª��Ù�ijG�tc���ê9

 õÃ�½�3ý°.

10 ÄÄÄ!!!êêê

/ ��#�ªEPê�ª��³G�t�$�ùGSè�ú�Å¿ ��Sù±µ-¯âU��°.

��:÷¿,±µ-¯âU�ù±µ� 0�I¨Ù��½õm��£EÍ O(m)�GSè�ú�Å¿¢°.

�[t� 1<�t#�ý°j¢[C��tÃ� A� �±µ�Lm ∈ O(n)�°.

ε��½¿¡ò�¾�õv��æ©?Û¢�Äú½± �õÙ¢°L�A �:�, ‖e(i)‖ ≤ ε‖e(0)‖.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 47

Page 48: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 34]\×½�¥½¿t��ªEPê(�Ä�°)�½¶. �� 20Yüp �.

é 28ù/ ��ªúPÌ ��¢Gõ�T �æ©Åuü�/Â��Äì½�°üY�üúÃ

�u°.

i ≤⌈

12κ ln

(1ε

)⌉,

¢`,é 52�� 8�ªEPê�ª�Åu �/Â��Äì½�°üY�°.

i ≤⌈

12√κ ln

(2ε

)⌉.

/ �ù O(mκ)�è�Ä!êõ��6,�ªEPê� O(m√κ)�è�Ä!êõ��°�@Á�°.�

NLý�?�W�Ä!ê� O(m)�°.

d−òÙ A���t �ò ÍÙ EG [C�ú î¢òÛY î¢Å�ª÷¿ �P£ :� Ä!ê� [[

κ ∈ O(n2/d)� ý°. 0�t, /  � �ªù �òÙ [C�� © O(n2)� è�Ä!êõ ���� �

ªEPê� O(n3/2)�°.Z¢XòÙ[C��©/ �ù O(n5/3)�Ä!ê���ªEPê�ªù

O(n4/3)�è�Ä!êõ��°.

11 èèè���YYY[[[ÒÒÒ

Xtz<¢/ �Y�ªEPêNLý��t,=���Ù: ,��f�ü}°;"&,è�>úqN

3xØ LsC[Ò �=�[C�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 48

Page 49: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 35]�ªEPê�¢¥��Ä��X �æ©Åuü�/ ���Äì½.

11.1 èèè���

è��[ �z<£,ù��M°.�� x��©Â�: 8AX��°8,�,úè� x(0)¿

PÌ �.�©�M°8, x(0) = 0÷¿�q�;xÍèßðú{�EÍ/ ��#�ªEPê�ªù@

v ½¶ 3 üq �°.üxÍ /�Ü(14<�t z<)� � �� ��/�� U&£ ½ �L è� æX�

xØ�0�q�/�>�½¶£��@Aü6,qEEÍ��½¶��Øq���©�Mú�õ@A 

�:[�«°¿Ï[C�ý°.

11.2 [[[ÒÒÒ

/ ��#�ªEPê�/�>�êµ£:#"�(residual)� 0�üL,��é11�# 48��ó�¢

¥��ÄúÌ 3¢°8,Û?� 0�ý°.0�t#"�(residual)� 0�ü8�èGS�[Òüqb¢

°. #"�(residual)� >Üé Í×(47)GS�t �¤� ¡ò� �:ü� EÍ�� #"�� 0� I¨Úê

0÷¿GSþ½��°;�[C�é45õ�Ì �GSú^¿è�¥÷¿t©@þ½�°.

Ã��Eͽ¶�°;&�Øq��;�NLý��[Òü�õÙ¢°.¡ò¨ú�Ì£½{�:

[� #"�(residual)� ¾�� P;� <èý � ¿ 4²�8 [Ò � ,� Ã��°; [[ � ÷¿

#�#"�(residual)� 0ð�ù εúU©uù�ùúPÌ¢°. (‖r(i)‖ < ε‖r(0)‖). �C���

ÙÀ B�tǽ�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 49

Page 50: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

12 ;;;���ýýý

;�ý�±µ�S&½(condition number)õ�x �æ¢�ª�°.±µM�±µ Aõ�P �Âa�

8tj�AÙÒ ±µ�L,�±µúÖ3GS£½�°L �.�©°8°üéú�qt�?:÷¿

Ax = bõ�½�°.

M−1Ax = M−1b. (53)

�� κ(M−1A) � κ(A)�$#M−1A�Lî�� A�Lî�ðÌ�?��°8,Ù��[Cð

é 53�[C��Īú�©Ì�½¶¢°.[C�±µM�# A��Âa�Lj�AÙÒ±µ�� 

Ì�ê±µM−1Aù��:÷¿�©�M°�,�°.

Âa�6j�AÙÒ ?�±µM�© EET = M "�ú���±µ E� (î�¢,ùI¦�

�)U& �:[�,�q²Óù�£½�°. (�¢±µ E�&ªßÅ(Cholesky)Û©®�ù�ª÷¿u

£½�°.)±µM−1A® E−1AE−T�ô�¢Lî�ú��°.�,��� ��î� v�Lî λõ

���±µM−1A�Lî¯â�:, ET v�°üY��Lî λõ���±µ E−1AE−T�Lî¯â�

�:[�°.

(E−1AE−T )(ET v) = (ETE−T )E−1Av = ETM−1Av = λET v.

xÍèßð Ax = b�°ü�[C¿ºÍþ½�°.

E−1AE−T x = E−1b, x = ETx,

�[C�tÍý�Íx x�©�L#t x�©}°.±µ E−1AE−T�Âa�8tj�AÙÒ��

:[�, x� / ��# �ªEPê� �© u£ ½ �°.�ª EPê �ªú PÌ � � [Cõ ©@ 

�YAú “ºÞý;�ý�ªEPê�ª(Transformed preconditioned conjugate gradient method)”��Ùô

°.

d(0) = r(0) = E−1b− E−1AET x(0),

α(i) =rT(i)r(i)

dT(i)E

−1AE−T d(i),

x(i+1) = x(i) + α(i)d(i),

r(i+1) = r(i) − α(i)E−1AE−T d(i),

β(i+1) =rT(i+1)r(i+1)

rT(i)r(i)

,

d(i+1) = r(i+1) + β(i+1)d(i).

copyright c©Jonathan Richard Shewchuk and VisCom Commune 50

Page 51: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

��ªù±µ EõGS©b¢°�[C��°.� #c��s�©tº½õXÞ 8±µ EõC

$£½�°. r(i) = E−1r(i)® d(i) = ET d(i)¿�L x(i) = ETx(i)® E−TE−1 = M−1��¨#éú:Ì

 8,°üY �� “ºÞü� Mù ;�ý �ª EPê �ª(Untransformed preconditioned conjugate gradient

method)”õuú½�°.

r(0) = b−Ax(0),

d(0) = M−1r(0),

α(i) =rT(i)M

−1r(i)

dT(i)Ad(i)

,

x(i+1) = x(i) + α(i)d(i),

r(i+1) = r(i) − α(i)Ad(i),

β(i+1) =rT(i+1)M

−1r(i+1)

rT(i)M

−1r(i),

d(i+1) = M−1r(i+1) + β(i+1)d(i).

±µE���Aé�t�#Í#�M�°;��t�±µM−1��Å °.�ù�ª��©,±µEõ

PÌ �M�;�ý/ ��ªú��q4�,ê�  °.

;�ý�M�îî�ù±µM−1A�S&½��©@AüL,EÍ�0�Lî��Á ß!�A

ê��©@Aý°.[C��¥�Ä��q(:�°¢¥BM−1r(i) U�ú½±©b �:[��f

 �üÌú\©£½�úAê¿ Aõ��P �½¶�êõm�½��;CS&�õü�,�°. (<

è:÷¿M Z�M−1ú GS£ �Å� {°; �Å¢ ,ù ³� ±µM−1ú ¯â� :Ì°ú : �f �

îYõGS �,�°.)� ¢C¢S&��t,g�3ê�Ù¢� �����Ú��t�½�0¤

�®���Ù�°Û,�°.

�[:÷¿,;�ý� �ò Íéú ÌÎ uÍ� �²3 ��²� èê�°.0�t Lî�ù t¿�3

�?¢°.°;;�ý�(perfect preconditioner)�¢��M = A�°;�;�ý��©,M−1A�S&½

� 1� ü6,�ò Íéù °¹¢ uÍ� üq ³ ¢ ¥� �Ä�÷¿ê ©õ u£ ½ �°.� #,ݱ 3

ê,;�ý³G���Mx = bõ{�,�qt�;�ý��@�îÌ¢;�ý��I¦°.

�$�³¢;�ý��Â��Û��±µ AYô�¢Â�±µ�°.�;�ý�õ:Ì �YA

ùÂ�;�ýZ�b�ü;�ý���ö÷¿N²K�÷6,�YAù_v9�³÷¿�òÍé�¾

�õºE �,Y�°. (üpõ 8,°;;�ý�M = A�Lî¯â9ú0��òÍé�¾�õS<

¢°.)Â�±µ��±µúu �,ù�Í�³ ��,�±µùÃ�e©¢½u�;�ý��<�°.

Â�;�ý�ó��C[C��òÍé����#Lx����36�Ã��°.��3Yüp 8,�A

copyright c©Jonathan Richard Shewchuk and VisCom Commune 51

Page 52: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 36]�C[CõÂ�;�ý¢�óu��òÍé�#Lx�

Aê�xü}°�,�<� °. 3.5�ÏS&½�Â� 2.8«�³\ý°.]Á,� ¢�xù n� 2 x

Íèßð�tÌîÌ °.

Ì Ap¢ ;�ý�� Ý°; &ªßÅ ;�ý(incomplete Cholesky preconditioning)�°.&ªßÅ Û©

�±µ Aõ LLT�Í׿۩ ��Á�°.��t LùI�X�±µ(lower triangular matrix)�°.Ý°

;¢&�ßÅÛ©�ýÓ�´Ìü�M�ºÍ�°; A� LLT®�ù±µUÍ׿�Pü�Ú,�: L�

A®ô�¢Í׿ 0�I¨Ù��#Í#b¢°; L�°ôÙ��ù?�£²�°. LLTõ;�ý�¿P

Ì �æ©, LLTw = z�©��XÞ�ª÷¿GSý°. (LLT��±µù@�<è:÷¿GS£�Å

�{°).ݱ 3êÝ°;¢&�ßÅ;�ý�¨\KA:���M°.

�ù ;�ý��� (� | �Ù� �Í Ä! °) ��ü}°.;�ý�õ qE ,ú xØ ��� ��,

Â�?[C�t�ªEPêõPÌ£:��$�sC#;�ý�õ�Ì �[Cõ�qb¢°�Pì

���:÷¿�I���L�°.

13 ���ªªªEEEPPPêêê

�ªEPê�Âa�6tj�AÙÒ�I¨,�ýLî�qA�±µ�I¨EÍ�êPÌþ½�°.°

üY�ù/��ç[CõÃ�.

minx‖Ax− b‖2 (54)

copyright c©Jonathan Richard Shewchuk and VisCom Commune 52

Page 53: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

�[C�©�é54�yÛú 0÷¿�÷¿tu£½�°.

ATAx = AT b. (55)

�� A�A�Í�ü"�(non-singular)±µ�� 8,é 55�©� Ax = b�©®ô� °.�� A�

A� ±µ� I¦L Ax = b� º½Ã° Ì�ù xÍ ë� �Aéú ��� Y Cc(overconstrained) E

Í, Ax = b� ©� U&£ ½ê �L U& � Mú ½ê �°. �� � xÍ�Aé� ¡òCUú ¦¢ ½

é54ú/�Ü � xúü�,ù¨\�  °.

ATA�Âa�Lj(��� x�©,xTATAx = ‖Ax‖2 ≥ 0)�°.��Ax = b�Y�Cc(underconstrained)�

I¦8,ATA�ü"�±µ�L,/� ��#�ªEPê�ªY�ù�ª�é55õ��æ©PÌþ½

�°.³�¢��[C� ATA�S&½� A����S&½�CU�L,0�t½¶��ý°�,�°.

��t�Á:÷¿|Å¢P¨ù±µ ATA� AÃ°Ñ �±µ��:[�<è:÷¿Í�ü�M

�°�,�°.Âê, Adõu¢�� ATAdõu¥÷¿�, ATA�� d�U¢\×�ý°.Z¢ Adõ��

�êY4: � dTATAd (é46)õGS 8½X: KA��³\ý°.

14 üüüxxxÍÍÍ���ªªªEEEPPPêêê���ªªª

�ªEPê�ªù�òÍé(quadratic form)�/�>úü�Ú�PÌþ<�I¦��Ð� f ′õGS£

½��qE��¥½ f(x)�©tê/�Üõæ©PÌþ½�°.��W¡zG,êE�¡å,�ýL

üxÍç�(regression)#Y�ù°j¢[ê�/:Ü[C�ÿÌþ½�°.

14.1 üüüxxxÍÍÍ���ªªªEEEPPPêêê���ªªª������ÅÅÅ

üxÍ�ªEPê�ªúîê£:xÍNLý�Yüp ����ºÜ��°:#"�(residual)�Â

¢&�:½é�PÌþ½{÷6,�< αõGS �,�ÌÎÄ! 6, β�©=��°ôxØ�

ª��°�,�°.

üxÍ�ªEPê�ª�t,#"��¨\�Ð��ÙÒõ�¿¢,÷¿zAý°; r(i) = −f ′(x(i)).

Ò_�³ùxÍ�ªEPê�ª�¤#"��¢��-Ùy!(Gram-Schmidt)�ªÜõ�Ì �GS

ý°.� Ò_ �³ú 0� �x Ò_ú  � ,ù xÍ EÍ� ü© ÌÎ q²Í6,°j¢ <ò� PÌþ

½�°.xÍ�ªY��, f(x(i) + α(i)d(i))õ/�Ü � α(i)�ù�Ð��Ò_�³��p êÀ¥

÷¿�üú½�°.�õæ© [f(x(i) +α(i)d(i))]T d(i)� 0�ü�úü�qE[ê�NLý���êP

Ì£½�°.

xÍ �ª EPê �ª�t� β� � ¢ = �� ôX(Ý>) vÇ�� �°. üxÍ �ª EPê �

ª�t�t¿°ô�vÇ��Ì�\ôX�I¦°;�u��ù/x�xØ�X| �I�êSP 

copyright c©Jonathan Richard Shewchuk and VisCom Commune 53

Page 54: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

L�°.���xØ�ÏÚ #�GS�Ì��:[�xÍ�ª�PÌü}Ï�ª�-ýõß(Fletcher-

Reeves)é�6,°ô #�m�-ýü�ò(Polak-Ribiere)é�°:

βFR(i+1) =

rT(i+1)r(i+1)

rT(i)r(i)

, βPR(i+1) =

rT(i+1)(r(i+1) − r(i)

rT(i)r(i)

.

�ª�-ýõß�ªùè�>�Ù �/�>Y?Û&�«Ð:�½¶ 6,m�-ýü�ò�ªù�

]¡ ���Ù&½¶ �ML�Ä£½�°.� #m�-ýü�ò�ª�[[Ì�ò3½¶¢°.

°±&m�-ýü�ò�ª�½¶ù β = max{βPR, 0}÷¿xØ¥÷¿�Ã$þ½�°.�úPÌ

 �,ù βPR < 0 EÍ��ªEPê�ªú^¿è� �,Y�°.�ªEPêõ^¿è� �,

ù�;�Ò_�³�ú?��L�$� 3 � �EP�³ú0�^Ä3�ªEPê�ªú½±

 �,�°.

üxÍ�ªEP�ª��Å�°üY�°:

d(0) = r(0) = −f(x(0)),

Find α(i) that minimizes f(x(i) + α(i)d(i)),

x(i+1) = x(i) + α(i)d(i),

r(i+1) = −f ′(x(i+1)),

β(i+1) =rT(i+1)r(i+1)

rT(i)r(i)

or β(i+1) = max

{rT(i+1)(r(i+1) − r(i))

rT(i)r(i)

, 0

},

d(i+1) = r(i+1) + β(i+1)d(i).

üxÍ�ªEPê�ªùxÍ�ªEPê�ª�½¶Ã$���$�:Ìü�M�°.¥½ f��

ò¥½®îP �M÷8Mú½À,Ò_�³�ùÌý�ª�(conjugacy)ú��°. (üxÍ�ªEP

ê�ª�tê “�ª�”���Ìq��;&�yõ��°�,�O<�©�,�°.)Z°ô[C��

�: ¥½ f��ù��/�>ú��½�°�,�°.�ªEPê�ªù;�/�>�½¶£,�

�Ã$£½{÷6,î�q f� ¢(lower bound)�{�EÍ��/�Sòü�G£½ê�°.

�� 37ùüxÍ�ªEPê�ªú��÷¿Ã�L�°.�� 37(a)�°½���/�õ��L��

¥½�°.�� 37(b)��ª�-ýõß(Fletcher-Reeves)éúPÌ¢üxÍ�ªEPê�ª�½¶úÃ�

L�°.���txÍ�EÍ�¤îY:��M°;�¥½�/�Ü ���Íq·°.�� 37(c)���

37(b)�/#�xÒ_�³ú0�_8ú<³¢8úÃ�L�°.�<³8�°½�/�>�U&¥�

î� �;�xÒ_ù�«Ï/�>�©¾ � αúü�°.�� 37(d)�m�-ýü�ò(Polak-Ribiere)

�ª�Ì#ù½¶�úà °.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 54

Page 55: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 37]üxÍ �ª EPê �ª� ½¶. (a)°½� �� /�® /Âõ �� Ä!¢ ¥½. (b)�ª�-ýõß(Fletcher-

Reeves)�ª�½¶E¿.xÍ�ªYµý�³G��½¶ �M�°. (c)/#��xÒ_ú0ôv8<³@Y. (d)

m�-ýü�ò(Polak-Ribiere)�ª�½¶E¿.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 55

Page 56: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

14.2 ������:::���xxxÒÒÒ___

f ′��0�, f ′T d� 0�ü�©õü��ôNLý�úPÌ �,��  °.�õ�q, f ′� α�°

¨é�:,°¨é��úu �îñ: NLý�úPÌ£½�°.� #Íý�©ÌNLý��©

t�L²£,�°.

�[ê��Ä�ª÷¿�ä-�á(Newton-Raphson)�ªY£xª(}=� secant method)��°.��

ªù?� f��òyÛ��  �b¢°.�ä-�á�ªù f(x+αd)� α� � 2òyÛúGS£½

�qb¢°.

�ä-�á�ªùì� (Taylor)�½�Põ�Ì¢°.

f(x+ αd) ≈ f(x) + α

[d

dαf(x+ αd)

]α=0

+α2

2(56)

= f(x) + α[f ′(x)]T d+α2

2dT f ′′(x)d

d

dαf(x+ αd) ≈ [f ′(x)]T d+ αdT f ′′(x)d (57)

�: f ′′(x)�°üY�ù½èK(Hessian)±µ�°.

f ′′(x) =

∂2f∂x1∂x1

∂2f∂x1∂x2

· · · ∂2f∂x1∂xn

∂2f∂x2∂x1

∂2f∂x2∂x2

∂2f∂x2∂xn

......

...

∂2f∂xn∂x1

∂2f∂xn∂x2

· · · ∂2f∂xn∂xn

.

¥½ f(x+ αd)�é 57� 0�üêÀ¥÷¿��P:÷¿/�Ü£½�÷6�õ�©°üúu�°.

α = − f ′Td

dT f ′′d.

�²� ì� (Tayler)�½� f(x + αd)õ j]x÷¿ �P¢°; j]x� /�>«� 4²�° (��

39úÃ�).Pì f��òÍé��8,�j]x�P�AÝ¢,�ý°.��î� f ′′��¿Z�¢±µ

A��:[�°.��:÷¿,Ò_�³ù f ′′-�p (f ′′-orthogonal) EÍ��ª�ú �°L¢°.� #

“�ª”��y�G�©t��Ú,�� f ′′� x�0�º �:[�°. f ′′� x�0��ò3º 8º

£½À Ò_ �³� P�� �ª�� P��°.�8 x(i)� ©� �«Í8 �«Ð½À, f ′′� �Ä�° º 

�Aê�vq�°.è�>�©��«Ð½ÀüxÍ�ªEPê�ª�½¶��xÍ�ªEPê�ª

Yüæ©�°.

�ò� I¨ ¥½� AÝ¢ �x Ò_ú ½± � æ©t� f ′T� 0� þ :«� �xú 0� Ò_ú �

Ä©b� ¢°; 0�t  #� �ª EPê �ª �Ä 4� °½� �ä-�á �Ä� j¥üq �ú ½ �°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 56

Page 57: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 38]üxÍ�ªEPê�ªùs�:&è�úPÌ£:ÌîY:�°.

[�� 39]�òÙ¥½(ìx)õ/�Ü ��ä-�á�ª. > x�tè� � 1ò, 2ò꥽õGS L,�õ�Ì �

¥½�¢ 2ò�Põ¢°(>x). j]x�/�>�t^¿Ï> zõxØ¢°. ½¶£:«�� ¢³Gõ�Ä¢°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 57

Page 58: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 40] 1òÙ ¥½(ìx)ú /�Ü � £xª. > x�t è� � t¿ °ô � �>�t 1ò ꥽õ GS 6

(��t� α = 0® α = 2), �õ �Ì � ¥½� ¢ 2ò �P(>x)ú ½±¢°. α = 0 æX® α = 2 æX�t

� Mx� ?� ô�¢ �Ð�õ �°� ,� î� �. �ä-�á �ªY ��, ^¿Ï > z� j]x� /�>�t

xØü6,½¶þ:«��Ä¢°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 58

Page 59: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

f ′Td® dT f ′′d��ù�³G�°GSüqb¢°.� ¢GSù dT f ′′d��©u:÷¿��Üþ½

�°8üÌ�¾�M��, f ′′ ±µ;���Ä:÷¿GSüqb� t¿NLý��PÌ£½{úA

ê¿�ý°.�ÙÿÌ�t, f ′′�Â��Û�úPÌ ��P: �xÒ_ú½± �� ¢[Cõç

�£½�°.]Á f ′′õ;ÅGS£½{�¥½�ê�°.

f ′′õGS �MLAÝ¢�xÒ_ú½± �æ©£xª(secant method)� α = 0® α = σ ��

�t¿°ôæX�t f(x + αd)� 1ò꥽õGS L�õ�Ì � 2ò꥽õ�P¢°.�: σ�

0�I¨����ù½�°:

d2

dα2f(x+ αd) ≈

[ ddαf(x+ αd)]α=σ − [ d

dαf(x+ αd)]α=0

σσ 6= 0 (58)

=[f ′(x+ σd)]T − [f ′(x)]T d

σ,

�½� α® σ� 0��«Ð½ÀÌ^ù�P�ý°.é 58úì� �½(é ??)� 3¥«¨÷¿�ô8

°üúu�°:d

dαf(x+ αd) ≈ [f ′(x)]T d+

α

σ

{[f ′(x+ σd)]T d− [f ′(x)]T d

}.

꥽õ 0÷¿��q f(x+ αd)õ/�Ü � αõ°üY��u£½�°:

α = −σ [f ′(x)]T d[f ′(x+ σd)]T d− [f ′(x)]T d

(59)

�ä-�á �ªY ��, £xª(secant method)�è f(x + αd)õ j]x Í׿ �P ��, ¢ >�t

� 1ò, 2ò ꥽õ GS � ,� I¦� t¿ °ô � >�t 1ò ꥽õ GS � (�� 40)j]x

ú xØ¢°. £xª� �Ä �z� � ¥ « ³G�t� ��� σõ xØ¢°; �q�� �Ä ³G�t�

x + σd��;³G� x�üêÀxØ¢°.°è©, α[i]�£x¥ i¥«�ijG�tGSý α��

8 σ[i+1] = −α[i]�°.

�ä-�áY £xª ?� x� ©� ?Û& �«Í8 [Òüqb ¢°.�#X3 0ù A}êõ Åu 8

½¶�ìC£½ê���,�#X3mùA}êõÅu 8Ý�Å 3GS��²�L"»¢��úu

�G¢°.�� f ′′(x)� x�©��º£EÍ�ª���ò3��ü�:[�°.0�t,�ò��Ù

AÝ¢ �x Ò_� [[ Ì #ù Aþ�° (�õ �q,�ä-�á�# £xª�t C¢ý ½� �Ä�ú ½

± �,�°).ݱ 3êÙAÝ¢�xÒ_ù � ��³�I¨Ò_�³úf� 3þ½ê�°.

��: ©ªù� ¢P×õyý*P �,�6 (rT d�j½�I¨�?),�Å¢EÍ d = r¿zA 

��ªEPê�ªú^¿è� �,�°.

��ª�ÌÀ[C����ª�/�>Y/Â>úuÛ �G¢°�,�°.üxÍ�ªEPê�

ª�@Y�è�>�� 3�U 6,�ä-�áÓù£xªúPÌ¢�ªEPê�ª���/�>�

��tè�ü}°8©¾æX¿½¶£� ��¾°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 59

Page 60: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[�� 41]m�-ýü�ò(Polak-Ribiere)éú�Ì¢üxÍ�ªEPê�ªúÂ�;�ý±µ¿;�ý¢@Y. W��

“a�K” /�>sº#Lx�ÌÙÍ(Ìþ)��«×PüúÃ�L�°.

����ªù#ö�$>ú��°.�ä-�á�ªùÌ#ù½¶�êõ��6, dT f ′′d��ò3 (�,

O(n)è�K�)GSþ½��EÍ� (Óù�Pþ½��EÍ�)xÒý°.£xªù f� 1òyÛ�ú

Åu ��,� �ª� �Wù 7�yâ σõ v�# � xØ �=� _Íý°.�� �ª ½� °ô °j¢

�ªêÖ3îꣽ�°.�õ�q fõ���t¿°ô�>�tb��(sampling)¥÷¿� f� 1òê

¥½SòGS£�Å{� f(x+ αd)õ�P �j]xúf� �,ê�  °.

14.3 ;;;���ýýý(Preconditioning)

üxÍ�ªEPê�ªù f ′′õ�P 8tM−1r�Ö3GSþ½��;�ý±µ(preconditioner matrix)

MúxØ �;�ýõ£½�°.xÍ�ªEPê�ª�t;�ý±µù�òÍéúºÞ �u®î

P 3 ��²L ¢°; üxÍ �ª EPê �ª� ;�ý ±µù � ¢ ºÞú x(i) ��� ��� © ½

±¢°.

;� ½èK ±µ f ′′õ GS � ,� �#X3 mù üÌú �:�ê � ±µ� Â� �Ûú GS �

�õ ;�ý ±µ¿ �Ì � ,� [[ îÌ °.� # x� �� /�>�t ?Û& FqK �ú :� ½

èK ±µ� Â� �Û |� j(�)� I¨ ,� �ú ½ê �°� ,� s� �. ;�ý ±µù j� AÙ

Ò(positive-definite)�qb  t¿ j� I¨ Â� �Ûù ´Ìþ ½� {°.ý: ©@þù ½èK� j

copyright c©Jonathan Richard Shewchuk and VisCom Commune 60

Page 61: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

�AÙÒ�úÃ$£½{ú:��;�ýõ �M�,�° (M = I¿zA).�� 41ùÂ��Û�©

;�ýõ ½±¢ m�-ýü�ò(Polak-Ribiere)üxÍ �ª EPê �ª� ½¶ú Ã�� ,÷¿ �� 37Y

ô�¢¥½�°.��t�³G�;�ýõæ©© x�t f ′′����Â��ÛúPÌ �½ªúPÌ 

�°2.

ÙÙÙÀÀÀ

A.[[[´ ���uuu

�ª�³�ªù 1908[Ùy!(Schmidt)��©[14]/#¿�vü}ú,÷¿Ã�6, 1948[�kß(Fox),

´ßÅ(Huskey),�ýLéÇá(Wilkinson)��©[7] ë�:÷¿�>ü}°. 50[Â#���ªEPê�

ª�½ß1ß(Hestenes)®[10]ß/Z(Stiefel)�[15]�©ë�:÷¿�>ü}°;�ýLO�q,�P�ù

Wô÷¿�ªEPê�ª�ªî: ÷L[¶÷¿�;��4Ìú;: �°[11].ËüË�(Chebyshev)

°¨é÷¿��ªEP꽶¢G�b¦�(Kaniel)��©��ü}°[12].�ªEPê�ªù 1971[�

��(Reid)��©Â�? �±µ�:Ìü��Ä:�ª÷¿Â|Üü}°[13].

�ªEPê�ªùÚ�ü�(Davidon)��u[4],�ýL�ª�(Fletcher)®7Ú(Powell)��u[5]õ�

�÷¿ 1964[�ª�(Fletcher)®ýõß(Reeves)��©üxÍ[C¿��Üü}°[6]. ÙAÝ¢�xÒ

_úPÌ¢üxÍ�ªEPê�ª�½¶ù°¦�(Daniel)��©Ûuü}°[3]. üxÍ�ªEPê�

ª�t βõxØ �[C�££!(Gilbert)®dèÑ(Nocedal)��©f�ü}°[8].

P¥(Golub)Y¤ýqý(O’Leary)� 70[Â|���ªEPê�ªY [´ý�P®�¢[¶�Â

¢suúCW �°.�óÂÙÛ��u�üÂaèßð�#>ú�:°.xÍèßð���õæ¢�

Ä:�ª�¢SP�u��ª!(Barrett)#��©�ØqP°[1].

B.NNNLLLýýý ���SSS���

�<�����(code)��fL�tf�ý� NLý �îñ: uÇúÃ�L�°.

B1.///   ���(Steepest Descent)

sq��³ A, b�©,è�>ù x�6,/Â�Äç½õ imax,¡ò´Ì©æõ ε < 1�°.

2��z:@v�© xõü�,�t¿,��©õü�æ©©õ�Ì¢ÙA±æ�©¾¢°.��t�;�ýõ�¢½¶�úÃ

��æ©�yNL��©õ�Ì �^ù;�ý±µúu¢,�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 61

Page 62: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

i⇐= 0r ⇐= b−Axδ ⇐= rT rδ0 ⇐= δWhile i < imax andδ > ε2δ0 do

q ⇐= Arα⇐= δ

rT q

x⇐= x+ αrIf i is divisible by 50

r ⇐= b−Axelse

r ⇐= r − αqδ ⇐= rT ri⇐= i+ 1

�NLý ù�Ä� imax ç�êµ $# ||r(i)|| ≤ ε||r(0)||�ü8|³¢°.

#"�� �ô >Üé÷¿ GSü�� 50¥ �Äþ :�° ¢ ¥Bù AÝ¢ ÷¿ &GSüq �:ý

Ùô�½> ¡êõ C$¢°.]Á �� PÌý 50��� ½� ��� ½�°; À n� ©√n� :¾ °.

¡ò´ÌX�¾°8,#"�(residual)õ½A£�Å�;Å{° (ìC¿½Aù$�PÌü�M�°).¡

ò´ÌX��G�Ùô�½>A}ê�¢G��?£EÍ, δ�GSý�� δ < ε2δ0 �Ý  �*P

�8�üqb 6,�*P@Y�÷��8AÝ¢#"�(residual)�°èGSüqb 6 δê°èe�

üqb¢°.�,ùÙô�½>�¤�¡ò��©NLý ��ê[Òü�,ú��°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 62

Page 63: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

B2.���ªªªEEEPPPêêê(Conjugate Gradient)

sq��³ A, b�©,è�>ù x�6,/Â�Äç½õ imax,¡ò´Ì©æõ ε < 1�°.

i⇐= 0r ⇐= b−Axd⇐= rδnew ⇐= rT rδ0 ⇐= δnew

While i < imax andδnew > ε2δ0 doq ⇐= Adα⇐= δnew

dT q

x⇐= x+ αdIf i is divisible by 50

r ⇐= b−Axelse

r ⇐= r − αqδold ⇐= δnew

β ⇐= δnew

δold

d⇐= r + βdi⇐= i+ 1

�;< B1�������4Ìú÷S �.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 63

Page 64: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

B3.;;;���ýýýýýý���ªªªEEEPPPêêê(Preconditioned Conjugate Gradients)

sq� �³ A, b� ©,è�>ù x�6,;�ý ±µùM (Rè:÷¿ A�þ ½ê �ü),/ �Ä ç½

õ imax,¡ò´Ì©æõ ε < 1�°.

i⇐= 0r ⇐= b−Axd⇐= M−1rδnew ⇐= rT dδ0 ⇐= δnew

While i < imax andδnew > ε2δ0 doq ⇐= Adα⇐= δnew

dT q

x⇐= x+ αdIf i is divisible by 50

r ⇐= b−Axelse

r ⇐= r − αqs⇐= M−1rδold ⇐= δnew

δnew ⇐= rT sβ ⇐= δnew

δold

d⇐= s+ βdi⇐= i+ 1

“s ⇐= M−1r”ù;�ý�õ:Ì©b¥úRè L�÷6,�:;�ý��±µ�Í×õ���M

ú½ê�°.

�è�;< B1�������4Ìú÷S �.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 64

Page 65: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

B4.���äää-���ááá(Newton-Raphson)���ªªªúúú���ÌÌÌ¢¢¢;;;���ýýý���ªªª���-ýýýõõõßßß(Fletcher-Reeves)

üüüxxxÍÍÍ���ªªªEEEPPPêêê

sq�¥½ f�©,è�>� x�6,�ªEPê�ª�/Â�Ä� imax,�ªEPê�ª�¡ò´Ì

X� ε < 1�L,�ä-�á�ª�/Â�Äç½� jmax,�ä-�᪡ò´ÌX� ε < 1�°.

i⇐= 0k ⇐= 0r ⇐= −f ′(x)d⇐= rδnew ⇐= rT rδ0 ⇐= δnew

While i < imax andδnew > ε2δ0 doj ⇐= 0δd ⇐= dT dDo

α⇐= − [f ′(x)]T ddT f ′′(x)d

x⇐= x+ αdj ⇐= j + 1

While j < jmax andα2δd > ε2

r ⇐= −f ′(x)δold ⇐= δnew

δnew ⇐= rT rβ ⇐= δnew

δold

d⇐= r + βdk ⇐= k + 1If k = n or rT d ≤ 0

d⇐= rk ⇐= 0

i⇐= i+ 1

�NLý ù/Â�Äç½ imaxõ#Y $#, ||r(i)|| ≤ ε||r(0)|| EÍ�[Ò¢°.

����ä-�á�Ä�° αdõ x�Ì¢°;��Äù αd�sq�´ÌX� ¿Fq�: (||αd|| ≤ ε)

Óù�Äç½� jmaxõ#Y£:[Ò¢°.�ò��ÙAÝ¢�xÒ_ù jmaxõ�ù÷¿PÌ $

#½èK(Hessian)±µ f ′′(x)õÂ��Û�÷¿�P �£½�°.

üxÍ�ªEPê�ªùÒ_�³� � ��³�I¨,÷¿GSþ:�°^¿è�ý° (d⇐=

r¿zA).�ù½ n�©½¶�êõm��æ©,� n¥��Ä�°�袥B&è�ý°.

α�GSù ‘0÷¿#��(divide-by-zero)’¡êõ�÷Ƚ�°.�,ùè�> x(0)�Ù �/�

�?Û&�²�M$# f� 2òyÛ��  �MúEÍ��f£½�°.;��EÍÌ#ùè�>

úxØ $#ÌÄ!¢�xÒ_úPÌ �©@£½�°.ó��EÍ���ªEPê�ª�/�Ü

�:¦¢NLý �I©½�°.

�è�;< B1�������4Ìú÷S �.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 65

Page 66: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

B5.£££xxxªªªúúú���ÌÌÌ¢¢¢;;;���ýýýmmm���-ýýýüüü���òòò(Polak-Ribiere)üüüxxxÍÍÍ���ªªªEEEPPPêêê

sq�¥½ f�©,è�>� x�6,�ªEPê�ª�/Â�Ä� imax,�ªEPê�ª�¡ò´Ì

X� ε < 1�L,£xª�<7�yâ� σ0,£xª�/Â�Äç½� jmax,£xª¡ò´ÌX� ε < 1�

°.

i⇐= 0k ⇐= 0r ⇐= −f ′(x)Calculate a preconditionerM ≈ f ′′(x)s⇐= M−1rd⇐= sδnew ⇐= rT dδ0 ⇐= δnew

While i < imax andδnew > ε2δ0 doj ⇐= 0δd ⇐= dT dα⇐= −σ0

νprev ⇐= [f ′(x+ σ0d)]T dDo

ν ⇐= [f ′(x)]T dα⇐= α ν

νprev−ν

x⇐= x+ αdνprev ⇐= νj ⇐= j + 1

While j < jmax andα2δd > ε2

r ⇐= −f ′(x)δold ⇐= δnew

δmid ⇐= rT sCalculate a preconditionerM ≈ f ′′(x)s⇐= M−1rδnew ⇐= rT sβ ⇐= δnew−δmid

δold

k ⇐= k + 1If k = n or β ≤ 0

d⇐= sk ⇐= 0

elsed⇐= s+ βd

i⇐= i+ 1

�NLý ù/Â�Äç½ imaxõ#Y $#, ||r(i)|| ≤ ε||r(0)|| EÍ�[Ò¢°.

���£xª�Ä�° αdõ x�Ì¢°;��Äù αd�sq�´ÌX� ¿Fq�: (||αd|| ≤ ε)Ó

ù�Äç½� jmaxõ#Y£:[Ò¢°.�ò��ÙAÝ¢�xÒ_ù jmaxõ�ù÷¿PÌ �½

±£½�°.7�yâ σ0ùé 59� σú@A 6£xªú�Ì¢/�Ü����³G�PÌý°.Ý

± 3ê,�7�yâ�½¶úæ©SAüqb£½ê�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 66

Page 67: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

m�-ýü�ò(Polak-Ribiere)β 7�yâ� δnew−δmid

δold¿ ��

rT(i+1)s(i+1)−rT

(i+1)s(i)

rT(i)s(i)

® �L, � ù Z¢rT(i+1)M

−1(r(i1)−r(i))

rT(i)M

−1r(i)�°.;�ý ±µ M� ¨\ j� AÙÒ(positive-definite)�êÀ s�õ �Ð�b ¢°.

;�ý��Û±µ�Í×õ���,ùI¦°.

� üxÍ �ª EPê �ªù m�-ýü�ò(Polak-Ribiere)β 7�yâ� ü½� :�° ^¿ è�ý°

(d⇐= r¿zA).�ù½ n�©½¶�êõm��æ©,� n¥��Ä�°�袥B&è�ý°.

üxÍ�ªEPê�ªù=��xØúC袰:;�ýõ£,��®�©�Mù�,�ä-�á(Newton-

Raphson)�ª, £xª(Secant method)Óù �Í� �ª, �ª�-ýõß(Fletcher-Reeves)Óù m�-ýü�

ò(Polak-Ribiere)#�xØ��Å.æ�tCèýxØú�©°j¢ºÍ��  °. (m�-ýü�ò(Polak-

Ribiere)õxØ �,ùsC#��� °.)

copyright c©Jonathan Richard Shewchuk and VisCom Commune 67

Page 68: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

C.888¢¢¢���<<<

C1. Ax = b���©©©������òòòÍÍÍéééúúú///���ÜÜÜèèèÇÇÇ°°°

A�Âa��L�A �. x� Ax = bõ�T �>�6,�òÍé (é 3)õ/�Ü �>��L L, e�

¡ê¨��L �.� 8°ü��T¢°.

f(x+ e) =12(x+ e)TA(x+ e)− bT (x+ e) + c (by Equation 3)

=12xTAx+ eTAx+

12eTAe− bTx− bT e+ c (by symmetry of A)

=12xTAx− bTx+ c+ eT b+

12eTAe− bT e

= f(x) +12eTAe.

A�j�AÙÒ(positive-definite)�L 8,���¨ù 0�I¨?� e� �¨\j(positive)�°;

0�t x� fÂ/�Ü¢°.

C2.ÂÂÂaaa±±±µµµùùù n���������pppLLLîîî¯âââõõõ������°°°

qE±µ��/�¢ #�Lî¯â(eigenvector)õ��°.�õÝ  �æ© det(A− λI)� λ�°¨é

���,�s@ 8,/�¢ #(|�)�\�©õ��°;�õ λA� �.±µ A−λAI�±µé� 0�

6,0�t"�±µ(singular)�t¿ A−λAI)v = 0 0�I¨¯â v���èU&©b¢°.Av = λAv�

t¿,�¯â v��¿Lî¯â�°.

?�Âa±µù n���pLî¯âõ��°.�õ�< �æ©, 4×4±µ�EÍ�©�,��

�¥úÃ�,�6,���¾�õ��±µ¿�<�¢��Ü�ë���3�£,�°.±µ A�/�

¢ #�Lî λA®Lî¯â võ��°�,ù�y�<ü}°. x1 = v/||v||�L 8��³æ£�õ

��°.t¿�p 6³æ£�õ������¯â x2, x3, x4õxØ � (� ¢¯â�ù��-Ùy!

�pÜõ�©üú½�°). X = [x1, x2, x3, x4]�L �. xi�A��p(orthonormal)�t¿ XTX = I�

6,XT = X−1�°.Z¢ i 6= 1�©, xTi Axi = xT

i Ax1 = xTi λAx1 = 0�t¿°üY�°.

XTAX =

xT

1

xT2

xT3

xT4

A[x1x2x3x4] =

xT1

xT2

xT3

xT4

[λAx1Ax2Ax3Ax4]

=

λA 0 0 000 B0

copyright c©Jonathan Richard Shewchuk and VisCom Commune 68

Page 69: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

�:,B� 3× 3Âa±µ�°.B�W λBõ �Lî¯â wõ�Kb�¢°.�: wõ�¥«Ù�

� 0�L#"�Ù�� w®ô�¢ 4�Ù�¯â�L �.� 8°ü�<� °.

X−1AXw = XTAXw =

λA 0 0 000 B0

w = λBw

°è©, AXw = λBXw�6,0�t Xw� A�Lî¯â�ý°.Z¢, xT1 Xw = [1000]w = 0�t¿,

x1Y Xw��p�°.0�t/�¢����pLî¯â�U&¢°!

Âa ±µ� n �� �p Lî¯âõ ��°� 4Ì� ÌÎ ��Üý �Áù �,ª÷¿ �<ý°.æ�

��t,��� 3×3±µ (B®�ù)� 3���pLî¯âõ��°L�A �;��ú�� w1, w2, w3�

�L �.� 8Xw1,Xw2,Xw3� A�Lî¯â��6,X����A��p�t¿�p¯â�ú�p

¯â¿��(mapping) t¿�Lî¯â��è�p�°.0�t A� 4���pLî¯âõ��°.

C3.���üüüËËË���(Chebyshev)°°°¨ééé���///:::���

�üË�(Chebyshev)°¨éùé50Y�ùvÇ�/�Ü�/:�°.���°¨é�©æ [−1, 1]�4Ù

�t¾�� 1úJ�MêÀC¢ý°¨é�|�t©æ [−1, 1]��t�$�ò3¾���� �°¨

é��:[�°.

ò½ i��üË�°¨éù°üY�L,

Ti(ω) =12[(ω +

√ω2 + 1)i + (ω −

√ω2 − 1)i],

��©æ [−1, 1]�©°üY��vÇþ½�°.

Ti(ω) = cos(i cos−1 ω), − 1 ≤ ω ≤ 1.

�vÇ�t (�ýL�� 32�t),�üË�°¨é�°üY�ù"�ú �°�,úN½�°.

|Ti(ω) ≤ 1, − 1 ≤ ω ≤ 1

�ýL -1Y 1P��t�ò3�ô¢°:

Ti

(cos(κπi

))= (−1)k, k = 0, 1, · · · , i.

Ti� i��©�©æ [−1, 1]4��� Ti� i+ 1���(extrema)P��n °.�õ�q T5(ω)� 5

�©��� 32®�°.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 69

Page 70: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

îP¢EÍ¿°ü¥½õÃ�.

Pi(λ) =Ti(λmax+λmin−2λ

λmax−λmin)

Ti(λmax+λmin

λmax−λmin)

æ� ¥½� A�� [λmin, λmax]� © ±Ti(λmax+λmin

λmax−λmin)−1� ©æ 4�t �ô¢°. Pi(λ)� Pi(0) =

1���ÅuS&ú�T¢°.

�<ù�³¢�põPÌ¢°.ò½ i °¨é÷¿t Qi(0) = 1�8t©æ [λmin, λmax]�© PiÃ

° #ù °¨é Qi(λ)� U& � M�°.�õ �< � æ©,� ¢ °¨é� U&¢°L �A �; � 

8 qjadnl [λmin, λmax]4�t Qi(λ) < Ti(λmax+lambdamin

λmax−λmin)−1�°.�,ùO Pi − Qi� λ = 0�t©õ

�°�,�°.0�t Pi −Qi� iò°¨é�8t/�¢ i+ 1��©õ��3ü�Ú,��Ý� ¢�

�°.� ¢?¿��© iò�üË�°¨é�é 50ú/:Ü¥ú�<£½�°.

D.���åååYYYCCC

°ü �[� ©, (°ô � {� ¢) � Û� AÝ¢ SÁú PÌ � Ùô�½> �¤� ¡ò� {°L

�A¢°.

1. (#�);�ýý�ªEPê�ªù;�ý±µ�ße�úU©ê�³ú��Müú�< �.

°è©,��ù 0�I¨\½ γ�©,;�ý±µ γMúPÌ �uù x(0), x(1), · · ·�;�ý±

µMúPÌ �uù³G�Yô�¥úÃ��.

2. (L�,� #�y¿Ï[C)Âa�6j�AÙÒ n× n±µ A�j¥ý Ax = b�©õu ²L

¢°L�A °.ݱ 3ê,xͽ½z�Aê:?<÷¿�ªEPê�ª�¢�r�#�

M�°.�:¾ê�L{úÆó¢Lî¯âÅA�#Í#t�¾ê�3±µ A���� d��t

¿°ôLî(eigenvalue)úN²s}° (Lî¯â�N²s�M�°).� #���Lî�v�

#|Äüq#Í#���N�G¢°.

¾êù[[¢P���t,¡�I^ ��t°üNLý�ú|v$ý3ü}°.

Choose an arbitrary starting pointx(0)

For i⇐= 0 to d− 1r(i) ⇐= b−Ax(i)

Remove an arbitrary eigenvalue from the list and call itλi

x(i+1) ⇐= x(i) + λ−1i r(i)

Lî��¥PÌü�EÍ�{÷6;[Òè�Lî@À(list)ùü3ý°.

(a) �NLý��[Ò£:, x(d)� Ax = b�©�ÿúÃ��. ¡(!¿??<ús�ª3�qÃ�.

2òÙ��½¶ú�²Ã�;Lî¯â9ú�ÿ�����Lîú$9xØ ²L©Ã�.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 70

Page 71: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

(NLý��X|ú ��õ�[:÷¿z< �,��$|Å °;�3�  8����

<��Å¢½éúCè �.)

(b) �NLý�� dç��Äú�©AÝ¢©õü�� ��,� Ûù�¥u3ü� x(i)��

 ¢¢ ©� �«Ï ú �õ Ù£ ,�°.� �Ä :�° Lî @À�t qE Lîú x

Ø � ,� ^ù�õ @A � Â�� ªYú Cè© Ã�. (°è©,qE ¿½¿ Lîú P

Ì©b ��?)

(c) Ùô�½>�¤�¡ê��q(EÍ�NLý�ùqN3~��þ½���?

(d) Ùô�½>�¤�¡ê�ÝÂü�,ú��æ©��Äè��°@À�tqELîúx

Ø©b ��Â���ªúCè �. ¡(!¿�[C�»ù[C (b)�»Yô� °.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 71

Page 72: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

÷÷÷LLL[[[¶¶¶

[1] Richard Barrett, Michael Berry, Tony Chan, James Demmel, June Donato, Jack Dongarra, Victor Eijkout,

Roldan Pozo, Charles Romine, and Henk van der Vorst.Templates for the Solution of Linear Systems:

Building Blocks for Iterative Methods. SIAM, Philadelphia, Pennsylvania, 1993.

[2] William L. Briggs. A Multigrid Tutorial. SIAM, Philadelphia, Pennsylvania, 1987.

[3] James W. Daniel. Convergence of the conjugate gradient method with computationally convenient modifica-

tions. Numerische Mathematik, 10:125–131, 1967.

[4] W. C. Davidon. Variable metric method for minimization.Technical Report ANL-5990 (Argonne National

Laboratory, Argonne, Illinois), 1959.

[5] R. Fletcher and M. J. D. Powell. A rapidly convergent descent method for minimization.Computer Journal,

6:163–168, 1963.

[6] R. Fletcher and C. M. Reeves. Function minimization by conjugate gradients.Computer Journal, 7:149–154,

1964.

[7] L. Fox, H. D. Huskey, and J. H. Wilkinson. Notes on the solution of algebraic inear simultaneous equations.

Quaterly Journal of Mechanics and Applied Mathematics, 1:149–173, 1948.

[8] Jean Charles Gilbert and Jorge Nocedal. Global convergence properties of conjugate gradient methods for

optimization.SIAM Journal on Optimization, 2(1):21–42, 1992.

[9] Gene H. Golub and Dianne P. O’Leary. Some history of the conjugate gradient and lanczos algorithms:

1948-1976.SIAM Review, 31(1):50–102, 1989.

[10] Magnus R. Hestenes. Iterative methods for solving linear equations.Journal of Optimization Theory and

Applications, 11(4):323–334, Originally published in 1951 as NAML Report No. 52–9, 1973.

[11] Magnus R. Hestenes and Eduard Stiefel. Methods of conjugate gradients for solving linear systems.Journal

of Research of the National Bureau of Standards, 49:409–436, 1952.

[12] Shmuel Kaniel. Estimates for some computational techniques in linear algebra.Mathematics of Computation,

20:369–378, 1966.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 72

Page 73: 고뇌의 고통이 없는 켤레 경사도 기법 개론(1과 1/4판)

VCTR-2006-F-002 Ls�L��{��ªEPê�ª�Á (1 14:)-��{†,�ô�‡,î�|††,�êõ‡‡ b

[13] John K. Reid. On the methods of conjugate gradients for the solution of large sparse systems of linear

equations. Large Sparse Sets of Linear Equations, pages 231–254, (John K. Reid, ed.), Academic Press,

London and New York, 1971.

[14] E. Schmidt. Title unknown.Rendiconti del Circolo Matematico di Palermo, 25:53–77, 1908.

[15] Eduard Stiefel.Uber einige methoden der relaxationsrechnung.Zeitschrift fur Angewandte Mathematik und

Physik, 3(1):1–33, 1952.

[16] A. van der Sluis and H. A. van der Vorst. The rate of convergence of conjugate gradients.Numerische

Mathematik, 48(5):543–560, 1986.

copyright c©Jonathan Richard Shewchuk and VisCom Commune 73