TRNG I HC KHOA HC T NHIN

KHOA CNG NGH THNG TIN

Chuyn ngnh Khoa hc My tnh

Lp Cao hc Kha 22

BO CO

MN MY HC

SUPPORT VECTORS MACHINE

TP.HCM, 2012

TRNG I HC KHOA HC T NHIN

KHOA CNG NGH THNG TIN

Chuyn ngnh Khoa hc My tnh

Lp Cao hc Kha 22

BO CO

MN MY HC

SUPPORT VECTORS MACHINE

T GIA DIU 1211013

NGUYN KHA 1211030

HUNH DUY KHOA 1211031

L ANH T 1211078

TRN THIN VN 1211084

GING VIN HNG DN

TS.TRN THI SN

i

MC LC

MC LC ................................................................................................................... 1

DANH MC CC HNH ........................................................................................... 3

DANH MC CC BNG ......................................................................................... 5

Phn 1 Support Vectors Classifier .............................................................................. 1

1.1. Tuyn tnh ............................................................................................. 1

1.1.1. Bi ton ......................................................................................... 1

1.1.2. Gii quyt...................................................................................... 2

1.1.3. Nhn t Largrange(Lagrange multiplier) ..................................... 6

1.1.4. Soft Margin ................................................................................... 7

1.2. SVC phi tuyn ....................................................................................... 8

1.2.1. Bi ton ......................................................................................... 8

1.2.2. V d minh ha ........................................................................... 10

1.2.3. Th thut Kernel ......................................................................... 11

1.2.4. Mt s v d hm kernel ............................................................. 13

1.3. Multiple Classification ........................................................................ 14

1.3.1. One Versus the Rest: .................................................................. 14

1.3.2. Pairwise Support Vector Machines ............................................ 17

1.3.3. Error-Correcting Output Coding (Thomas G.Dietterich,

Ghumlum Bakiri) ........................................................................ 19

1.4. Sequential Minimal Optimization (SMO) .......................................... 21

1.4.1. Tm kim theo hng (Direction Search) ................................... 22

1.4.2. Thut ton [6] ............................................................................. 23

ii

Phn 2 Support Vectors Regression .......................................................................... 24

2.1. Bi ton hi quy (Regression)............................................................. 24

2.2. Hm li ................................................................................................ 24

2.3. SVR ............................................................................................... 25

Phn 3 Th vin h tr lp trnh v cc hng pht trin......................................... 28

3.1. Cc th vin h tr lp trnh SVM ..................................................... 28

3.2. Mt s hng nghin cu ................................................................... 29

a. Hiu qu tnh ton ....................................................................... 29

b. La chn hm kernel .................................................................. 29

c. Hc SVM c cu trc ................................................................. 30

Ti liu tham kho ..................................................................................................... 32

iii

DANH MC CC HNH

Hnh 1-1 Cc dy c th chia tp d liu [1] .............................................................. 2

Hnh 1-2 S lc v hyperlane [1] .............................................................................. 2

Hnh 1-3 Minh ho Largrange Multiflier ................................................................... 6

Hnh 1-4 Vi im nhiu trong b d liu .................................................................. 7

Hnh 1-5 (a) SVC tuyn tnh vi Soft Margin. (b) SVC phi tuyn. ............................ 8

Hnh 1-6 Mt mt phn tch phi tuyn trong khng gian gi thuyt c th tr thnh

1 siu phng trong khng gian c trng. ................................................................... 9

Hnh 1-7 V d minh ha SVM phi tuyn. (a)Khng gian gi thuyt. (b)Khng gian

................................................................................................................................... 10

Hnh 1-8 Siu phng phn lp trong khng gian c trng ...................................... 11

Hnh 1-9 D liu phn b dng ng cong ............................................................. 14

Hnh 1-10 Cc classifiers ca OVR .......................................................................... 15

Hnh 1-11 Trng hp c nhiu fi(x) > 0 (chm hi ) v khng c f(i)>0 (chm

hi xanh). ................................................................................................................... 15

Hnh 1-12 : Minh ha fuzzy 1. .................................................................................. 16

Hnh 1-13 Cc ng en l kt qu ca s dng fuzzy. ......................................... 16

Hnh 1-14 Cc classifiers ca Pairwise. .................................................................... 17

Hnh 1-15 Trng hp c nhiu i tha mn iu kin chn lp ............................... 18

Hnh 1-16 Cy DDAG .............................................................................................. 18

Hnh 1-17 Kt qu DDAG ........................................................................................ 19

Hnh 1-18 Kt qu fsvm. ........................................................................................... 19

Hnh 1-19 Kt qu codewords................................................................................... 20

Hnh 1-20 Hamming decoding .................................................................................. 21

iv

Hnh 1-21 Cho im bt u v mt hng u kh thi, tm kim theo hng s cc

i ha hm , vi vi lun tha iu kin KKT

[6] .............................................................................................................................. 22

Hnh 2-1 ng hi quy ca tp im cho trc. .................................................... 24

Hnh 2-2 Nhn thy cc t trng cho ngng li, c hm tri u, c hm phi n

ngng mi tnh li, c hm tuyn tnh, c hm phi tuyn. ..................................... 25

Hnh 2-3 Hm li ................................................................................................... 26

Hnh 3-1 Phn lp bng LibSVM ............................................................................. 28

v

DANH MC CC BNG

Bng 1-1 Mt s thut ton ti u tm [6] ........................................................ 21

1

Phn 1

Support Vectors Classifier

- SVM Support Vectors Machine l mt m hnh hc c gim st, trong lnh

vc my hc.

- SVM thng c s dng phn lp d liu (classification), hoc phn

ch hi quy (regression annalysis). L nn tng cho nhiu thut ton trong

khai thc d liu.

- SVM c gii thiu vi Vladimir Vapnik v cc ng s vo nm 1979.

Paper c cng b chnh thc vo nm 1995.

- tng chnh ca phng php SVM l phn chia b d liu vo cc phn

lp bng siu phng (hyperlane). T tng chnh, nhiu phng php ci

tin tu bin t phng php nguyn thu, cho nhiu trng hp s dng

khc nhau.

1.1. Tuyn tnh

Vi b d liu c th chia thnh 2 lp mt cch tuyn tnh (two-class linearly

separable data), SVC s tm ra mt siu phng (hyperlane) chia b d liu

bng cch cc i ho bin (Maximal Margin).

1.1.1. Bi ton

- Cho tp im D = {(xi, yi)} , D l mt tp c th phn lp tuyn tnh

(separable data set). Trong , l vector i din cho phn t th

i ,vi mi s c nh nhn { }.

- Bi ton t ra l tm dy rng nht c th phn chia tp d liu

ban u thnh 2 lp

2

Hnh 1-1 Cc dy c th chia tp d liu [1]

1.1.2. Gii quyt

- Trong khng gian 2 chiu, mt dy phn cch c th c biu din

bng 1 ng thng (l ng thng chnh gia dy - hyperlane)

- Hyperlane c biu din di phng trnh

Hnh 1-2 S lc v hyperlane [1]

3

l vector php tuyn ca hyperlane

- l khong cch gia 2 l ca dy phn cch. Nh vy, dy phn

cch rng nht s l dy c ch s ln nht.

- c tnh bng 2 ln khong cch ngn nht t cc im trong tp

d liu n hyperlane (xem hnh Error! Reference source not

found.). Cc im gn hyperlane nht l nhng im nm trn 2 l, s

c ngha cho vic tnh khong cch , c gi l Support Vector.

thit lp cng thc tnh trc ht, cn tnh c khong cch

t 1 im trong tp d liu n hyperlane.

- Ta c unit vector :

- Gi x l mt im nm trn hyperlane, x l mt im trong tp d

liu, ta c biu thc sau :

suy ra

- V x nm trn hyperlane, nn

(

)

suy ra

- r l khong cch t mt im x trong tp d liu n hyperlane (cn

c gi l geometric margin)

4

- Gi

- Gi r l khong cch ngn nht t tp d liu n hyperlane, khi ,

ta lun c :

- Nhn thy, gi tr r c lp vi b ( ,b),

Vy, c th chn sao cho . Khi , biu

thc trn c thu gn nh sau :

c gi l functional margin

*** functional margin c ln ph thuc vo t l scale ca vector ,

cn geometric margin th c lp vi t l ny.

*** Ch , ta c . iu ny khng thay i hyperlane

(phng trnh v hon ton tng ng. Mc ch rt

gn ny ch nhm mc ch d dng cho vic tnh ton v sau.

T , cho mt mu n hyperlane, ta tnh c :

Geometric margin

Functional margin

Suy ra

5

T , tm c biu thc cho margin :

Nh vy, tm dy phn cch rng nht, cng ng ngha vic tm margin

ln nht. Tr thnh bi ton cc i ho , hay cc tiu ho

(primal

problem) :

{

gii quyt bi ton ny, ta p dng nhn t largrange (Lagrange

multipliers)

[ ]

|w| cc tr khi L t cc tr trn w v b

{

Tng ng

{

thay vo biu thc Lagrange pha trn v tip tc tm cc tr trn ,

a bi ton v dng dual problem

Thay w =

6

ma

Vi rng buc

i n

Vn Convex quadratic programming optimization

iu kin Karush-Kuhn-Tucker (complementary slackness conditions):

[ ] i n

Gii bng thut ton Sequential Minimal Optimization (SMO) tm

b . T , tm c w,b bng cc biu thc pha trn.

*** Sau khi gii xong bi ton ti u ho, ta tm c b , th ch c

nhng gi tr tng ng vi nhng im nm trn 2 bin (Support

Vector) mi c gi tr khc 0, cn gi tr nhng im cn li iu

bng 0.

1.1.3. Nhn t Largrange(Lagrange multiplier)

- Gi thit :

{

- Xt rng buc

Vector php tuyn

Vector tip tuyn

Suy ra Hnh 1-3 Minh ho Largrange

Multiflier

7

Khi , f(x,y) s t cc tr khi .

{

v song song

Suy ra

Vit li, cch khc

Biu thc trn c ngha, f(x,y) t cc tr th

*** C th tm hiu thm v nhn t Largrange trong cc ti liu [2] [3]

1.1.4. Soft Margin

Trong thc t, tp d liu khng lun tho tnh cht separable (phn chia c),

m i khi, c mt vi im b nhiu (inseparable points) . V vy, t phng php

nguyn thu ban u, cn thm mt vi ci tin chp nhn nhng im nhiu ny.

Hnh 1-4 Vi im nhiu trong b d liu

8

chp nhn cc nhiu, cng thc ban u c tu chnh thm vi tham s

min

Trong :

Slack variable m t nhiu cho phn lp .

C nh hng ca li (C cng ln th mc nh hng ca cc

im nhiu cng ln, ngc li C cng nh, th iu kin ni lng hn,

d dng chp nhn nhiu hn).

Rng buc functional margin s c dng :

,

1.2. SVC phi tuyn

1.2.1. Bi ton

Phn trc trnh by v phng php SVC tuyn tnh vi tng chnh l tm

mt siu phng vi margin ln nht phn tch d liu. Tuy nhin, bi ton ti u

ch gii c nu tp d liu phn tch tuyn tnh c. Trong thc t, tp d liu

a s l khng phn chia tuyn tnh c bng siu phng (Hnh 1-5).

Hnh 1-5 (a) SVC tuyn tnh vi Soft Margin. (b) SVC phi tuyn.

9

gii quyt vn d liu khng kh tch tuyn tnh, chng ta c hai cch tip

cn. Cch tip cn th nht ( c trnh by) l s dng phng php cc i ha

bin mm (soft margin) nh Hnh 1-5(a). Tuy nhin, trong cc trng hp d liu

nh Hnh 1-5(b) th cch tip cn bin mm khng kh dng.

Trong trng hp , chng ta phi s dng mt cch tip cn khc. tng chnh

ca hng tip cn ny l nh x cc mu trong khng gian ban u (khng gian

gi thuyt) sang 1 khng gian c s chiu ln hn (khng gian c trng). Sau p

dng phng php SVM tuyn tnh tm ra 1 siu phng phn hoch trong khng

gian c trng. Siu phng ny s ng vi mt phi tuyn trong khng gian gi

thuyt. Hnh 1-6 minh ha cho phng php ny. Ta thy, d liu, sau khi c nh

x t khng gian 2 chiu sang khng gian 3 chiu, c th phn tch hon ton bng

1 siu phng trong khng gian mi.

Hnh 1-6 Mt mt phn tch phi tuyn trong khng gian gi thuyt c th tr thnh 1

siu phng trong khng gian c trng.

10

1.2.2. V d minh ha

Hnh 1-7 V d minh ha SVM phi tuyn. (a)Khng gian gi thuyt. (b)Khng gian

Gi s ta c tp d liu nh Hnh 1-7(a). Mc tiu t ra l tm 1 siu phng phn

lp d liu chnh xc. Ta thy trong khng gian gc, tp d liu ny khng th phn

tch tuyn tnh. Do , ta phi nh x vo khng gian c trng mi. Gi s ta c

hm nh x nh sau

( ) {

(

)

( )

D liu trong khng gian gi thuyt sau khi nh x vo khng gian c trng mi s

nh Hnh 1-7(b). Sau khi nh x, ta s s dng phng php SVC tuyn tnh tm

siu phng phn lp trong khng gian c trng. Quay li bi ton SVC tuyn tnh,

v thay cc gi tr bng . Tnh tng t nh trng hp phi tuyn ta c

w = (1,1) v b = -3. Th vo phng trnh siu phng ta c siu phng: y = 3 x

nh hnh Hnh 1-8. Sau khi c c phng trnh siu phng, ta s s dng n

phn lp tng t nh trng hp tuyn tnh.

11

Hnh 1-8 Siu phng phn lp trong khng gian c trng

1.2.3. Th thut Kernel

Tuy nhin, cch tip cn trn gp phi 1 vn l khng gian c trng c th c s

chiu ln hn rt nhiu khng gian gi thuyt (c th v hn chiu), v do tn

nhiu thi gian tnh ton. Tuy nhin, ta thy rng trong cc php tnh, (x) ch xut

hin di dng tch v hng tc l dng (x)(y) m khng xut hin n l. V

vy, thay v s dng dng tng minh ca (x) th ch cn s dng hm biu din

gi tr v hng (x)(y).

t K(x,y)=(x)(y), K(x,y) c gi l hm ht nhn (kernel function) [4].

Nh vy l ch cn bit dng ca hm ht nhn K(x,y) m khng cn bit c th nh

x (x). Lc hm phn lp tr thnh:

12

Tuy nhin khng th chn ty hm K(x,y) m phi chn K(x,y) sao cho tn ti mt

hm m K(x,y) = (x)(y). iu kin bo m vn ny l iu kin Mercer.

Sau y l 1 v d v K v :

Vi x = (x1,x2) R2, (x) = (1, , ,

, , ) R

6 th

K(x,y) = (x)(y) = (1+x.y)2.

nh x v khng gian ban u: Sau khi gii bi ton phi tuyn, ta c c siu

phng phn lp trong khng gian c trng. Da vo phng trnh siu phng ta

xc nh c cc im support vector. Sau , nh x cc support vector ny v

khng gian gi thuyt. Cui cng, t cc im support vector ta xc nh c

ng phn lp trong khng gian ban u.

Vn quan trng l tm hm kernel K(x,y) nh th no: R rng y l mt

vn ph thuc vo bi ton nhn dng. i vi nhng bi ton nhn dng n

gin trong s phn b cc mu ca 2 lp khng qu phc tp th c th tm hm

K(x,y) n gin sao cho s chiu ca l khng qu ln.

Cch xy dng mt hm kernel: Hm kernel phi tha mn nh l Mercer.

nh l ca Mercer [4] Cho K(x, x) l mt kernel i xng lin tc c xc nh

trong khong cch ng a x b v tng t cho x. Kernel K(x, x) c th c

m rng trong chui:

vi cc h s dng, i > 0 vi tt c i. S m rng ny l c c s v hi t, n l

cn thit v iu kin

gi cho tt c cc (-) m

13

C th tm tt cc c im hu ch nht trong vic xy dng kernel, m c gi

l Mercer kernel. l, i vi bt k tp hp con gii hn ngu nhin no ph

thuc vo khng gian u vo X, ma trn tng ng c xy dng bi hm kernel

K(x, x): l mt ma trn i xng v bn xc nh dng

(tr ring ca ma trn >= 0), c gi l mt ma trn Gram [5].

1.2.4. Mt s v d hm kernel

c tnh ca hm kernel:

Nu K1(x,y), K2(x,y) l cc hm kernel th K3(x,y) cng l hm kernel vi:

1. K3(x,y) = K1(x,y) + K2(x,y) [**]

2. K3(x,y) = aK1(x,y) vi a > 0 [**]

3. K3(x,y) = K1(x,y).K2(x,y) [**]

4. K3(x,y) = aK1(x,y) + bK2(x,y) vi a,b > 0 [**]

T cc cng thc trn c th suy ra mt s hm kernel thng dng nh sau:

1. Linear: ( )

2. Hm a thc bc p: ( ) ; ( )

3. Hm Gaussian (Radial-basis function):

( )

Trong trng hp no nn s dng hm kernel no l ty thuc vo s phn b ca

d liu. V d, nu d liu phn b dng ng cong nh Hnh 1-9 th chng ta phi

dng hm nhn a thc dng K(x,y)=(1+x.y)p.

14

Hnh 1-9 D liu phn b dng ng cong

Chiu khng gian c trng ng vi kernel ny l d =

. Kernel ny c th

chuyn tt c cc mt cong bc p trong khng gian Rn thnh siu phng trong khng

gian c trng.

Tm li phng php SVM phi tuyn l tm mt hm kernel K(x,y), sau gii bi

ton SVM tuyn tnh vi vic thay x1x2 = K(x1,x2) tm ra w v b.

1.3. Multiple Classification

1.3.1. One Versus the Rest:

1.3.1.1. Continuous Decision Functions

Hun luyn

Vi mt tp m class, xy dng mt tp classifier D1, ..., Dm. Mi mt fi s phn

bit 1 lp vi cc lp cn li.

15

Hnh 1-10 Cc classifiers ca OVR

Nh vy ta sau khi hun luyn ta s c tp D1,..,Dm classifier nhn din.

Nhn din:

ng vi mi gi tr x cn phn lp:

- x s thuc lp i nu c duy nht mt Di(x) > 0.

- Trng hp c nhiu hn mt Di(x) > 0 hoc khng c Di(x) >0, ta s dng

cch tnh:

Ngha l x thuc lp i vi Di(x) ln nht.

Hnh 1-11 Trng hp c nhiu fi(x) > 0 (chm hi )

v khng c f(i)>0 (chm hi xanh).

16

1.3.1.2. Fuzzy Support Vector Machines (FSVM):

S dng fuzzy gii quyt trng hp khng nhn din c khi c nhiu hn

mt Di(x) > 0 hoc khng c Di(x) >0.

S dng 2 fuzzy:

1. For i = j: mii(x)= { o i i o i

2. For i = j: mij(x)= { o i i o i

Hnh 1-12 : Minh ha fuzzy 1.

Sau khi tm c mij(x), ta tm mi(x): mi(x) = min mij(x) , j =1,...,n

x s thuc v lp: arg max mi(x), i=1,...,n

Hnh 1-13 Cc ng en l kt qu ca s dng fuzzy.

17

==> FSVM v SVMs with Continuous Decision Functions l tng ng.

1.3.2. Pairwise Support Vector Machines

Hun luyn

Vi mt tp m class, xy dng mt tp classifier D1, ..., Dn(n-1)/2. Mi mt fi s

phn bit 1 lp vi 1 lp khc trong tp.

Hnh 1-14 Cc classifiers ca Pairwise.

Nh vy sau khi hun luyn ta s c n(n-1)/2 classifier nhn din.

Nhn din:

nhn bit x thuc v lp no, ta tnh tt c cc gi tr Di(x), i=1,...,n(n-1)/2. x s

thuc v lp:

Di(x) =

arg max Di(x) , i=1,...,n.

Tuy nhin, s xy ra trng hp c nhiu i tha mn iu kin chn lp, khi m x

thuc v phn gch cho trong trng hp di.

18

Hnh 1-15 Trng hp c nhiu i tha mn iu kin chn lp

C nhiu phng php x l trng hp ny:

Decision Directed Acyclic Graph (Platt, Cristianini, and J. Shawe-Taylor)

Phng php ny xy dng mt cy quyt nh quyt nh da trn ph nh ca

i. Nh vy s khng gp trng hp khng th phn lp nh trn.

Hnh 1-16 Cy DDAG

Kt qu ca DDAG :

19

Hnh 1-17 Kt qu DDAG

Fuzzy SVM

Tng t trng hp OVR.

Hnh 1-18 Kt qu fsvm.

1.3.3. Error-Correcting Output Coding (Thomas G.Dietterich, Ghumlum

Bakiri)

Hun luyn

tng ca phng php ny l to ra cho mi lp trong m class mt chui nhi

phn duy nht gi l "codewords" t n hm phn lp f.

Mi hm phn lp f c xc nh da vo tng c s d liu.

V d: trong bi ton phn lp cc ch s t 0-9. Ta c th c cc hm f nh sau:

f1: phn lp 1,4,5.

f2: phn lp 1,3,5,7,9.

...

Sau khi xc nh cc hm phn lp f da vo d liu, ta tin hnh to cc

codewords cho cc lp. V d kt qu codewords:

20

Hnh 1-19 Kt qu codewords.

Lu : trng hp cc codewords c khong cch khc bit qu nh, th thng

thng s nh ngha thm ct (hm phn lp f) tng khong cch ch gia cc

codeword ny

Nhn din

Vi gi tr x cn nhn din, ta xc nh codeword ca x vi ci hm f xc nh.

Sau khi c codeword ca x ta so vi codewords ca d liu xc nh codeword

ca x gn vi lp i nht th x s thuc v lp i.

so snh codeword ca x vi codewords cc lp ta dng khong cch Hamming

(m s lng cc cc bit khc nhau).

1.3.3.1. Mt ci tin ca EOCO (Erin L. Allwein, Robert E. Schapire, Yoram Singer):

Thay v s dng codeword l 0,1 th tc gi s dng cc gi tr codeword l -1,0,+1.

Trong gi tr 0 biu din rng ta khng quan tm n hm phn lp f i vi lp.

xc nh gi tr codeword gn vi gi tr codewords lp no nht th tc gi s

dng phng php Hamming decoding:

dH(M(r), f (x)) = 1/2

Bn di l 1 v d tnh Hamming decoding:

21

Hnh 1-20 Hamming decoding

1.4. Sequential Minimal Optimization (SMO)

Nu nh gii h phng trnh theo phng php trn th s rt tn chi

ph v cc im khng phi l Support Vector s c gi tr xp x 0.

Cn c thut ton vi chi ph thp hn ti u ha bi ton cng vi cc

h s .

Bng 1-1 Mt s thut ton ti u tm [6]

Ra i nm 1999, l thut ton hiu qu u tin dng hun luyn tp d liu, ti

u cc h s (Trc c thut ton Chunk ca Vapnik v Chervonenkis).

Bi ton ban u :

Trc khi tm hiu thut ton, ta i vo mt khi nim mi :

22

1.4.1. Tm kim theo hng (Direction Search)

Gi s ta c tha mn iu kin ti u ca hm i ngu D().

Ta gi u={u1, u2, , un} l mt hng kh thi nu ta c th thm vo mt on

th vn tha mn D() ( ).

Gi l tt cc cc h s tha iu kin trn, nh vy [ ].

Nh vy, ta c c bi ton ti u .

ma

Hnh 1-21 Cho im bt u v mt hng u kh thi, tm kim theo hng s cc

i ha hm , vi vi lun tha iu kin KKT [6]

Xt hm D(+ ) l hm li.

Nh vy, theo cng thc Newton, vi s thay i rt nh, ta c th tnh c

nh sau [6] :

23

1.4.2. Thut ton [6]

Tm tt s lc :

u tin khi to cc tr s , v gk.

ng vi mi bc lp, tm hai ch s ca sao cho chng xa nhau nht. Sau

tin hnh tm kim c hng trn hai gi tr , ri tin hnh cp nht

li trng s ca o hm .

24

Phn 2

Support Vectors Regression

2.1. Bi ton hi quy (Regression)

Bi ton hi quy l bi ton kh ph bin, mc tiu l tm hm c trng cho mt

tp im ri rc ban u.

Hnh 2-1 ng hi quy ca tp im cho trc.

Cho tp im :

Hm hi quy s c dng :

[7]

Mc tiu l xy dng hm f vi li l thp nht.

2.2. Hm li

nh gi li cho hi quy, ngi ta a ra cc hm li. Dng tng qut :

[7]

Trong :

P l o xc sut trn tng quan ton b tp d liu

25

c(f,x,y)=(f(x) y2)

Mt s hm li thng dng :

Hnh 2-2 Nhn thy cc t trng cho ngng li, c hm tri u, c hm phi n

ngng mi tnh li, c hm tuyn tnh, c hm phi tuyn.

2.3. SVR

Hm mc tiu ti u s l :

26

Hnh 2-3 Hm li

[7]

Hm Lagrange

[7]

iu kin

Thay th cc iu kin trn vo ta hm i ngu

[7]

27

Gi tr c thay th v bin mt v ta c

iu kin KKT [7]

p dng cc iu kin trn, ta gii h phng trnh c cc * *

b c tnh theo cng thc

28

Phn 3

Th vin h tr lp trnh v cc hng pht trin

3.1. Cc th vin h tr lp trnh SVM

Hai th vin ph bin ci t thut ton SVM l LibSVM v SVMlight.

Trong LibSVM h tr nhiu nn tng h iu hnh khc nhau, cng nh

nhiu ngn ng lp trnh khc nhau (C++, Java) d dng ci tin v ng

dng. c bit LibSVM cho php tinh chnh nhiu tham s hn mt s phn

mm hoc th vin khc v h tr nhng b tham s mc nh h tr gii

quyt nhiu vn thc t mt cch hiu qu.

Hnh 3-1 Phn lp bng LibSVM

SVMlight l mt bn ci t khc ca SVM bng ngn ng C. SVMlight

thng qua k thut chn la hiu qu v kh thi nht v hai phng php tnh

ton hiu qu l nn v cache ca nh gi kernel. SVMlight gm hai

chng trnh C chnh: SVM_learn s dng hun luyn b phn lp v

SVM_classify kim chng.

29

Ngoi ra cn mt s b cng c c giao din hoc trc quan h tr thut

ton SVM nh Torch (C++), Spider (MATLAB) hay Weka (Java).

3.2. Mt s hng nghin cu

Trong mt thp k qua SVM pht trin rt nhanh v c l thuyt ln ng

dng nhng vn cn rt nhiu hng nghin cu trin vng. Sau y l mt s

hng chnh.

a. Hiu qu tnh ton

Mt trong nhng nhc im t u ca SVM l chi ph tnh ton cho

bc hun luyn ln, dn n kh p dng cho nhng b d liu ln. Tuy

nhin, vn ny c gii quyt thnh cng. Mt cch tip cn l

chia nh bi ton ti u ha thnh nhng bi ton nh hn m mi bi

ton ny ch lin quan n mt s bin c chn lc v th ti u ha

c gii quyt mt cch hiu qu. Tin trnh lp li cho n khi nhng

bi ton nh c gii quyt hon tt.

Mt vn gn y hn ca my hc SVM l tm mt mt cu ti tiu

bao quanh cc thc th. Nhng thc th khi nh x vo khng gian N-

chiu th hin mt li c th dng xy dng mt mt cu ti tiu bao

quanh. Gii quyt bi ton SVM trn nhng tp li s cho gii php tt

gn ng vi tc rt nhanh.

b. La chn hm kernel

Khi s dng hm kernel trong SVM, vic la chn hm kernel mt cch

tng qut phi tha nh l Mercer. Do , nhng hm kernel ph bin

thng thuc v mt trong ba loi: hm sigmoid, hm a thc v hm bn

knh c s. Gn y Pekalska v cc cng s a ra quan im mi v

vic thit k hm kernel da trn nh x mi lin quan gn. Nhng hm

kernel mi ny khng tha iu kin Mercer a ra cng nh khng gii

hn trong mt khng gian c trng v thc nghim ban u cho thy

30

hiu qua tt hn nhng hm kernel Mercer. Tuy nhin, nn tng l thuyt

ca th h hm kernel mi ny cn c nghin cu su hn na.

Ngoi ra, mt cch tip cn khc l s dng nhiu kernel hn l ch mt.

Thng qua s kt hp c th t c kt qu tt hn. Bng vic xc

lp ng hm mc tiu, vic la chn cc tham s cho kernel c th hin

thc cho php s dng nhiu kernel.

c. Hc SVM c cu trc

Cc i ha bin gia cc lp l ng lc ban u ca SVM. iu ny

dn n SVM tp trung vo vic phn tch cc lp ca mu hc nhng

khng quan tm n s phn b ca d liu trong tng lp. nh l

Khng c ba n tra min ph pht biu rng khng tn ti phng

php phn lp mu no tuyt i u th hn nhng phng php khc

hoc thm ch l so vi vic on m mt cch ngu nhin. Thc t cho

thy, ty vo tng bi ton, mi lp khc nhau c th c cu trc khc

nhau. B phn lp phi cn chnh ng bin sao cho khp vi cu trc

ca chng, c bit cho vn tng qut hot ca b phn lp. Tuy nhin,

SVM ban u khng quan tm n cu trc, dn n vic xc nh siu

phng phn cch mt cch cng nhc ngay gia nhng support vector,

dn n b phn lp khng ti u ha cho cc vn thc t.

Gn y, mt s thut ton c pht trin quan tm n cu trc ca

thng tin hn SVM ban u. Chng mang li mt quan im mi v b

phn lp, khi m n c th cm c cu trc ca s phn b d liu.

Nhng thut ton ny c chia thnh hai cch tip cn. Cch tip cn

th nh l manifold learning. Gi thit rng d liu thc t nm trong

nhng submanifold trong khng gian u vo, v thng thng cc thut

ton lin quan n Laplacian Support Vector Machine (LapSVM). Xy

dng LapSVM u tin thng qua th Laplacian trong mi lp. Sau ,

31

to ra cu trc manifold ca d liu tng ng vi ma trn Laplacian

trong SVM truyn thng.

Cch tip cn th hai l khai thc cc thut ton phn cm vi gi nh d

liu c cha nhiu cm vi thng tin phn phi. Gi nh ny dng nh

tng qut hn gi nh manifold. Mt cch tip cn gn y c bit n

l Structureed Large Margin Machine (SLMM). SLMM ng dng k

thut gom cm ly c thng tin v cu trc vo trong cc rng buc.

Mt s Large Margin Machines ph bin l Minimax Probibility Machine

(MPM), v Maxi-min Margin Machine (M4) c th xem nh mt dng

c bit ca SLMM. Thc nghim cho thy SLMM c kh nng phn

lp tt hn. Tuy nhin bi ton ti u ha ca SLMM l Second Order

Cone Programming (SOCP) thay v SP ca SVM, SLMM c chi ph tnh

ton cao hn bc hun luyn khi so snh vi SVM truyn thng. Hn

na khng n gin tng qut ha hay bi ton nhiu lp. T , mt

SVM c cu trc mi (SSVM) c pht trin. Kt qu l bi ton ti

u ha c th c gi vi QP nh SVM, v d dng m rng. Hn na,

SSVM cho thy v mt l thuyt v thc nghim tt hn SVM v

SLMM trong vn tng qut ha.

32

Ti liu tham kho

[1] Prabhakar Raghavan & Hinrich Schtze Christopher D. Manning. Standford.

[Online]. http://nlp.stanford.edu/IR-book/html/htmledition/support-vector-

machines-the-linearly-separable-case-1.html

[2] Jeff Knisley. Multivariable Calculus Online. [Online].

http://math.etsu.edu/multicalc/prealpha/Chap2/Chap2-9/index.htm

[3] Yan-Bin Jia, "Lagrange Multipliers," Nov 27, 2012.

[4] Hui Xue, Qiang Yang, and Songcan Chen. Chapter 3: Support Vector Machines.

[5] http://www.encyclopediaofmath.org/index.php/Gram_matrix.

[6] Lin, Leon Bottou, Chih-Jen, "Support Vector Machine Solvers".

[7] Bernhard Schlkopf, Alexander J. Smola, Learning with Kernels: Support Vector

Machines, Regularization, Optimization, and Beyond., 2001.