8 machine learning

Gii thiu hc my

Ng Xun Bch

Hc vin Cng ngh Bu chnh Vin thng Khoa Cng ngh thng tin 1

Nhp mn tr tu nhn to

Ni dung

http://www.ptit.edu.vn 2

Gii thiu

Hc cy quyt nh

Phn loi Bayes n gin

Hc da trn v d

Ti liu tham kho


N. Nilsson. Introduction to machine learning http://ai.stanford.edu/people/nilsson/mlbook.html

T. Mitchell. Machine learning. McGraw-Hill, 1997.

E. Alpaydin. Introduction to machine learning. MIT Press, 2004.

M. Mohri, A. Rostamizadeh, A. Talwalkar. Foundations of Machine Learning. MIT Press, 2012.

Cng c v d liu


B cng c Weka o http://www.cs.waikato.ac.nz/~ml/weka

Kho d liu mu UC Irvine o http://www.ics.uci.edu/~mlearn/ML/Repository.html

Mt s ng dng ca hc my (1/3)


Nhng ng dng kh lp trnh theo cch thng thng do khng tn ti hoc kh gii thch kinh nghim, k nng ca con ngi o Nhn dng ch vit, m thanh, hnh nh

o Li xe t ng, thm him sao Ho

Chng trnh my tnh c kh nng thch nghi: li gii thay i theo thi gian hoc theo tnh hung c th o Chng trnh tr gip c nhn

o nh tuyn mng



Khai ph (phn tch) d liu o H s bnh n tri thc y hc

o D liu bn hng quy lut kinh doanh



Hu ht cc ng dng tr tu nhn to ngy nay c s dng hc my

Hc my l g?


Hc: o thu thp kin thc hoc k nng

Hc my: o Gii quyt vn t kinh nghim

o c thc hin bi chng trnh my tnh c kh nng: Thc hin cng vic tt hn

Theo tiu ch

Nh s dng d liu mu hoc kinh nghim

V d


Hc nh c o : nh c

o : s vn thng

o : kinh nghim t chi

Hc nhn dng ch o : nhn dng ch ci t nh

o : phn trm ch nhn dng ng

o : nh s ca ch v ch tng ng

Dch my o : dch mt cu ting Anh sang ting Vit

o : o dch my (v d s cu ng, s mnh ng,)

o : cp cu ting Anh v ting Vit tng ng

Vn cn quan tm (1/2)


Kinh nghim c th nh th no? o Kinh nghim trc tip v gin tip

Trc tip: trng thi c th + nc i ng tng ng

Gin tip: ton b vn c v kt qu

o C gim st (hng dn) v khng gim st C gim st

Khng gim st

Bn gim st

Cn phi hc ci g? Biu din kin thc hc c th no?

o Tri thc cn hc c biu din nh mt hm ch, cn la chn hm ch c th

o V d nh c:

Chn_nc_i:

im_s:

Vn cn quan tm (2/2)


S dng thut ton g hc? o S dng hm

VD: _ = 11+22+ 33+ 44+

o S dng cc lut

o S dng mng n ron

o S dng cy quyt nh

o S dng cc m hnh xc sut

o

Thit k chng trnh hc my


La chn d liu hoc kinh nghim

La chn hm ch

La chn biu din cho hm ch

La chn thut ton hc

Tin hnh hc (hun luyn)

Mt s khi nim


Mu, hay v d (samples): l i tng cn x l (v d phn loi) o V d: khi lc th rc th mi th l mt mu

Mu thng c m t bng tp thuc tnh hay c trng (features) o V d: trong chun on bnh, thuc tnh l triu chng ca ngi

bnh, v cc tham s khc nh chiu cao, cn nng,

Nhn phn loi (label): th hin loi ca i tng m ta cn d on o V d: nhn phn loi th rc c th l rc hoc bnh thng

Mt s dng hc my ph bin


Hc c gim st (supervised learning) o Phn lp (classification)

o Hi quy (regression)

Hc khng gim st (unsupervised learning) o Hc lut kt hp (association)

o Phn cm (clustering)

Hc tng cng (reinforcement learning)

Phn lp


chiu cao

cn nng

Hi quy (regression)


Ti sn

Tui th = +

ng dng: d on gi c, li xe,

Hc lut kt hp


V d o Phn tch giao dch, mua bn (ha n mua hng)

(|) o Xc sut ngi mua hng cn mua hng

V d lut kt hp o Ngi mua bnh m thng mua b

o Ngi mua lc rang thng mua bia

Phn cm


Nhm nhng trng hp tng t vi nhau

Khng c gi tr u ra

ng dng o Phn cm khch hng, phn cm sinh vin

o Phn on nh

o Thit k vi mch

Hc tng cng


Kinh nghim khng c cho trc tip di dng u vo / u ra

H thng nhn c mt gi tr thng (reward) l kt qu cho mt chui hnh ng no

Thut ton cn hc cch hnh ng cc i ha gi tr thng

V d: hc nh c o H thng khng c ch cho nc i no l hp l cho tng tnh

hung c th

o Ch bit kt qu thng thua sau mt chui nc i

Ni dung


Gii thiu

Hc cy quyt nh (decision tree learning)

Phn loi Bayes n gin

Hc da trn v d

D liu hun luyn


Ngy Tri Nhit m Gi Chi tennis D1 nng nng cao yu khng

D2 nng nng cao mnh khng

D3 u m nng cao yu c

D4 ma trung bnh cao yu c

D5 ma lnh bnh thng yu c

D6 ma lnh bnh thng mnh khng

D7 u m lnh bnh thng mnh c

D8 nng trung bnh cao yu khng

D9 nng lnh bnh thng yu c

D10 ma trung bnh bnh thng yu c

D11 nng trung bnh bnh thng mnh c

D12 u m trung bnh cao mnh c

D13 u m nng bnh thng yu c

D14 ma trung bnh cao mnh khng


Ngy Tri Nhit m Gi Chi tennis D1 nng nng cao yu khng

D2 nng nng cao mnh khng

D3 u m nng cao yu c

D4 ma trung bnh cao yu c

D5 ma lnh bnh thng yu c

D6 ma lnh bnh thng mnh khng

D7 u m lnh bnh thng mnh c

D8 nng trung bnh cao yu khng

D9 nng lnh bnh thng yu c

D10 ma trung bnh bnh thng yu c

D11 nng trung bnh bnh thng mnh c

D12 u m trung bnh cao mnh c

D13 u m nng bnh thng yu c

D14 ma trung bnh cao mnh khng

thuc tnh nhn

mu

D liu


n mu hun luyn, mi mu l mt cp < , > o l vector cc thuc tnh

o l nhn phn loi, (tp cc nhn)

V d mu D4 o = , , ,

o =

V d cy quyt nh


Tri

m Gi C

khng c khng c

u m nng ma

bnh

thng mnh yu cao

Cy quyt nh l g?


L m hnh phn loi c dng cy o Mi nt trung gian (khng phi l) ng vi mt php kim tra

thuc tnh, mi nhnh ca nt ng vi mt gi tr ca thuc tnh ti nt

o Mi nt l ng vi mt nhn phn loi

Qu trnh phn loi thc hin nh sau

o Mu phn loi i t gc cy xung di o Ti mi nt trung gian, thuc tnh tng ng vi nt c kim

tra, ty gi tr thuc tnh, mu c chuyn xung nhnh tng ng

o Khi ti nt l, mu c nhn nhn phn loi ca nt

Biu din di dng quy tc


Cy quyt nh c th biu din tng ng di dng cc quy tc logic

Mi cy l tuyn ca cc quy tc, mi quy tc bao gm cc php hi

V d

(Tri = nng m = bnh_thng)

(Tri = u_m)

(Tri = ma Gi = yu)

Hc cy quyt nh


Cy quyt nh c hc (xy dng) t d liu hun luyn

Vi mi b d liu c th xy dng nhiu cy quyt nh o Chn cy no?

Qu trnh hc l qu trnh tm kim cy quyt nh ph hp vi d liu hun luyn o Cho php phn loi ng d liu hun luyn

Thut ton ID3


Xy dng ln lt cc nt ca cy bt u t gc

Thut ton o Khi u: nt hin thi l nt gc cha ton b tp d liu hun luyn o Ti nt hin thi , la chn thuc tnh

Cha c s dng nt t tin

Cho php phn chia tp d liu hin thi thnh cc tp con mt cch tt nht

Vi mi gi tr thuc tnh c chn thm mt nt con bn di

Chia cc v d nt hin thi v cc nt con theo gi tr thuc tnh c chn

o Lp ( quy) cho ti khi Tt c cc thuc tnh c s dng cc nt pha trn, hoc

Tt c v d ti nt hin thi c cng nhn phn loi

Nhn ca nt c ly theo a s nhn ca v d ti nt hin thi

La chn thuc tnh ti mi nt th no?

Tiu chun chn thuc tnh ca ID3


Ti mi nt o Tp (con) d liu ng vi nt

o Cn la chn thuc tnh cho php phn chia tp d liu tt nht

Tiu chun: o D liu sau khi phn chia cng ng nht cng tt

o o bng tng thng tin (Information Gain - IG)

o Chn thuc tnh c tng thng tin ln nht

o IG da trn entropy ca tp (con) d liu

Entropy


Trng hp tp d liu c 2 loi nhn: ng (+) hoc sai (-)

= ++ +: % s mu ng, : % s mu sai

Trng hp tng qut: c loi nhn

() =

=

: % v d ca thuc loi

V d (,

+, -) = (/)(/) (/)(/)

= .

tng thng tin IG


Vi tp (con) mu v thuc tnh

Trong :

values (A): tp cc gi tr ca

Sv l tp con ca bao gm cc mu c gi tr ca bng

|| s phn t ca

)(

)(||

||)(),(

Avaluesv

vv SEntropyS

SSEntropyASIG

V d tnh IG


Tnh ,

() = *,+

= 9+, 5 , = 9

1429

145

1425

14= 0.94

= 6+, 2 ,H = 6

826

82

822

8= 0.811

= 3+, 3 , H = 3

623

63

623

6= 1

, = 8

14H

6

14H

= 0.94 8

140.811

6

141

= 0.048

Cc c im ca ID3


ID3 l thut ton tm kim cy quyt nh ph hp vi d liu hun luyn

Tm kim theo kiu tham lam, bt u t cy rng

Hm nh gi l tng thng tin

ID3 c khuynh hng (bias) la chn cy n gin o t nt

o Cc thuc tnh c tng thng tin ln nm gn gc

Vn qu va d liu


qu va d liu nu tn ti sao cho

Chng qu va bng cch ta cy


Chia d liu thnh hai phn o Hun luyn

o Kim tra

To cy ln trn d liu hun luyn

Tnh chnh xc ca cy trn tp kim tra

Loi b cy con sao cho kt qu trn d liu kim tra c ci thin nht

Lp li cho n khi khng cn ci thin c kt qu na

Sau khi ta cy


Chng qu va d liu

bng cch ta lut (C4.5)


Bin i cy thnh cc lut

Ta mi lut c lp vi cc lut khc o B mt s phn trong v tri ca lut

Sp xp cc lut sau khi ta theo mc chnh xc ca lut

S dng thuc tnh c gi tr lin tc


To ra nhng thuc tnh ri rc mi

V d, vi thuc tnh lin lc , to ra thuc tnh ri rc nh sau o = nu A >

o = nu A

Xc nh ngng th no? o Thng chn sao cho em li tng thng tin ln nht

C th chia thnh nhiu khong vi nhiu ngng

Cc o khc


o Information Gain (IG) u tin thuc tnh c nhiu gi tr, v d, thuc tnh ngy s c tng thng tin cao nht

Thng tin chia

Tiu chun nh gi thuc tnh

S

S

S

SASmationSplitInfor

iic

i

2

1

log),(

),(

),(

ASmationSplitInfor

ASnGainInformatioGainRatio

Ni dung


Gii thiu

Hc cy quyt nh

Phn loi Bayes n gin (Nave Bayes classification)

Hc da trn v d

Phng php phn loi Bayes (1/2)


Trong giai on hun luyn ta c mt tp mu, mi mu c cho bi cp < , >, trong o l vector c trng (thuc tnh)

o l nhn phn loi, ( l tp cc nhn)

Sau khi hun luyn xong, b phn loi cn d on nhn cho mu mi =< 1, 2, , >

S dng quy tc Bayes

= (|1, 2, , )

= 1, 2, , | ()

(1, 2, , )

= 1, 2, , | ()

Phng php phn loi Bayes (2/2)


S dng gi thit v tnh c lp (n gin!!!)

= 1, 2, , | ()

Tn xut quan st thy nhn trn tp d liu D:

()

||

1, 2, , | = 1| 2| |

S ln xut hin cng vi chia

cho s ln xut hin : (,)

()

V d


Xc nh nhn phn loi cho mu sau < = , = , = , = >

= , = = = = ()

Ni dung


Gii thiu

Hc cy quyt nh

Phn loi Bayes n gin

Hc da trn v d (Instance based learning)

Nguyn tc chung


Khng xy dng m hnh

Ch lu li cc mu hun luyn

Xc nh nhn cho mu mi da trn nhng mu ging mu mi nht

Gi l hc li (lazy learning)

Thut ton hng xm gn nht


-nearest neighbors (-NN)

Chn mu ging mu cn phn loi nht, gi l hng xm

Gn nhn phn loi cho mu ch s dng thng tin ca hng xm ny

o V d ly theo a s trong s hng xm

Chn hng xm th no?

Tnh khong cch


Gi s mu c gi tr thuc tnh l < 1(), 2(), , () > , thuc tnh l s thc

Khong cch gia hai mu v l khong cch Euclidean instance

, = ( )2

=1

Thut ton -NN


8 machine learning

Documents