chuong 3-hoc dua tren xac suat

My Hc (Machine Learning)

TS. on Thanh Ngh

An Giang University

Long Xuyn, Ngy 4 thng 3 nm 2014

Gii thiu

Xc sut hu nghim cc i (MAP)

nh gi kh nng c th nht (MLE)

Phn loi Nave Bayes

Cc i ha k vng EM

Chng 3 - Cc phng php hc da trn xc sut

1

Gii thiu

2


3


4

Phn loi Nave Bayes

5

Cc i ha k vng EM

2 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

Cc phng php hc da trn xc sut

Cc phng php thng k cho bi ton phn loi

Phn loi da trn m hnh xc sut c s

Vic phn loi da trn kh nng xy ra (probabilities) ca

cc lp

Cc ch chnh:

Gii thiu v xc sut

nh l Bayes

Xc sut hu nghim cc i (Maximum a posteriori)

nh gi kh nng c th nht (Maximum likelihood

estimation)

Phn loi Nave Bayes

Cc i ha k vng (Expectation maximization)

3 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

Cc khi nim c bn v xc sut

Gi s chng ta c mt th nghim (v d: mt qun xc

sc) m kt qu ca n mang tnh ngu nhin

Khng gian cc kh nng S . Tp hp tt c cc kt qu cth xy ra

V d: S = {1, 2, 3, 4, 5, 6} i vi th nghim qun xc scS kin E . Mt tp con ca khng gian cc kh nngV d: E = {1}: kt qu qun sc xc ra l 1V d: E = {1, 3, 5}: kt qu qun sc xc ra l mt s lKhng gian cc s kin W . Khng gian (world) m cc kt

qu ca s kin c th xy ra

W bao gm tt c cc ln sc xc

Bin ngu nhin A. Mt bin ngu nhin biu din (din t)mt s kin, v mt mc v kh nng xy ra s kin ny

4 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

Biu din

P(A): Phn ca khng gian (world) m trong A l ng

Khng gian tt c cc gi tr c th xy ra ca A

khng gian m trong A l ng

khng gian m trong A l sai

5 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

Cc bin ngu nhin 2 gi tr

Mt bin ngu nhin 2 gi tr (nh phn) c th nhn mt

trong 2 gi tr ng (true) hoc sai (false)

Cc tin

0 P(A) 1P(true) = 1

P(false) = 0

P(A B) = P(A) + P(B) P(A B)Cc h qu

P(not A) P(A) = 1 P(A)P(A) = P(A B) + P(A B)

6 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

Cc bin ngu nhin a tr

Mt bin ngu nhin nhiu gi tr c th nhn mt trong s

k(> 2) gi tr v1, v2, ..., vk

Cc tin

P(A = vi A = vj) = 0 if i 6= jP(A = v1 A = v2 ... A = vk) = 1P(A = v1 A = v2 ... A = vi ) =

ij=1 P(A = vj)k

j=1 P(A = vj) = 1

P(B [A = v1 A = v2 ... A = vi ] =i

j=1 P(B A = vj)

7 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

Xc sut c iu kin (1)

P(A|B) l phn khng gian m trong A l ng, vi iukin ( bit) l B ng

V d:

A: Ti s i bng vo ngy maiB: Tri s khng ma vo ngy maiP(A|B): Xc sut ca vic ti i bng vo ngy mai nu( bit rng) tri s khng ma (vo ngy mai)

8 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

Cc bin c lp v xc sut (1)

Hai s kin A v B c gi l c lp v xc sut nu xcsut ca s kin A l nh nhau i vi cc trng hp:

Khi s kin B xy ra, hocKhi s kin B khng xy ra, hocKhng c thng tin (khng bit) v vic xy ra ca s kin B

V d:

A: Ti s i bng vo ngy maiB: Tun s tham gia trn bng ngy mai

P(A|B) = P(A)D Tun c tham gia trn bng ngy mai hay khng cng

khng nh hng ti quyt nh ca ti v vic i bng vo

ngy mai

10 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

Cc bin c lp v xc sut (2)

T nh ngha ca cc bin c lp v xc sut

P(A|B) = P(A), chng ta thu c cc lut nh sau:P(A|B) = P(A)P(B|A) = P(B)P(A,B) = P(A) P(B)P(A,B) = P(A) P(B)P(A, B) = P(A) P(B)P(A, B) = P(A) P(B)

11 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

Xc sut c iu kin vi nhiu hn 2 bin

P(A|B,C ) l xc sut ca A i vi (bit) B v C

V d:

A: Ti i do b sng vo sng maiB: Thi tit sng mai rt pC : Ti s dy sm vo sng maiP(A|B,C ): Xc sut ca vic ti s ido dc b sng vo sng mai, nu (

bit rng) thi tit sng mai rt p v

ti s dy sm vo sng mai

B C

AP(A | B, C)

12 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

c lp c iu kin

Hai bin A v C c gi l c lp c iu kin i vibin B, nu xc sut ca A i vi B bng xc sut ca A ivi B v C

Cng thc nh ngha: P(A|B,C ) = P(A|B)V d:

A: Ti s i bng vo ngy maiB: Trn bng ngy mai s din ra trong nhC : Ngy mai tri s khng ma

P(A|B,C ) = P(A|B)Nu bit rng trn u ngy mai s din ra trong nh, th xc

sut ca vic ti s i bng vo ngy mai khng ph thuc

vo thi tit

13 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

nh l Bayes

P(h|D) = P(D|h)P(h)P(D)

P(h): Xc sut trc (tin nghim) ca gi thit (phn loi) h

P(D): Xc sut trc (tin nghim) ca vic quan st cd liu D

P(D|h): Xc sut (c iu kin) ca vic quan st c dliu D, nu bit gi thit (phn loi) h l ng

P(h|D): Xc sut (c iu kin) ca gi thit (phn loi)h lng, nu quan st c d liu D

Cc phng php phn loi da trn xc sut s s dng xc

sut c iu kin (posterior probability) ny!

15 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

nh l Bayes - V d (1)

Gi s chng ta c tp d liu sau (d on 1 ngi c chi tennis

hay khng?)

Ngy Ngoi tri Nhit m Gi Chi tennis

N1 Nng Nng Cao Yu Khng

N2 Nng Nng Cao Mnh Khng

N3 m u Nng Cao Yu C

N4 Ma Bnh thng Cao Yu C

N5 Ma Mt m Bnh thng Yu C

N6 Ma Mt m Bnh thng Mnh Khng

N7 m u Mt m Bnh thng Mnh C

N8 Nng Bnh thng Cao Yu Khng

N9 Nng Mt m Bnh thng Yu C

N10 Ma Bnh thng Bnh thng Yu C

N11 Nng Bnh thng Bnh thng Mnh C

N12 m u Bnh thng Cao Mnh C

16 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

nh l Bayes - V d (2)

D liu D. Ngoi tri l nng v Gi l mnh

Gi thit (phn loi) h. Anh ta chi tennis

Xc sut trc P(h). Xc sut rng anh ta chi tennis (bt kNgoi tri nh th no v Gi ra sao)

Xc sut trc P(D). Xc sut rng Ngoi tri l nng vGi l mnh

Xc sut trc P(D|h). Xc sut Ngoi tri l nng v Gil mnh, nu bit rng anh ta chi tennis

Xc sut trc P(h|D). Xc sut anh ta chi tennis, nu bitrng Ngoi tri l nng v Gi l mnh

Chng ta quan tm n gi tr xc sut sau (posteriorprobability) ny!

17 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM


Vi mt tp cc gi thit (cc phn lp) c th H, h thnghc s tm gi thit c th xy ra nht (the most probable

hypothesis) h( H) i vi cc d liu quan st c DGi thit h ny c gi l gi thit c xc sut hu nghimcc i (Maximum a posteriori MAP)

hMAP = arg maxhH P(h|D)hMAP = arg maxhH

P(D|h)P(h)P(D) (bi nh l Bayes)

hMAP = arg maxhH P(D|h) P(h) (v p(D) l nh nhaui vi cc gi thit h)

18 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

MAP - V d

Tp H bao gm 2 gi thit (c th)h1: Anh ta chi tennish2: Anh ta khng chi tennis

Tnh gi tr ca 2 xc sut c iu kin: P(h1|D),P(h2|D)Gi thit c th nht hMAP = h1 nu P(h1|D) P(h2|D);ngc li th hMAP = h2

Bi v P(D) = P(D, h1) +P(D, h2) l nh nhau i vi c haigi thit h1 v h2, nn c th b qua i lng p(D)

V vy, cn tnh 2 biu thc: P(D|h1) P(h1) vP(D|h2) P(h2), v a ra quyt nh tng ngNu P(D|h1) P(h1) P(D|h2) P(h2), th kt lun l anh tachi tennis

Ngc li, th kt lun l anh ta khng chi tennis

19 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM


Phng php MAP: Vi mt tp cc gi thit c th H, cntm mt gi thit cc i ha gi tr: P(D|h) P(h)Gi s (assumption) trong phng php nh gi kh nng c

th nht (Maximum likelihood estimation MLE): Tt c cc

gi thit u c gi tr xc sut trc nh nhau:

P(hi ) = P(hj),hi , hj HPhng php MLE tm gi thit cc i ha gi tr P(D|h);trong P(D|h) c gi l kh nng c th (likelihood) cad liu D i vi h

Gi thit c kh nng nht (maximum likelihood hypothesis)

hML = arg maxhH P(D|h)

20 / 44

Gii thiu



Phn loi Nave Bayes

Cc i ha k vng EM

MLE V d

Tp H bao gm 2 gi thit c thh1: Anh ta chi tennish2: Anh ta khng chi tennis

D: Tp d liu (cc ngy) m trong thuc tnh Outlook cgi tr Sunny v thuc tnh Wind c gi tr StrongTnh 2 gi tr kh nng xy ra (likelihood values) ca d liu

D i vi 2 gi thit: P(D|h1) v P(D|h2)P(Outlook = Sunny ,Wind = Strong |h1) = 1/8P(Outlook = Sunny ,Wind = Strong |h2) = 1/4Gi thit MLE hMLE = h1 nu P(D|h1) P(D|h2); v ngcli th hMLE = h2Bi v P(Outlook = Sunny ,Wind = Strong |h1)

chuong 3-hoc dua tren xac suat

Documents