Digital Signature

Tm tt ni dung

Ting ni l cng c giao tip v cng hiu qu v khng th thiu ca con ngi. Ngy nay, vi tng m rng vic giao tip ngi vi my mc qua ting ni thay v nhng thit b u vo phc tp v khng d nh, rt nhiu nh nghin cu khoa hc u t cng sc vo vic xy dng nhng h thng nhn dng ting ni t ng cho nhiu kiu ging ni v nhiu ngn ng. c im chung ca cc h nhn dng ny l u bt u bng qu trnh tm hiu v m phng cc c im ca ting ni, hay cn gi l qu trnh trch chn c trng. Cng vic ny t nn tng quan trng cho vic p dng cc phng php nhn dng v quyt nh ti tnh chnh xc ca ton h thng.Tip tc nhng nghin cu trn, kha lun ny tm hiu nhng c im ca ting ni ni chung v ting ni ting Vit ni ring vi mc ch kt xut c cc c trng ting ni ting Vit di dng s thc cho qu trnh nhn dng. ng thi p dng m hnh thng k HMM nhn dng s dng phng php phn bit thanh iu c kt qu kim chng mc chnh xc ca qu trnh trch chn c trng v hng ti ng dng.

Li cm n

u tin, ti xin chn thnh cm n tin s L Anh Cng, ng cm n tin s L S Vinh hin cng ang cng tc ti b mn Khoa Hc My Tnh - khoa Cng ngh Thng Tin - trng i Hc Cng Ngh - i Hc Quc Gia H Ni, hai thy hng dn trc tip v cng hng dn ti hon thnh kha lun ny. Nh s ng vin gip nhit tnh cng nhng li khuyn b ch, nhng tng sng to ca hai thy trong qu trnh hng dn gip ti hon thnh kha lun ny mt cch tt nht.Tip theo ti xin dnh li cm n ti PGS.TS Lng Chi Mai v anh V Tt Thng hin ang cng tc Vin Khoa Hc v Cng Ngh Vit Nam, hai ngi gip v cho ti rt nhiu li khuyn cng nh kinh nghim hu ch khi gp nhng kh khn, b tc trong qu trnh hon thnh kha lun.Ngoi ra, xin gi li cm n ti ngi bn cng nhm nghin cu: m Tin Dng, ngi cng st cnh, gip v ng vin ti rt nhiu t khi bt u ti khi hon thnh. ng cm n ti cc bn cng lp v cc anh ch hc kha trn vi nhng chia s v kinh nghim b ch.Cui cng ti xin gi li cm n ti gia nh ti, ba v m l nhng ngi lun ng h v l ch da tinh thn vng chc cho nhng nm hc i hc ni chung v vic hon thnh kha lun cui kha ni ring.

Mc lcChng 1. M U11.1.t vn 11.2.Hng nghin cu v phng php s dng21.3.Gii hn v mc tiu ca ti3

Chng 2. K THUT TRCH CHN C TRNG MFCC TRONG NHN DNG TING NI 42.1.X L TN HIU M THANH V TRCH CHN C TRNG42.2.TRCH CHN C TRNG MFCC 52.2.1.Pre-emphasis62.2.2.Windowing62.2.3.DFT (Discrete fourier transform)92.2.4.Mel filter-bank and log92.2.5.Discrete consinse transform102.2.6.Feature extraction112.2.7.Tng kt12

Chng 3. C TRNG V THANH IU CA TING VIT133.1.Khi nim ngn iu, ng iu v thanh iu 133.2.Tm ng nt F0 v nghin cu c im ca tng thanh iu trong ting Vit143.2.1.Tnh ng nt thanh iu 143.2.2.c im ca tng thanh iu da vo ng nt 15

Chng 4. S DNG C TRNG TING NI NI CHUNG V TING VIT NI RING CHO M HNH NHN DNG TING NI TING VIT194.1.M hnh Markov n (Hidden Markov Model)194.1.1.Xch Markov, qu trnh Markov194.1.2.M hnh Markov n (Hidden Markov Model - HMM)204.2.p dng m hnh HMM cho bi ton nhn dng ting ni, s dng trch chn c trng MFCC 214.2.1.M hnh ha nhn dng ting ni bng HMM214.3.S dng ng nt F0 phn bit thanh iu ting Vit294.3.1.Thanh ng294.3.2.Thanh ngang294.3.3.Thanh huyn304.3.4.Thanh sc304.3.5.Thanh nng304.3.6.Thanh hi314.3.7.Phn lp thanh iu31

Chng 5.KT QU THC NGHIM325.1.Kt qu ca nhn dng ting ni s dng c trng MFCC325.2. Kt qu phn bit thanh iu335.2.1.M t b d liu335.2.2.Kt qu355.3.Nhn xt35

Chng 6.KT LUN CHUNG V NH HNG NGHIN CU PHT TRIN TRONG TNG LAI376.1.Cc vn nghin cu v hon thnh376.2.Cc vn tn ti376.3.nh hng nghin cu pht trin trong tng lai38

PH LC39Ti liu tham kho:41

Danh mc hnh minh ha

Hnh 1 : V tr ca vic trch chn c trng trong h thng nhn dng ting ni3Hnh 2 (ngun [2]): S qu trnh trch chn c trng MFCC6Hnh 3 (ngun [2]): Mt on tn m thanh trc v sau Pre-Emphasis7Hnh 4 (ngun [2]): Minh ha qu trnh Windowing8Hnh 5 (ngun [2]): So snh Rectangular (tri) v Hamming window (phi)9Hnh 6 (ngun [2]): Bin i DFT cho mt Hamming window10Hnh 7 (ngun [2]): M hnh cc bng lc trong thang o tn s bnh thng v thang o mel11Hnh 8: ng nt th ca thanh ngang, m v a17Hnh 9: ng nt th ca thanh huyn, m v 17Hnh 10: ng nt th ca thanh sc, m v 18Hnh 11: ng nt th ca thanh ng, m v 18Hnh 12: ng nt th ca thanh nng, m v 19Hnh 13: ng nt th ca thanh hi, m v 19

M U

Chng u tin dnh t vn v ti nhn dng ting ni ni chung v nhn dng ting ni ting Vit ni ring qua xc nh tm quan trng ca vic tm hiu c trng ca ting ni i vi h thng nhn dng. ng thi, chng ny ch ra nhng nghin cu hin ti v hng nghin cu s dng cng nh mc tiu ca kha lun i vi ti ny.

t vn Mt trong nhng mc ch v xu hng quan trng nht ca khoa hc ngy nay l hng ti vic t ng ha cc cng vic chn tay, v thc t my mc vi hiu sut v tc vt tri thc s thay th sc lao ng ca con ngi trong rt nhiu lnh vc nh iu khin my, ch to linh kin, vt liu Mt cch t nhin, iu ny ny sinh ra nhu cu giao tip gia con ngi vi my mc, khi vic giao tip bnh thng thng qua cc thit b u vo dn tr nn phc tp nh chnh nhng c my . Trong nhiu nm qua, vic nghin cu v xy dng h thng nhn dng ting ni phc v giao tip ngi my c rt nhiu nh nghin cu khoa hc trn th gii u t thi gian cng sc v t c nhiu kt qu kh quan. (VD: Framework nhn dng ting ni Sphinx4, xy dng bi cng ty Sun, nhn dng c nhng cu nm trong tp t in khong 65.000 t) [8]c im chung ca cc h thng nhn dng ting ni, d s dng phng php no, l trc ht phi s ha tn hiu ting ni my tnh c th hiu c, qua tm c nhng c trng ring ca ting ni so vi nhng c trng ca cc m thanh khc nh nhc c, ting n Chnh v vy vic trch chn c trng c th ni l vn quan trng hng u cho mt h thng nhn dng ting ni, trch chn c cc c trng cng chnh xc, chnh xc trong vic nhn dng ca h thng cng cao, iu ny hon ton ng vi tt c ting ni ca mi ngn ng, trong bao gm c ting ni ting Vit.

Hnh sau y ch ra v tr ca qu trnh trch chn c trng trong h thng nhn dng ting ni bt k:

TextframeframeframeTing niHun luyn nhn dngTrch chn c trng

Hnh 1 : V tr ca vic trch chn c trng trong h thng nhn dng ting ni

Hng nghin cu v phng php s dng C ba phng php chnh, u kh hiu qu tm hiu c c trng ca ting ni, th nht l m phng li qu trnh pht ra ting ni ca b my pht m (bao gm cc b phn bn trong ming, mi), th hai l m phng qu trnh thu nhn m thanh ca b my cm nhn m thanh (cc b phn bn trong tai) v phng php phn tch ph - tng hp ca hai phng php trn. Hin nay phng php chn c trng m phng qu trnh pht m v thu nhn m l phng php m ha d on tuyn tnh LPC (Linear predictive coding) [6][9] v phng php ly hm bin trung bnh AMDF [12] (Average magnitude different function) m t s cm nhn cao m thanh ca tai, tuy nhin hn ch ca hai phng php ny th hin vic kt qu nhn dng cn cha tht cao. [12]Trong kha lun ny, ta s cp ti k thut trch chn c trng MFCC [2] kt hp ca 2 phng php trn trch chn c trng ting ni ting Vit, v s dng li phng php AMDF [9] trch chn c trng c th ca ting Vit l thanh iu. Trch chn c trng MFCC c xem l mt phng php rt hiu qu v c p dng trong nhiu h nhn dng ni ting nh Sphinx ca cng ty Sun.S dng kt qu trch chn c trng, ta p dng mt phng php nhn dng rt hiu qu l dng m hnh HMM [5] hun luyn v nhn dng ting ni, s dng c trng thanh iu phn bit thanh iu ting Vit cho mi tn hiu m thanh ban u.

Gii hn v mc tiu ca tiMc tiu ca vic tm hiu c trng trong ting ni hng ti vic xy dng mt h thng nhn dng ting ni ting Vit vi chnh xc cao, tuy nhin trong phm vi thi gian v khun kh ca mt kha lun c nhn Cng Ngh Thng tin, ti gii hn ni dung nghin cu trong nhng vn di y:Th nht, tuy vic trch chn c trng MFCC v AMDF c p dng cho c h nhn dng ting ni lin tc (ting ni c ni theo cu) v h nhn dng ri rc (ni tng t ring bit), nhng h thng m kha lun ny xy dng l h nhn dng ri rc, vi b t in l b ch s m ting Vit (KHONG, MOT, HAI, BA, BON, NAM, SAU, BAY, TAM, CHIN), v phn bit thanh iu khng ph thuc t in.Th hai, h nhn dng ca chng ti (bao gm c ngi cng nhm nghin cu) xy dng l Ph thuc ngi ni, do cha c iu kin thu m hun luyn v kim th vi nhiu kiu ging ni nn khng th coi h thng xy dng l Khng ph thuc ngi ni c. H thng xy dng c s ch hun luyn v nhn dng vi ging ni ca mt ngi.T vic xc nh mc tiu r rng ca mnh, chng ti nh hng trong tng lai s nghin cu su hn v cc k thut trch chn c trng cng nh k thut nhn dng m rng b t vng nhn dng, hng vo cc ng dng giao tip ngi my, iu khin my bng ging ni v cc ng dng khc trong giao tip truyn thng

K THUT TRCH CHN C TRNG MFCC TRONG NHN DNG TING NI [footnoteRef:2] [2: ng nghin cu v c phn ni dung chung vi kha lun Cc k thut nhn dng ting ni, 2010 ca sinh vin m Tin Dng i hc cng ngh.]

X L TN HIU M THANH V TRCH CHN C TRNGTn hiu m thanh ngoi i thc l tn hiu lin tc, hay tn hiu tng t. Trc khi thc hin bt c bc x l no, tn hiu m thanh cn c s ha. Vic ny c thc hin t ng bi cc thit b thu m, bng cch ly mu tn hiu u vo [1]. Nh vy, mt tn hiu m thanh bt k khi c a vo my tnh, l mt tp cc mu lin tip nhau, mi mu l gi tr bin ca tn hiu ti mt thi im nht nh. Mt tham s quan trng trong vic ly mu tn hiu m thanh l tn s ly mu, Fs, tc l s mu c ly trong mt giy. c th o lng chnh xc, cn phi ly t nht 2 mu trong mt chu k ca tn hiu tng t u vo. Nh vy, tn s ly mu phi ln hn 2 ln tn s cao nht ca tn hiu m thanh u vo. Tuy nhin, trn thc t tai ngi ch c th nhn bit c cc m thanh c tn s nh hn 10.000Hz [12][3], do tn s ly mu l 20.000Hz l cho vic nhn dng vi chnh xc rt cao. Trong lnh vc nhn dng ting ni qua in thoi, tn s ly mu ch cn l 8.000Hz v ch c cc tn hiu c tn s nh hn 4.000Hz c truyn i bi in thoi [10]. Cc thit b thu m th thng c tn s ly mu l 16.000Hz [3] .Trch chn c trng i vi nhn dng ting ni l vic tham s ha chui tn hiu m thanh dng sng u vo, bin i tn hiu m thanh thnh mt chui cc vector c trng n chiu, mi chiu l mt gi tr thc. Hin nay, c rt nhiu phng php trch chn c trng nh: LPC(Linear predictive coding D on tuyn tnh [6][9]), AMDF(Average magnitude different function hm bin trung bnh), MFCC(Mel-frequency cepstral coefficients), hoc kt hp ca cc phng php trn [12]. Phn tip theo s gii thiu c th v phng php trch chn c trng MFCC.Trong bi ton nhn dng ting ni ang xt, vi tn s ly mu mc nh 16.000Hz, mt on mu vi mt s lng nht nh to thnh mt frame, nh vy tn hiu ting ni l tp cc frame lin tip nhau, trch chn c trng MFCC cho ta tp c trng cho mi frame ting ni ny. Ti sao phi chia thnh cc frame v cc frame c th chng c c trng th no, ta s cp ti ngay phn sau y.

TRCH CHN C TRNG MFCC [footnoteRef:3] [3: Ni dung tham kho t cun Speech and Language Processing, 2007, chapter 9. Tc gi Daniel Jurafsky & Jame H.Martin.]

Trong nhn dng ting ni, k thut trch chn c trng MFCC l phng php ph bin nht. MFCC l vit tt ca Mel-frequency cepstral coefficients. K thut ny da trn vic thc hin bin i chuyn d liu m thanh u vo ( c bin i Fourier cho ph) v thang o tn s Mel, mt thang o din t tt hn s nhy cm ca tai ngi i vi m thanh. K thut trch chn c trng ny gm cc bc bin i lin tip, trong u ra ca bc bin i trc s l u vo ca bc bin i sau. u vo ca qu trnh trch chn c trng ny s l mt on tn hiu ting ni. V tn hiu m thanh sau khi c a vo my tnh c ri rc ha nn on tn hiu ting ni ny bao gm cc mu lin tip nhau, mi mu l mt gi tr thc, th hin gi tr bin ca m thanh ti 1 thi im.Trch chn c trng MFCC gm su bc nh trong hnh v sau, kt qu l mt tp gm 39 gi tr c trng cho mi mt frame ting ni.

Hnh 2 (ngun [2]): S qu trnh trch chn c trng MFCC

Pre-emphasisTn hiu m thanh thng c thu mi trng i thng, ting ni bnh thng ca mt ngi cng khng c to, tr khi ni to c ch nh, do nhiu ca mi trng (tn s thp) nhiu khi c cng ln bng mt phn ng k (nghe c th d dng nhn ra) ca ting ni khi thu m, bc u tin ca qu trnh trch chn c trng MFCC s x l vn ny, bng vic thc hin tng cng ca nhng tn s cao ln nhm lm tng nng lng vng c tn s cao vng tn s ca ting ni, mt cch d hiu l lm ting ni ln hn ln nh hng ca cc m thanh mi trng v nhiu tr thnh khng ng k. Vic tng cng ca vng tn s cao ln ng thi lm cho thng tin r rng hn i vi mu ting ni. Hnh sau m t trc v sau qu trnh Pre-emphasis ca mt on tn hiu m thanh:

Hnh 3 (ngun [2]): Mt on tn m thanh trc v sau Pre-Emphasis

WindowingTrong h thng nhn dng ting ni c trnh by kha lun ny, vi mc ch nng cao chnh xc ca vic nhn dng ting, thay v nhn dng tng t ring bit, mi mt t trong on hi thoi s c phn tch thnh cc m v (subphone) [7], v h thng s nhn dng tng m v. m v y l n v pht m ca mt t, cc m v cu thnh ting ni, trong ting Anh, n l n v cu thnh phin m ca t (chng hn ONE: w-ah-n, m v y l w, ah v n), trong cch pht m ca ting Vit, cch vit ca t chnh l hnh thc vn bn ca m v (chng hn MOT = m-oo-t, m v l m, oo v t).V l do , cc c trng cn phi c trch chn trn tng m v, thay v c t hay c on ting ni di. Windowing l vic ct on tn hiu m thanh u vo ra thnh cc mu tn hiu c thi lng nh, gi l cc frame. Mi frame ny sau s c nhn dng n thuc m v no. Ni cch khc, mt frame s l mt tp gm mt s mu ca tn hiu ban u ta cp phn 2.1. Mt l do khc cho thy s cn thit ca vic windowing l v tn hiu m thanh thay i rt nhanh, do cc thuc tnh nh bin , chu k s khng n nh. Khi tn hiu m thanh c ct ra thnh nhng on nh th mi on, c th coi tn hiu l n nh, cc c trng ca tn hiu l khng i theo thi gian. Hnh v sau m t qu trnh Windowing:

Hnh 4 (ngun [2]): Minh ha qu trnh Windowing thc hin vic ny, chng ta s dng mt ca s (window) chy dc tn hin m thanh v ct ra cc on tn hiu nm trong ca s . Mt ca s c nh ngha bng cc thng s: Frame size: rng ca ca s, cng l ln ca frame tn hiu s c ct ra. Frame shift: bc nhy ca ca s, l di on m ca s s trt ct ra frame tip theo.Mi frame sau s c nhn vi mt h s, gi tr ca h s ny ty thuc vo tng loi ca s.

Trong x[n] l gi tr ca mu th n, y[n] l gi tr ca mu th n sau khi nhn vi h s, w[n] l h s cho mu th n trong frame .Loi ca s n gin nht l ca s Rectangular, gi tr ca cc h s w[n] c cho bi cng thc sau:

Ni cch khc, ca s Rectangular vi bc nhy l frame shift, ta ly frame size gi tr lin tip ca tn hiu lm mt frame.Mt loi ca s khc thng dng hn trong trch chn c trng MFCC l ca s Hamming. Trong loi ca s ny, gi tr ca tn hiu s gim dn v 0 khi tin dn ra hai bin ca frame. Ni cch khc, nu s dng ca s Hamming ly ra cc frame, nng lng ca mi frame s tp trung gia frame, mt u im na l cc gi tr bin ca ca s Hamming tin dn v 0 s lm bc bin i Fourier ngay sau tr nn d dng hn (vi ca s Rectangular cc gi tr gi nguyn so vi mu ting ni, bn ngoi ca s nhn gi tr 0, cc gi tr s b tng t ngt hai bin). H thng nhn dng trong kha lun ny trnh by s s dng ca s Hamming. Biu thc h s ca ca s ny l:

So snh hai loi ca s Rectangular v Hamming

Hnh 5 (ngun [2]): So snh Rectangular (tri) v Hamming window (phi)

DFT (Discrete fourier transform)Bc bin i tip theo l thc hin bin i Fourier ri rc i vi tng mu tn hiu c ct ra. Qua php bin i ny, tn hiu s c a v khng gian tn s. Cng thc ca bin i Fourier:

Trong x[n] l gi tr ca mu th n trong frame, X[k] l mt s phc biu din cng v pha ca mt thnh phn tn s trong tn hiu gc, N l s mu trong mt frame. Thng thng ngi ta s dng bin i FFT (Fast fourier transform) thay v DFT. Bin i FFT nhanh hn nhiu so vi bin i DFT, tuy nhin thut ton ny i hi gi tr N phi l mt ly tha ca 2. Hnh sau m t trc v sau khi bin i DFT ca mt ca s:

Hnh 6 (ngun [2]): Bin i DFT cho mt Hamming window

Mel filter-bank and logKt qu ca qu trnh bin i Fourier th hin nng lng ca tn hiu nhng di tn s khc nhau. Tuy nhin, tai ca ngi li khng c s nhy cm nh nhau i vi mi di tn s. Do vic m hnh ha tnh cht ny ca tai ngi trong qu trnh trch chn c trng lm tng kh nng nhn dng ca h thng. Trong m hnh trch chn c trng MFCC, tn s s c chuyn sang thang o tn s mel theo cng thc:

Trong f l tn s thang o thng, fmel l tn s thang o mel. Ngi ta s dng cc bng lc tnh cc h s mel. S dng bao nhiu bng lc th s cho ra by nhiu h s mel, v cc h s mel ny s l u vo cho qu trnh tip theo ca trch chn c trng MFCC. Hnh v sau biu din m hnh cc bng lc trong thang o tn s bnh thng v thang o mel:

Hnh 7 (ngun [2]): M hnh cc bng lc trong thang o tn s bnh thng v thang o melCui cng ca giai on ny, ta ly logarit c s t nhin ca ph tnh theo thang o Mel, thao tc ny c 2 nguyn nhn, mt l do tai ngi nhy cm vi m thanh cng thp hn, hai l lm cc gi tr c trng nh i, tin cho vic tnh ton.

Discrete consinse transformBc tip theo ca vic trch chn c trng MFCC l bin i fourier ngc vi u vo l cc h s ph mel ca bc trc, u ra s l cc h s cepstrum (MFCC Mel Frequency Cepstrum Coefficients).Sau khi thc hin bin i Fourier th dy tn hiu theo thi gian c chuyn thnh ph tn s, v vic p dng cc bng lc tn s mel gip c ng ph tn s v mt s h s nht nh (bng vi s bng lc). Cc h s ny th hin cc c trng ca ngun m thanh nh tn s c bn, xung m thanh Tuy nhin, cc c trng ny khng quan trng i vi vic phn bit cc m khc nhau. Thay vo , cc c trng v b my pht m (khoang ming, khoang mi, thanh qun, hu) rt cn thit cho vic nhn dng cc m. Vic thc hin bin i fourier ngc s gip tch bit cc c trng v ngun m v b my pht m t cc h s (cc c trng v b my pht m l cc h s u tin).

Feature extractionT cc h s mel thu c t qu trnh trc, thng thng chng ta ch ly ra 12 h s u tin chn lm c trng. 12 h s ny ch c trng cho cc b phn ca b my pht m. Nh vy chng ta c 12 c trng u tin.c trng th 13 l nng lng ca m. Nng lng ca mi khung tn hiu c tnh ngay t sau bc windowing:

Vi 13 c trng , chng ta thm vo 13 c trng delta th hin tc thay i ca ca m gia cc khung tn hiu, c tnh bng cng thc:

trong d(t) l c trng delta ca khung t, c(t+1) v c(t-1) l cc c trng ph ca khung ngay sau v trc khung t; v 13 c trng double delta th hin gia tc thay i ca m gia cc khung tn hiu. Cng thc tnh cc c trng double delta ging vi cng thc tnh cc c trng delta, khi coi c(t) l gi tr ca cc c trng delta.

Tng ktTrch chn c trng MFCC s thu c cc c trng sau y:12 gi tr c trng ph Mel c bin i Fourier ngc12 gi tr delta ph12 gi tr double delta ph1 gi tr mc nng lng1 gi tr delta mc nng lng1 gi tr double delta mc nng lngTng cng: 39 c trng cho mi frame ting ni.

C TRNG V THANH IU CA TING VIT

chng trc ta trnh by v vic chn c trng cho ting ni ting Vit thng qua m phng b my pht m, y cng l c trng chung cho cc ngn ng khc. Tuy nhin, ting ni ting Vit cn c nhng c im ring, c th v c o, vic tm hiu nhng c trng ny v a chng vo phc v nhn dng s lm tng chnh xc ton cc ca h thng nhn dng ting ni.

Khi nim ngn iu, ng iu v thanh iu [footnoteRef:4] [4: Ni dung tham kho trong ti liu: M hnh Fujisaki v p dng trong phn tch thanh iu ting Vit ca Bch Hng Nguyn, Nguyn Tin Dng.]

Ni mt cch nm na, trong ngn ng ni, ngn iu l ci mang li m sc cho ting ni, m sc l biu hin t nhin ca ging ni, mang ngha nhn mnh hoc th hin sc thi tnh cm, li ni khng c ngn iu ging nh li ni ca robot, khng ging ting ni t nhin. Cc nh ngn ng hc cho rng bn cht ngn iu l s ph ln m tit cc yu t trng m, thanh iu, ng iu v trng . Vai tr ca ngn iu rt quan trng trong tng hp ting ni, nu khng x l c vn ngn iu th khng th tng hp c ting ni t nhin ca con ngi c. c trng quan trng nht ca ngn iu l cao, di, to, tng ng l cc i lng tn s c bn F0, thi gian ca m tit, m v D, v cng I.Ngn iu ca li ni lin kt cht ch vi khi nim ng iu. C th ni ng iu l s nng cao h thp ca li ni trong cu, khi xt l mt m tit (trong ting Vit gi l mt ting) ng iu lc ny tr thnh thanh iu ca m tit . c trng chnh cho tnh cht ny l tn s c bn ca ging ni: F0. Vic ly cc gi tr F0 theo thi gian to thnh ng nt F0. Trong li ni lin tc, ng nt F0 cho mi thanh iu c cc c trng khc nhau, tn hiu th ban u l dng th ca ng nt F0, chng ny, ta i nghin cu cch lm mn ng nt F0 cho mi m tit ring bit, theo ng gii hn ban u ca bi ton.

Tm ng nt F0 v nghin cu c im ca tng thanh iu trong ting VitTrong ting Vit, c 6 thanh iu c s dng: thanh ngang, huyn, sc, hi, nng v ng. Trong vn hc xa tng xut hin lut bng trc: thanh bng ch m tit c ng nt c chiu hng i ngang hoc i xung (l thanh ngang, huyn) thanh trc ch m tit c ng nt i ln (thanh sc, nng, ng), tuy nhin phn loi nh vy l cha cht ch v y . Sau y, ta s a ra mt cch lm mn ng nt F0 th hin thanh iu ting ni v nghin cu c im ca tng thanh iu.

Tnh ng nt thanh iu [footnoteRef:5] [5: Ni dung tham kho trong ti liu Nhn dng ting Vit dng mng Neuron kt hp trch c trng dng LPC v AMDF, 2005, tc gi Hong nh Chin.]

Hm bin trung bnh (AMDF Average Magnitude Difference Fucntion)Hm hiu bin trung bnh ca mt tn hiu l hiu bin ca chnh n ri i p mu, c tnh bi cng thc:d(p) = | y x(n) l gi tr bin th n ca tn hiu, N l s gi tr bin (thng l s gi tr c ly ra trong 1 khong thi gian c nh, vi tn s ly mu l Fs)Nu x(n) l tn hiu tun hon vi chu k T th khi p tin dn ti gi tr T, hm d(p) s t gi tr nh nht. Do tn hiu l ri rc nn s tn ti gi tr nguyn p0 sao cho d(p0) l nh nht, khi gi tr f0 = Fs/p0 c coi l tn s c bn ca on tn hiu , ni cch khc n c trng cho thanh iu ca on tn hiu , f0 l mt gi tr trong ng nt F0 c trng cho thanh iu ca ton b tn hiu ging ni ban u. Ging ni ca ngi bnh thng c tn s c bn l khong 90Hz vi ging nam v 200Hz vi ging n, ta s ly p0 s nm trong khong rng hn t Fs/250 n Fs/80.C mi on tn hiu ko di t 10-25ms ta li ly mt gi tr f0 nh vy, tp f0 theo thi gian thu c chnh l ng nt F0.

Thc hin tm ng nt F0

Ct xn tn hiu lm ni r chu k c bny[n] = Trong C c chn vo khong 1/3 gi tr bin cc i trn ton tn hiuTnh hm bin trung bnh: tn hiu sau khi c ct xn c a vo hm ly bin trung bnh nh trong mc 3.2.1.1 vi N l di ca mt khung (gm cc gi tr c ly trong 1 khong thi gian nht nh, y ly s gi tr trong 1 frame (khong 10-25ms) nh mc I trnh by).Lm mn: vi cc d(p0) > 0.7*dmax(p) ta coi l khung v thanh, tnh gi tr c trng f0 = 0. Sau khi c tp { f0 } tip tc lm mn ng nt F0 bng cch: nu cc khung v thanh u hoc cui m tit th s c thay th bi gi tr f0 k cn, nu khung v thanh gia m tit th thay bng trung bnh ca 2 gi tr f0 ngay cnh. Cui cng lm trn ng nt F0 bng b lc vi p ng xung h = [0.1, 0.2, 0.4, 0.2, 0.1]Ly c trng: Ty vo nhu cu s dng bao nhiu c trng m ly cc gi tr t ng nt F0, c th ly cc gi tr trn ng nt, hoc c th bin i ri rc ng nt v mt s gi tr c trng nht nh.

c im ca tng thanh iu da vo ng nt [footnoteRef:6] [6: Ni dung tham kho t M hnh Fujisaki v p dng trong phn tch thanh iu ting Vit, tc gi Bch Hng Nguyn, Nguyn Tin Dng]

Thanh ngangng nt ca thanh ngang thng c xu hng gim nh, iu ny d hiu bi khi pht m, mc nng lng gn nh khng i v gim dn v cui m tit, thanh ngang d b nhm ln vi thanh huyn v ng nt ca chng tng t nhau (xu hng khng i hoc gim nh)Hnh m t ng nt th ca thanh ngang.

Hnh 8: ng nt th ca thanh ngang, m v a

Thanh huyn

ng nt thanh huyn khi pht m chun c xu hng khng tng, khng gim, gn ging vi thanh ngang, iu ny ta va cp ti, n gy kh khn trong vic phn bit ring hai thanh iu ny. Hnh sau l ph bin th ca thanh huyn:

Hnh 9: ng nt th ca thanh huyn, m v

Thanh scThanh sc c ng nt i ln, kh ging vi thanh ng v thanh nng, thanh sc c m vc bt u cao hn 2 thanh cn li, c bo co th nghim kt lun rng: cho ng nt ca thanh sc v thanh ng ging ht nhau, khi tng hp li ngi nghe vn phn bit c 2 thanh ny. Tuy nhin, thanh ng v thanh nng cng cn nhng c im quan trng khc phn bit vi cc thanh cn li.

Quan st ng nt ( dng ph) th ca thanh sc:

Hnh 10: ng nt th ca thanh sc, m v

Thanh ng

ng nt thanh ng b gy gia, khng ch gy F0 m thanh ng cn b gy ph, chnh l khc bit ln nht gia thanh ng vi cc thanh cn li. Hnh sau m t iu ny

Hnh 11: ng nt th ca thanh ng, m v

Thanh nngThanh nng c c trng b gy, t v i xung t ngt cui m, thanh nng cng gp kh khn khi phn bit vi thanh sc, nu cho thanh nng ng nt F0 ca thanh sc th ngi nghe vn phn bit c l thanh nng, c iu phn cui m tit cm gic b nhn ln, nu m tit c pht m r rng, chun ch ng h ging cui m tit c thanh nng, kh nng phn bit 2 thanh ny s cao hn. Sau y l hnh m t ng nt th dng ph ca thanh nng:

Hnh 12: ng nt th ca thanh nng, m v

Thanh hi

ng nt ca thanh hi c c trng l c nng cao hai u v cao thp gia m tit, tuy nhin trong ting ni t nhin, c trng ny khng c th hin r rng do nhng yu t nh tc ni, kiu ni ca mi ngi v ty ng cnh thanh iu ny c nhn nh th no, thanh hi trong ging ni t nhin, khng ng cnh hay b nhm ln vi thanh huyn v thanh ngang. Trng hp pht m l tng cho m tit c thanh hi:

Hnh 13: ng nt th ca thanh hi, m v

S DNG C TRNG TING NI NI CHUNG V TING VIT NI RING CHO M HNH NHN DNG TING NI TING VIT

Nh ta trnh by chng 1, trch chn c trng MFCC m phng qu trnh pht ra ting ni ca b my pht m thng qua 39 c trng cho mi frame tn hiu, nh vy mi frame s c coi nh 1 vector 39 chiu gi tr thc v mt tn hiu ting ni l mt tp cc frame. Mc tiu bi ton tr thnh: vi ting ni u vo bt k, ta gn nhn cho cc frame (sau khi trch chn c trng) sao cho ph hp nht vi m hnh m hc ca h thng ta xy dng (khi nim m hnh m hc s c nhc li trong chnh chng ny) . Bng vic p dng m hnh Markov n - HMM gn nhn frame, ting ni s c nhn dng v hnh thc vn bn (text). Bn cnh , ta cng thc hin phn ng nt F0 (ng c trng cho thanh iu) minh ha vic phn bit thanh iu cho cc m tit pht m ging nhau.

M hnh Markov n (Hidden Markov Model) phn ny ta s gii thiu m hnh thng k HMM p dng m hnh ny vo bi ton nhn dng ting ni.

Xch Markov, qu trnh MarkovXch Markov (t theo tn nh ton hc ngi Nga Andrei Andreyevich Markov) l mt dy X1, X2, X3, ... gm cc bin ngu nhin. Tp tt c cc gi tr c th c ca cc bin ny c gi l khng gian trng thi S, gi tr ca Xn l trng thi ca qu trnh (h) ti thi im n.Nu vic xc nh (d on) phn b xc sut c iu kin ca Xn+1 khi cho bit cc trng thi qu kh l mt hm ch ph thuc Xn th:P(Xn+1 = x | X0, X1, , Xn) = P(Xn+1 = x | Xn)trong x l mt trng thi no ca qu trnh (x thuc khng gian trng thi S) . l thuc tnh Markov.

Mt cch n gin hnh dung mt kiu chui Markov c th l qua mt tmat hu hn (finite state machine). Nu h trng thi y ti thi im n th xc sut m h s chuyn ti trng thi x ti thi im n+1 khng ph thuc vo gi tr ca thi im n m ch ph thuc vo trng thi hin ti y. Do , ti thi im n bt k, mt xch Markov hu hn c th c biu din bng mt ma trn xc sut, trong phn t x, y c gi tr bng P(Xn+1 = x | Xn = y) v c lp vi ch s thi gian n (ngha l xc nh trng thi k tip, ta khng cn bit ang thi im no m ch cn bit trng thi thi im l g). Mt qu trnh mang tnh ngu nhin c c tnh ging nh xch Markov ta gi l qu trnh Markov bc 1. Qu trnh Markov bc n l dy bin ngu nhin m d on phn b xc sut c iu kin Xn+1 l mt hm ph thuc X1, X2, , Xn. Tuy nhin y, p dng cho bi ton nhn dng ging ni, ta ch xt ti qu trnh Markov bc 1 (hay xch Markov). tin cho vic trnh by, trong kha lun ny, nu khng c ghi ch thm, ta hiu qu trnh Markov chnh qu trnh Markov bc 1.

M hnh Markov n (Hidden Markov Model - HMM)M hnh Markov n l m hnh thng k trong h thng c m hnh ha c cho l mt qu trnh Markov vi cc tham s khng bit trc v nhim v l xc nh cc tham s n t cc tham s quan st c, da trn s tha nhn ny. Thng thng, cc tham s bit trc l xc sut chuyn trng thi trong xch Markov bng vic quan st cc chui bit, cc tham s cha bit c th l phn phi xc sut ca trng thi i vi quan st Chnh v vy c 3 vn m HMM cn gii quyt: Cung cp cho m hnh cc tham s, tnh xc sut ca dy u ra c th. Gii bng thut ton tin trc. Cung cp cho m hnh cc tham s, tm dy cc trng thi (n) c kh nng ln nht m c th sinh ra dy u ra cung cp. Gii bng thut ton Viterbi. Cung cp dy u ra, tm tp hp c kh nng nht ca chuyn tip trng thi v cc xc sut u ra. Gii bng thut ton Baum-Welch.Trong bi ton nhn dng ting ni, gii php cho vn th ba s p dng cho phn training v gii php cho vn hai l phn decode gn nhn thch hp nht cho tn hiu, hai thut ton Viterbi v Baum-Welch s c trnh by vo cc phn ngay sau y.

p dng m hnh HMM cho bi ton nhn dng ting ni, s dng trch chn c trng MFCC [footnoteRef:7] [7: Ni dung tham kho trong cun Speech and Language Processing, 2007, chapter 9, tc gi Daniel Jurafsky & Jame H.Martin.]

M hnh ha nhn dng ting ni bng HMM

M hnh haBy gi, ta s m hnh ha bi ton nhn dng ting ni mt cch khi qut bng HMM (cc cng thc v k hiu t mc ny s s dng ng nht):Tp trng thi Q: l trng thi cc thnh phn trong qu trnh Markov c th c:Q = q1 q2 qn y, trng thi chnh l cc m v khc nhau ca ton b t vng, bi ta ct tn hiu ting ni thnh cc frame, cc frame ny mang cc c trng pht m v cm nhn m ca mt m v (khi nim m v c nhc ti t Chng 2).Ta coi mi mt t khi ni l mt chui trng thi theo thi gian:O = o1 o2 ot Trong quan st th i: oi chnh l 1 frame ting ni sau khi p dng k thut trch chn MFCC, c trng bi 1 vector 39 chiu s thc. Ting ni l tp hp cc frame ting ni lin tip nh vy.Tp t vng: V = v1 v2 vvChng hn trong bi ton nhn dng ch s, tp t vng s l khong, mot, hai, ba, , chin y cng chnh l tp t vng ca h thng ta ang xy dng.Ma trn chuyn trng thiA = a01 a02 an1 annTrong aij l xc sut chuyn t trng thi i sang trng thi j i vi mt quan st bt k.

Ma trn likelihood cho dy quan st O, y bj(ot) l xc sut trng thi j nhn quan st th t (thi im t), cng c th hiu y l xc sut quan st ot nhn trng thi j.B = bj(ot)Ngoi ra, ta b sung 2 trng thi q0, qend c lp vi cc trng thi q1 qn vi ngha l trng thi bt u v kt thc cho mi dy quan st.

Vi mi tn hiu c c trng bi chui quan st O, ta cn tnh c cc xc sut A v B phc v cho qu trnh nhn dng, trong mt s bi ton ng dng HMM bnh thng, cc dy quan st trong tp training ta thng bit trc c trng thi ca mi quan st nh vic ch ng a vo cc dy quan st theo mun.Tuy nhin trong bi ton nhn dng ting ni, O l tn hiu lin tc, c ri rc ha v trch chn thnh cc frame quan st oi, vi mt tn hiu ting ni ta bit trc l t g, t c cu thnh bi bao nhiu subphone, nhng khng h bit bao frame no thuc subphone no. Chng hn t MOT c trch chn v ri rc bi 30 frames quan st. Cc m v cu thnh t mot l m oo t nhng ta khng h bit l bao nhiu frame u tin c trng thi m, bao nhiu frame tip theo c trng thi oo v bao nhiu frame cui c trng thi t. Vic gn nhn th cng - handed labeling cho tp training l thc hin c nhng rt tn cng sc ( gn nhn cho 1 gi ting ni ta s phi tn 400 gi gn nhn bng tay), b li vic gn nhn th cng s lm vic tnh cc xc sut A, B tr nn n gin v chnh xc hn, tuy nhin, trong bi ton m l tp t vng tng ln rt ln, tp training cn ln hn nhiu, chnh v vy mt phng php gn nhn t ng mang tn Embeded training c a ra nhm tng tc qu trnh gn nhn cho qu trnh training. Phng php ny s c gii thiu phn sau.

Hun luyn (Training) [footnoteRef:8] [8: Ni dung nghin cu chung vi kha lun Cc k thut nhn dng ting ni, 2010, sinh vin m Tin Dng, i Hc Cng Ngh.]

Mc ny s cp n vn hun luyn HMM, c coi l phn kh nht trong ba vn c nu ra phn ny. Nhim v ca vic hun luyn HMM l iu chnh cc tham s m hnh (A, B, ) t c mt m hnh ti u nht cho cc mu hun luyn. Mc tiu cui cng l a ra c cc tham s A, B (vi ngha nu trn) cho cc mu hun luyn. C kh nhiu k thut c a ra cho vn ny, tuy nhin trong mc ny ti s ch gii thiu mt k thut hun luyn kh thng dng, l k thut hun luyn s dng thut ton Baum-Welch, hay cn gi l thut ton Forward-Backward, mt trng hp ring ca thut ton ti u ha k vng (Expectation Maximization Algorithm). Thut ton ny da trn phng php lp t c cc t a phng ca hm xc sut P(O|) - xc sut ca quan st O vi m hnh hin ti. Trong mi vng lp, cc tham s ca m hnh s c iu chnh li, v m hnh mi s tt hn m hnh c, nh c chng minh bi Baum v nhiu ngi khc. Thut ton s dng li khi gp iu kin hi t, tc l khi xc sut m hnh P(O|) khng tng na hoc tng rt t, hoc khi gp phi iu kin ti hn ca tnh ton. M hnh lun lun hi t, tuy nhin ch c th m bo gi tr t c ca P(O|) l mt cc i a phng.Trc khi i vo thut ton c th, cn nh ngha hai xc sut: t(i) v t(i,j). u tin l xc sut t(i), c ngha l xc sut trng thi i ti quan st th t, vi mt dy quan st v m hnh cho trc:

V v nn ta c:

Xc sut th hai l t(i,j), c ngha l xc sut trng thi i ti quan st th t v trng thi j ti quan st th t+1, vi m hnh v dy quan st O cho trc:

Theo cng thc Bayes ta c:

S dng cc thut ton Forward v Backward, ta tnh c:

Do vy ta c:

Nu ta cng t(i) i vi tt c cc gi tr ca t (ngoi tr t=T), ta s thu c mt kt qu l gi tr k vng v s ln m trng thi i on nhn mt quan st trn tt c cc quan st. Mt khc, nu ta cng t(i,j) i vi tt c cc gi tr ca t (ngoi tr t=T), ta s thu c mt gi tr k vng v s ln trng thi i chuyn sang trng thi j. T nhng nh gi trn, cc tham s ca m hnh s c tnh ton li nh sau:

y, du ^ th hin cc tham s mi sau khi iu chnh li. Sau khi cp nht li cc tham s nh trn, chng ta s thu c mt m hnh mi ph hp hn m hnh c , i vi dy quan st O:

C th thut ton Baum-Welch c m t nh sau:1. Khi to A, B v .2. Lp: Bc k vng: tnh cc gi tr t(i) v t(i,j). Bc ti u ha: tnh li cc tham s A, B v .

Hun luyn nhng (Embedded training)C th v thut ton Baum-Welch trong bi ton nhn dng ting ni s c m t trong phn ny.Trong thc t, gn nhn cho mt tp d liu di mt ting, c th cn thi gian ln n 400 ting. Do phng php hun luyn hand-labeled word training l khng kh thi trong cc bi ton c b t vng ln. Mt k thut hun luyn khc, khng i hi d liu gn nhn sn, c xy dng da trn thut ton Baum-Welch (mc 4.2.1.2), l phng php hun luyn nhng (Embedded training). Phng php ny gm hai bc nh sau:a. Xy dng m hnh Markov n =(A,B, ) cho t cn hun luyn. Cc tham s A, B v c khi to nh sau: i s c gi tr bng 1 nu m hnh bt u vi trng thi i, ngc li i = 0. aj s c gi tr bng 0.5 nu i = j hoc bc chuyn t i sang j l mt bc chuyn tn ti trong m hnh, ngc li aj = 0. Ring ann s bng 1. Khi to cc gi tr k vng v phng sai , trong gi tr k vng v phng sai cho mi trng thi s l k vng v phng sai ca tt c cc vector u vo. Sau tnh bj(ot) da vo v .bjot = exp( - )Ta s gii thch k hn cng thc ny phn sau.b. Chy thut ton Baum-Welch cho m hnh .Bng sau m t HMM cho t MOT v cc gi tr khi to ca A v :

mot

100

mot

m0.50.50

o00.50.5

t001

. . . . . . . . .

Nh vy, mi mu ting ni training s c mt ma trn chuyn trng thi A (transition matrix) v mt ma trn phn phi trng thi B (observation likelihood) cho mi mu. By gi, vi mt mu ting ni O nm ngoi tp training cn nhn dng, ta cn tnh li ma trn A v phn phi cc trng thi vi mu ny i vi tp training xy dng. Cch tnh 2 ma trn ny nh sau:Gi Ak(i,j), Bk(j, t), Sk (k = 1..M vi M l s mu training) ln lt l ma trn chuyn trng thi, phn phi cc trng thi v s lng quan st ca mu th k trong tp training.

Ma trn chuyn trng thi A ca ton tp training c tnh theo cng thc:Aij = Tnh ma trn phn phi cc trng thi cho dy quan st O kh khn hn mt cht. Ta c cc ma trn Bk(j, t) vi ngha ti mu th k, xc sut quan st t c trng thi j l Bk(j, t). Nh vy, nu coi mi trng thi l mt bin ngu nhin nhn mt gi tr ot no , th mi trng thi s c mt phn phi nht nh. y ta gn phn phi ny l phn phi chun vi gi tr k vng v phng sai .Phn phi ca ta nh sau: f (x | | = exp( - )vi x y tm coi l gi tr trong khng gian vector ca bin quan st ot. Nh vy:bjot = exp( - ) c th y ot l mt vector 39 chiu thc, do bjot c tnh li theo cng thc phn phi i vi vector a chiubjot = y v ln lt l k vng v phn phi ca thnh phn th d i vi trng thi th j, hai gi tr ny c tnh theo cng thc k vng v phng sai i trong khng gian mt chiu: = = y Xjd chnh l gi tr chiu th d ca tt c cc bin ot ca ton b cc mu trong tp hun luyn v xc sut tng ng vi ot ca mu th k chnh bng Bk(j, t). Nh vy, cc xc sut trn u c th tnh ton c, do ta hon ton c c ma trn xc sut A, B nh yu cu ban u.

Nhn dng (Decoding)Thut ton Viterbi:Phn Decoding nhn u vo l mt dy quan st O = o1 o2 ... oT (c trng cho tn hiu ting ni) v cho ra mt dy c gn trng thi c xc sut ln nht i vi tp training. lm c iu ny, ta s dng gii thut Viterbi.Gi s A = aij v B = bj(ot) tng ng l ma trn chuyn trng thi ca tp training v phn phi cc trng thi i vi dy quan st O.Gi F = f(j, t) l xc sut ln nht quan st th t c trng thi j, vi gi thit cc ok (k