DSP HW2-1HMM Training and Testing

教授：李琳山助教：王君璇

Outline1. Introduction2. HiddenMarkovModelToolkit(HTK)3. HomeworkProblems4. SubmissionRequirements

Introduction● Constructadigitrecognizer- monophone

ling|yi|er|san|si|wu|liu|qi|ba|jiu

● FreetoolsofHMM:HiddenMarkovToolkit(HTK)http://htk.eng.cam.ac.uk/

● Trainingdata,testingdata,scripts,andotherresourcesallareavailableonhttp://speech.ee.ntu.edu.tw/DSP2019Spring/

Flowchart

Hidden Markov Model Toolkit (HTK)

Feature Extraction

Feature Extraction - HCopy

Convertwaveto39dimensionMFCC.-C lib/hcopy.cfg● inputandoutputformat● parametersoffeatureextraction● Chapter7- SpeechSignalsandFront-endProcessing

-S scripts/training_hcopy.scp● amappingfromInputfilenametooutputfilename

speechdata/training/N110022.wav

MFCC/training/N110022.mfc

Training Flowchart

Training Flowchart

Initialize model - HCompV

Computeglobalmeanandvarianceoffeatures-C lib/config.cfg

● setformatofinputfeature(MFCC_Z_E_D_A)-o hmmdef -M hmm

● setoutputname:hmm/hmmdef-S scripts/training.scp

● alistoftrainingdatalib/proto

● adescriptionofaHMMmodel,HTKMMFformat⇨ youcanmodifytheModelFormathere(#states)!

Initial MMF PrototypeMMF:HTKBookchapter7

Initial HMM● bin/macro

ProduceMMFcontainsvFloor

● bin/models_1mixsiladdsilenceHMM

hmm/hmmdef

hmm/models

Training Flowchart

Adjust HMMs - HERest

Basicproblem3forHMM● GivenOandaninitialmodelλ=(A,B,π),adjustλtomaximizeP(O|λ)

Adjust HMMs - HERest

AdjustparametersλtomaximizeP(O|λ)● oneiterationofEMalgorithm● runthiscommandthreetimes=>threeiterations

–I labels/Clean08TR.mlf● setlabelfileto“labels/Clean08TR.mlf”

-o lib/models.lst● alistofwordmodels(liN(零),#i(一),#er(二),…jiou(九),sil)

Add SP Model

Add ”sp”(shortpause)HMMdefinitiontoMMFfile“hmm/hmmdef”

Modify HMMs - HHEd

lib/sil1.hed● alistofcommandtomodifyHMMdefinitions

lib/models_sp.lst● anewlistofmodel(liN(零),#i(一),#er(二),…jiou(九),sil,sp)

Training Flowchart

Adjust HMMs Again - HERest

Increase Number of Mixtures - HHEd

Modification of Models

You can modify # of Gaussian mixture here.

This value tells HTK to change the mixture number from state 2 to state 4. If you want to change # state, check lib/proto.

You can increase # Gaussian mixture here.

Adjust HMMs Again - HERest

Training Flowchart

Hint：Increasemixtureslittlebylittle!

Testing Flowchart

Construct Word Net - HParse

lib/grammar_sp● regularexpression● easyforusertoconstruct

lib/wdnet_sp● outputwordnet● theformatthatHTKunderstand

Viterbi Search - HVite

-w lib/wdnet_sp● inputwordnet

-i result/result.mlf● outputMLFfile

lib/dict● dictionary:amappingfromwordtophonesequences

ling->liN,er->#er,….一 ->sic_ii,七->chi_ii

Compared With Answer - HResults

LongestCommonSubsequence(LCS)

Ref：See HTK book 3.2.2 (p. 33)

Report - Part 1 (40%) - Run Baseline

1. DownloadHTKtools(recommend:compiledbinary)andhomeworkpackage

2. SetPATHforHTKtools：set_htk_path.sh

3. Execute(bashshellscript)01_run_HCopy.sh02_run_HCompV.sh03_training.sh04_testing.sh

Report - Part 1 (40%) - Run Baseline (cont.)

3. Youcanfindaccuracyin“result/accuracy”thebaselineaccuracyis74.34%

4. Putthescreenshotofyourresultonthereport.

Useful tips1. Tounzipfiles

unzipXXXX.ziptar-zxvfXXXX.tar.gz

2. Tosetpathin“set_htk_path.sh”PATH=$PATH:“~/XXXX/XXXX”

3. Incaseshellscriptisnotpermittedtorun…chmod744XXXX.sh

Useful tips

4. IfyouencounterNosuchfileordirectoryonthecompiledbinary files,itisbecauseyouaretryingtoruna32-bitbinaryona64-bitsystemthatdoesn'thave32-bitsupportinstalled.Youmayneedtoinstalllibrarypackagessuchaslibc6:i386,libncurses5:i386,andlibstdc++6:i386.

Report - Part 2 (40%) - Improve Accuracy● Acc>95%forfullcredit;90~95%forpartialcredit

andputthescreenshotofyourresultonthereport.

proto 03_training.sh,mix2_10.hed...

Part 2 - Attention 1● Executing03_training.shtwiceisdifferentfrom

doublingthenumberoftrainingiterations.Toincreasethenumberoftrainingiterations,pleasemodifythescript,ratherthanrunitmanytimes.

Part 2 - Attention 2● Everytimeyoumodifiedanyparameterorfile,you

shouldrun00_clean_all.sh toremoveallthefilesthatwereproducedbefore,andrestartalltheprocedures.Ifnot,thenewsettingswillbeperformedonthepreviousfiles,andhenceyouwillbenotabletoanalyzethenewresults.(Ofcourse,youshouldrecordyourcurrentresultsbeforestartingthenextexperiment.)

Report - Part 3 (30%)● Writeareportdescribingyourtrainingprocessand

accuracy.Numberofstates,Gaussianmixtures,iterations,…HowsomechangeseffecttheperformanceOtherinterestingdiscoveries

● Well-writtenreportmayget+10%bonus.

Submission Requirements1. 4shellscripts

yourmodified01~04_XXXX.sh

2. 1accuracyfilewithonlyyourbestaccuracy(Thebaselineresultisnotneeded.)

3. proto,mix2_10.hedyourmodifiedhmmprototype andfilewhichspecifiesthenumberofGMMsofeachstate

4. hw2-1_bXXXXXXXX.pdfscreenshotforbaselineandthebestresult,orotherinteresting.

Submission Requirements (cont.)5. Putthose8 filesinafolder,compressthefolderto1

zipfileanduploadittoCEIBA.● FoldernameshouldbebXXXXXXXX (e.g.b04901000orr07922000)

● .ziponly● 20% ofthefinalscorewillbetakenoffforwrongformat

6. Deadline:2019/5/323:59:59● LatePenalty:10%offevery24hoursafterdeadline

(lessthan24hourswillbeviewedas24hours).● Submissionafter3dayswillgetzeropoint.

If you have any problem…● Checkforhintsinthelinuxandshellscripts.ex:鳥哥● ChecktheHTKbook.● AskfriendswhoarefamiliarwithLinuxcommandsor

Cygwin. (link：howtoHTKonCygwin)

Contact TA● email：ntudigitalspeechprocessingta@gmail.com

title:[HW2-1]ProblemDescription● OfficeHour:Monday14:30-15:30電二531王君璇

(Pleasesendanemailbeforecoming!)