萬能科技大學資工系 助理教授 徐旺興 email: [email protected] computational...
Post on 20-Dec-2015
282 views
TRANSCRIPT
OutlineMotivationComputational Intelligence
◦Fuzzy, ANFIS, SVM, HMM and GMMFrequency Calibration based on the
ANFISHandwriting Recognition on
Handheld Devices using Accelerometers
ConclusionFuture work
Motivation (1/2)Studied in past years
◦QoS: Simulation◦NGN: Framework improvement◦SIP: multimedia (client/server)◦Handoff: BS, Agent or Broker and
Mobile Device◦Time & Frequency: Control system◦3D Handwriting recognition
Motivation (2/2)Issue of time series
What is Computational IntelligenceCI related to other branches of
computer science, such as artificial intelligence (AI), classification, data mining, graphical methods, intelligent agents and intelligent systems, machine intelligence, machine learning, natural computing, parallel distributed processing, pattern recognition, probabilistic methods, soft computing, multivariate statistics, optimization and operation research.
Frequency Calibration based on the ANFIS
Implement a control systemNormal mode and holdover modeTechnology of CI
Fuzzy - a mathematical system that analyzes analog input values in terms of logical variables that take on continuous values in interval [0,1], in contrast to classical or digital logic, which operates on discrete values of either 0 or 1 (true or false).
ANFIS - learns features in the data set and adjusts the system parameters according to a given error criterion.
The requirements of time and frequency accuracy for the dominant wirelesstechnologies.
OCXO (Oven-Controlled Crystal Oscillator)
The stability of OCXO base on environmental effects such as vibration, temperature, pressure and humidity.
System architecture (1/2)
Voltage: +/-10V
Slave Clock1pps1pps
1pps=one plus per second
Frequencyoffset
System architecture (2/2)The frequency offset with respect to time Their change are the input variables of the fuzzy controller
)( ity
)()()( 1 iii tytyty
An incremental voltage generated by the fuzzy controller is used to update the voltage for steering the oscillator below.
)( itV
)(1 iii tVVV
Fuzzy rule table
NBNBNBNBNBPB
NBNSNSZEZEPS
NSNSZEPSPSZE
ZEZEPSPSPBNS
PBPBPBPBPBNB
PBPSZENSNB
NBNBNBNBNBPB
NBNSNSZEZEPS
NSNSZEPSPSZE
ZEZEPSPSPBNS
PBPBPBPBPBNB
PBPSZENSNBy
y
Ri : if y is Ai1 and y is Ai2 then u is Bi, for i 1,2…n
Ai1
Ai2
Bi
The input space is divided into five sets: negative big (NB), negative small (NS), zero (ZE), positive small (PS) and positive big (PB) for a frequency offset or its change.
Component of the system (1/2)Cesium (HP5071A)
◦10 MHz of a cesium atomic clock (10-
14)OCXO (FTS1130)
◦10 MHz of oven-controlled crystal oscillator (10-8)
TIC (SR620)◦time interval counter◦Time interval and frequency counter
Component of the system (1/2)D/A
◦ADLINK PCI-6208◦16-bit resolution with the bi-polar ◦Voltage: 10V to +10V
Fuzzy controller◦Software coding by C/C++, Matlab
ANFIS controller◦Software coding by C/C++, Matlab
Two mode in this systemNormal Mode
◦Fuzzy controller◦Collecting the control signal◦To train the ANFIS controller
simultaneouslyHandover Mode
◦ANFIS controller◦When the signal of the primary clock
is lost◦The voltage (control signal) is
predicted by the ANFIS controller
Experimental setupThe OCXO is steered every 10s
by the fuzzy controller or the ANFIS controller to syntonize with the primary clock.
Choice about five-day input-output data pairs:
The first four-day pairs were used for training the ANFIS.
The remaining about one-day pairs were used for validating the identified model.
)]();(),(),2(),3([ txtxtxtxtx
Experimental results and analysis
The desired data
The predicted
data
The Prediction
error
Frequency stability
100
101
102
103
104
105
10-14
10-13
10-12
10-11
10-10
10-9
Frequency stabilityM
odif
ied
All
an d
evia
tion
Normal
Holdover
Free Running
Averaging time (seconds), Averaging time (seconds),
)( yMod
Conclusion of this chapterThe frequency stability of the
OCXO could be improved from a few parts in 10-9 to 10-12 over a measurement period of one day. (Normal Mode)
Holdover Mode shows the frequency stability of the OCXO could be maintained within a few parts in 10-11 for an averaging time of on day.
3D Handwriting RecognitionAccelerometer3D gesturePattern RecognitionMobile Device’s Accelerometer
(30Hz~70Hz)Device
◦HTC G1Software component
◦Collectors (Java code)◦Training and recognition (matlab code), off-line.
Proposed methodWLCS + SVMHMM + GMM
The architecture of the proposed 3D handwriting recognition system
Three axes acceleration data of pattern ‘Kimble’.
Data preprocessing
Data Training
Longest Common Subsequence （ LCS ）Example of LCSs1: 2 5 7 9 3 1 2s2: 3 5 3 2 8
LCS: 5 3 2
Weight LCS
Data Classification – SVM (1/3)
To Find the Hyper-plane (e.g. 2D’s hyper-plane is a line)
SVM (2/3)
To Find the optimal Hyper-plan H
SVM (3/3)
Experimental setupWe collect a set of 26 gestures
(alphabet), 20 samples per gesture from 3 different persons, totaling 1560 gestures samples.
50 samples for training and 10 samples for testing.
The average length of the WLCS between letters
Models
Test data
Performance CriteriaThe classification performance
can be evaluated using mis-classification rate such as apparent error rate and/or graphical representation tools such as the receiver operating characteristic (ROC) curve.
Terms associated ROC curve
An example of ROC
The table shows 20 data and the score assigned to each by a scoring classifier
Sorting by score
ROC curve of the example
10 positive points at
x-axis
10 positive points at y-
axis
Max. and Min. AUC (Area under curve)
The Max. AUC is
alphabet’C’.
The Min. AUC is
alphabet ’G’
List of AUC from ‘A’ to ‘Z’the alphabet
such ‘C’, ‘L’, ‘P’, ‘S’, ‘U’, ‘V’ and ‘Z’ is good instance and
’G’ is a randomly chosen negative instance.
Summary about LCS +SVMLCS + SVM is the lite-computing
algorithm, the average accuracy is 86.85%
Hidden Markov Model (HMM)HMMs allow you to estimate
probabilities of unobserved events.
Given plain text, which underlying parameters generated the surface.
E.g., in speech recognition, the observed data is the acoustic signal and the words are the hidden parameters.
HMMs and their UsageHMMs are very common in
Computational Linguistics:◦ Speech recognition (observed: acoustic
signal, hidden: words)◦ Handwriting recognition (observed: image,
hidden: words)
Parameters of an HMMStates: A set of states S=s1,…,sn
Transition probabilities: A= a1,1,a1,2,…,an,n Each ai,j represents the probability of transitioning from state si to sj.
Emission probabilities: A set B of functions of the form bi(ot) which is the probability of observation ot being emitted by si
Initial state distribution: is the probability that si is a start state
i
The Three Basic HMM Problems (1/2)Problem 1 (Evaluation): Given the
observation sequence O=o1,…,oT and an HMM model
, how do we compute the probability of O given the model?
Problem 2 (Decoding): Given the observation sequence O=o1,…,oT and an HMM model
, how do we find the state sequence that best explains the observations?
(A,B, )
(A,B, )
Problem 3 (Learning): How do we adjust the model parameters , to maximize ?
The Three Basic HMM Problems (2/2)
(A,B, )
P(O | )
Example of HMM
The states
The observations
An Example model, the semi-code of HMM
The dynamic programming computation
Diary data and reconstructed weather
In this workGiven the observation sequence
O=o1,…,oT, ◦e.g.
5555222233344433377001111111….
Build 26 Model HHMA, HHMB, … HMMz
Training each model by EM algorithm. (Problem 3)
Recognition, compute the probability of O given the model. (Problem 1) (Forward-Backward Algorithm)
GMM – Gaussian Mixture model
Experimental setupWe collect a set of 26 gestures
(alphabet), 20 samples per gesture from 3 different persons, totaling 1560 gestures samples.
The list of log likelihood compare with “Model” and “Test Data”
Summary about HMM+GMMUsing GMM, the accuracy of
classification could be achieved at about 96.5%.
CONCLUSIONFrequency calibration:
◦This study demonstrates the feasibility of using the ANFIS for frequency calibration.
◦The frequency stability of the OCXO can be significantly improved more than three orders of magnitude in both normal mode and holdover mode.
3D handwriting gesture recognition: ◦This study demonstrates the feasibility of
using◦method of (SVM and LCS), and using method
of (HMM and GMM) for 3D handwriting gesture recognition.
Future workNew Tools of Computational
Intelligence ◦HHT(Hilbert Huang Transform)◦http://rcada.ncu.edu.tw/
New Area about signal process◦PhysioNet
PhysioNet包含：
◦PhysioNet ： the research resource for complex physiologic signals Publications Tutorials Challenges
◦PhysioBank ： database◦PhysioToolkit: software