Ch01_Introduction to Pattern Recognition (modified from 패턴인식개론/한학용)
2
Contents
01_ Philosophical Debates on AI
02_ Pattern Recognition (PR)
03_ Features and Patterns
04_ Components of PR and Design Cycle
05_ Category of PR and Classifiers
06_ Performance Evaluation of PR Algorithms
07_ Approaches of PR and its Application Areas
08_ Example of PR Applications
3
01_Philosophical Debates on AI
Questions Is computer merely a calculating machine?
Can computer think and understand languages like human?
4
01_Philosophical Debates on AI
Positive opinions on the possibility of AI
Negative opinions on the possibility of AI
Imitation Game A.M. Turing(1912~1954)
Chinese Room Arguments John Searle(1932~ )
5
02_Definition of Pattern Recognition
What is PR? An area of AI that deals with the problems to make computable machines
(Turing Machines) to recognize certain objects
PR
AI
Cognitive Science
6
03_Features and Patterns
What is feature? Discernible aspects, qualities, characteristics that a certain object has
What is pattern? A set of traits or features of individual objects
7
03_Features and Patterns
Easy features and difficult features
Categories of patterns
8
04_Components of PR and Design Cycle
Components of PR System and its Process
9
04_ Components of PR and Design Cycle
Design steps of PR system
Step 1 : Data gathering Most time-consuming tedious process in PR tasks
Necessary step to ensure stable PR performance
For stable performance, we need to consider how many samples are needed before the gathering.
Step 2 : Feature selection Essential part regarding PR system’s performance
We need to decide what features to choose through sufficient prior analysis on the object patters.
Step 3 : Model selection To decide what approach (model and algorithm) is to be constructed and applied
Need prior knowledge on the features
Need to set up parameters for the model according to the approach
10
04_ Components of PR and Design Cycle
Step 4 : Learning Using the feature sets extracted from the collected data and chosen models, the
learning algorithm generates or fills up the model (or hypothesis, classifier)
According to the methods, there are supervised learning, unsupervised learning and reinforcement learning.
Step 5 : Recognition Given a new feature set, the generated hypothesis decide a class or category that
the feature set belongs to.
11
05_ Category of PR and Classifiers
Categories of problems
Classification In classification problem, the system needs to output one label in a set of finite
number of labels.
Regression Generalized version of classification
Through regression, the PR system will return a real value score (usually between 0 and 1)
Clustering The problem of organizing a small number of multiple groups from a certain set
The output of clustering system is a set of pairs (example and its class).
The clustering can be processed in an hierarchical manner such as in phylogenetic tree.
Description The problem of expressing an object using a set of a prototype or primitive terms
12
05_ Category of PR and Classifiers
Classifier Most classification task in PR is done by classifiers
Classification is to partitioning a feature space composed of feature vectors into decision regions of nominal classes.
We call the boundaries of the regions as decision boundaries
Classification of a feature vector x is to decide what decision region the feature vector belongs to, and to assign x to the class that represents the region
13
05_ Category of PR and Classifiers
Classifier can be represented as a set of discriminant functions
∀j=i, if 𝑔𝑖 𝑥 > 𝑔𝑗 𝑥 , then we decide that the feature vector 𝑥 ∈ class 𝜔𝑖
14
06_ Performance Evaluation of PR Algorithms
Confusion Matrix
Recall rate = TP
TP+FN
Precision = TP
TP+FP
True Positive Rate (TPR) = TP
TP+FN
False Positive Rate (FPR) = FP
FP+TN
Actual Positive Actual Negative
Predicted Positive TP FP
Predicted Negative FN TN
15
06_ Performance Evaluation of PR Algorithms
Receiver operating characteristic (ROC) Curve
16
06_ Performance Evaluation of PR Algorithms
AUROC Ares under the region of ROC Curve
Closer the curve to top-left corner, more accurate the recognition algorithm
The performance can be evaluated by the amount of AUROC
17
07_ Approaches of PR and its Application Areas
Approaches of PR Template matching
Oldest and easiest
First, prepare the template for the object to compare.
Normalize the pattern to recognize for matching it with the template.
And calculate similarity value such as cross-correlation or distance to perform the recognition
Most important task is to prepare the most general template that explains all the samples in a certain category.
Fast running time, but weak in variation of features
Statistical approaches Decide the class of unknown pattern bases on decision boundaries of pattern sets.
Each of the pattern sets represent a certain class.
The statistical model of the patterns is a probability density function 𝑃 𝑥|𝑐𝑖 .
Learning is a process of creating a probability density function and calculating its parameters for each class
Neural networks Model the relation of connection and integration of the biological neurons
Calculate the response process of neural network for input stimulus
Classify patterns based on the responses
18
07_ Approaches of PR and its Application Areas
Knowledge of the patterns is stored as weights that represent the connection strength of synapse.
Learning is performed similar to biological ways, but the learning process is not a serial algorithm.
The learned knowledge is considered as a black box.
Minimal need for prior knowledge.
With sufficient number of neurons, theoretically any complicated decision boundaries can be constructed, so this approach is very attractive.
Structural approaches Instead of quantitative features, we consider the relationship among the basic
prototypes what construct the pattern.
Examples: Character, Fingerprint, Chromosome
19
07_ Approaches of PR and its Application Areas
Approaches of PR
20
07_ Approaches of PR and its Application Areas
Applications of PR Character recognition
Convert a scanned text image into character codes which can be edited in a computer
Mail classification, Handwriting recognition, Check and banknote recognition, License plate recognition
Biological recognition and human behavioral pattern recognition Voice recognition, fingerprint recognition, face recognition, DNA mapping, walking
pattern analysis and classification, utterance habit analysis and classification
Diagnostic systems Car malfunction, medical diagnostics, EEG, ECG signal analysis and classification, X-
Ray image pattern recognition
21
07_ Approaches of PR and its Application Areas
Prediction system
Weather forecasting based on satellite data, earthquake pattern analysis and earthquake prediction, stock price prediction, etc.
Security and military area Intrusion detection based on network traffic pattern analysis, security screening
system, search and attack of terrorist camp and targets using satellite terrain image analysis, radar signal classification, Identification Friend or Foe (IFF)
22
07_ Approaches of PR and its Application Areas
Related Areas Application Areas
•Adaptive signal processing •Machine learning •Artificial Neural networks •Robotics and Vision •Cognitive science •Mathematical Statistics •Nonlinear optimization •Exploratory Data analysis •Fuzzy and Genetic System •Detection and Estimation Theory •Formal language •Structural modeling •Biological cybernetics •Computational neuroscience
•Image processing/segmentation •Computer Vision •Speech recognition •Automatic target recognition •Optical character recognition •Seismic Analysis •Man-machine interaction •Bio recognition (fingerprint, vein, iris) •Industrial inspection •Financial forecast •Medical analysis •ECG signal analysis
23
08_ Example of PR Applications
Simple English character recognition system feature V : # of vertical lines
feature H : # of horizontal lines
feature O : # of slopes
feature C : # of curves
Character Feature
V H O C
L 1 1 0 0
P 1 0 0 1
O 0 0 0 1
E 1 3 0 0
Q 0 0 1 1
24
08_ Example of PR Applications
Automatic fish classification (Sea Bass or Salmon) A: Conveyor belt for fish
B: Conveyor belt for classified fish
C : Robot arm for grabbing fish
D: Machine vision system with CCD camera
E : Computer that analyze fish image and control the robot arm
25
08_ Example of PR Applications
Automatic fish classification Assume that fish is either salmon or sea bass
Using machine vision system for acquiring new fish image
Normalize the intensities of new fish image using image processing algorithm
Segment fish from the background in the image processing process
Using the prior knowledge that sea bass is bigger than salmon, extract features in the image to measure the length of the new fish
From the training samples of the two fish categories, calculate the distribution of the length, and decide the threshold of decision boundary that minimize the classification error
Accuracy : 60%
26
08_ Example of PR Applications
Adding features for enhancing recognition rate The accuracy should be over 95% for stable pattern recognition system
We find that average intensity level is a good feature.
27
08_ Example of PR Applications
Enhancing the recognition rate We generate 2 dimensional feature vector with length and average intensity.
Using a simple linear discriminant function, we enhance the recognition rate.
Accuracy : 95.7%
28
08_ Example of PR Applications
Cost vs. Classification Rate To minimize the cost, we adjust the decision boundary
29
08_Example of PR Applications
Generalization problem Using neural network, the performance can be enhanced to 99.9975%
Is this a good result?