1/21 2003 gold prize awarded a self-organized network inspired by immune algorithm...
TRANSCRIPT
1/21
2003 Gold Prize Awarded2003 Gold Prize Awarded A Self-Organized Network A Self-Organized Network Inspired by Immune AlgorithmInspired by Immune Algorithm免疫アルゴリズムに基づく自己組織化ネット免疫アルゴリズムに基づく自己組織化ネットワークワーク
Muhammad Rahmat WIDYANTO (01M35636)Hirota Laboratory
Dept. of Computational Intelligence & Systems Science
Tokyo Institute of Technology
2/21
BackgroundBackground
SONIASONIA [proposed][proposed]
ExperimentsExperiments
ConclusionsConclusions
ContentsContents
3/21
BP-NN BP-NN [Rumelhart, 86][Rumelhart, 86]
Global ResponseGlobal Response
OverfittingOverfitting
Low GeneralizationLow Generalization
Immune Algorithm Immune Algorithm [Timmis, 01][Timmis, 01]
Local ResponseLocal Response
Characteristics only Characteristics only
Diverse RepresentationDiverse Representation
Background Background (1/2)(1/2)
4/21
BP-NNBP-NN Immune Immune AlgorithmAlgorithm
SONIASONIA A A SSelf-elf-OOrganized rganized NNetwork etwork inspired by inspired by IImmune mmune AAlgorithmlgorithm[proposed][proposed]
Better RecognitionBetter Recognition
Better Generalization Better Generalization
Background Background (2/2)(2/2)
5/21
・・・
・・・
・・・
Input layer
Hiddenlayer
Outputlayer
BP-NN :BP-NN :[Rumelhart, 86][Rumelhart, 86]
SONIA SONIA [proposed] (1/5)[proposed] (1/5)
Input Vector Hidden Unit
Antigen
Immune Immune AlgorithmAlgorithm ::[Timmis, 01][Timmis, 01] Recognition Ball (RB)
6/21
Input Vector
Unit Centroid
Hidden UnitHidden UnitRecognition Ball (RB)Recognition Ball (RB)
B CellAntibody
Antigen
Paratope
Epitope EuclidianDistance
SONIA SONIA [proposed] (2/5)[proposed] (2/5)
7/21
Antibody GenerationAntibody Generation
Antigen[1..m]
SONIA SONIA [proposed] (3/5)[proposed] (3/5)
Input Vector [1..m]
Hidden Unit CreationHidden Unit Creation
Hidden Unit 1
Hidden Unit 2
Hidden Unit i
MutatedHidden Unit n
RB 2
RB i
Mutated RB n
RB 1
B Cell Mutation
B Cell Construction
8/21
Initialization, 1st Unit :1. Number of Vectors2. Unit Centroid
1. Distance Calculation2. Minimum Distance
Next input From first input
Hidden Unit CreationHidden Unit Creation
Distance ? Stimulation
SONIA SONIA [proposed] (4/5)[proposed] (4/5)
A New Unit1. Number of Vectors2. Unit Centroid
Unit Updating 1. Number of Vectors2. Unit Centroid
9/21
New Units:1. Initial Point2. Ending Point
Distance > LevelNumber of Vectors < Threshold
For every two adjacent units
No
A Mutated Unit1. Number of Vectors2. Unit Centroid
Yes
SONIA SONIA [proposed] (5/5)[proposed] (5/5)Mutated Hidden Mutated Hidden Unit CreationUnit Creation
10/21
ExperimentsExperiments
Sinusoidal ProblemSinusoidal Problem [Holmstrom, 92] [Karystinos, 00][Holmstrom, 92] [Karystinos, 00]
hh((xx) = 0.4 sin (2) = 0.4 sin (2xx) + 0.5, ) + 0.5, xx [0,1] [0,1]
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
h(x)
x
Real function
Training DataTraining Data
11/21
Bayesian Regularization Bayesian Regularization [MacKay, 92][MacKay, 92]
Improving GeneralizationImproving Generalization
Automatic Determination of Automatic Determination of
MSEreg = MSEreg = MSE + (1- MSE + (1- ) MSW) MSW
ExperimentsExperiments
12/21
BP-NN BP-NN [Rumelhart, 86][Rumelhart, 86]
Approximation Error : 0.03414
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
h(x)
Training Data
Approximation
BP-NN Regularization BP-NN Regularization [MacKay, 92][MacKay, 92]
Approximation Error : 0.00392
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
h(x)
SONIA without mutation SONIA without mutation Approximation Error : 0.01022
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
h(x)
SONIA with mutation SONIA with mutation [Proposed][Proposed]
Approximation Error : 0.00142
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
h(x)
13/21
Improving Generalization
ExperimentsExperimentsEffect of Mutated Hidden Unit Effect of Mutated Hidden Unit
SONIA without SONIA without Mutated Hidden UnitMutated Hidden Unit
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
h(x)
SONIA with SONIA with
Mutated Hidden Unit Mutated Hidden Unit [proposed][proposed]
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
h(x)
Training Data
Approximation
14/21
ExperimentsExperimentsMinistry of Agriculture ProjectMinistry of Agriculture Project
Supermarket
Food StoreMarket Area
Production Area
Frozen Truck
Perishable Food
Quality Control Server
Prediction Engine:Neural Networks
Quality Quality CheckCheck
15/21
Akita
Chiba
ExperimentsExperimentsReal Food Quality Control Data Real Food Quality Control Data
Data LodgerData Lodger Channel 1 : Meat surface Channel 2 : Packaging boxOctober – December 2001October – December 2001 15 Delivery Data
16/21
Data Collection :Data Lodger
011016_t
- 5051015202530
1 23
45
67
89
111
133
155
177
199
Series1Series2
Channel 1
Channel 2
Time-temperature Data
Time
oC( X 5 Minutes )
Feature Extraction :Mean & Standard Deviation
Pre-Processing :Range Selection
Range Selected
ExperimentsExperimentsPrediction System Prediction System [proposed][proposed]
A B C D E
Neural Networks
ch1:Mean
ch1:SD
ch2:Mean
ch2:SD
Quality
good
17/21
TOP
MIDDLE
BOTTOM
100 93.3 100
73.3 73.3 80
0
20
40
60
80
100
120
TOP MIDDLE BOTTOMRegions in Truck
Correctness (%)
SONIA BP
ExperimentsExperimentsRecognition PercentageRecognition Percentage
18/21
SONIA is proposed SONIA is proposed
20% 20% Recognition Improvement Recognition Improvement
ConclusionsConclusions
Approximation Error is 1/24 times lowerApproximation Error is 1/24 times lower
The World First Time-temperature basedThe World First Time-temperature basedFood Quality Control ApplicationFood Quality Control Application
19/21
Research Plan Research Plan (1/2)(1/2)
Forest Fires and Rainfall PredictionForest Fires and Rainfall Prediction
20/21
Prof. Hirota Laboratory Tokyo Institute of Technology
Prediction Engine Analysis
Dr. Kusumputro LaboratoryThe University of Indonesia
Data Collection Pre-Processing
Research Plan Research Plan (2/2)(2/2)
International Collaborative ResearchInternational Collaborative Research
21/21
International Conference PapersInternational Conference Papers R. Widyanto, Megawati, Y. Takama, and K. Hirota, "A time-temperature-based
food quality prediction using a self-organized network inspired by immune algorithm", In Proceedings of the 1st International Conference on Soft Computing and Intelligent Systems, Tsukuba, Japan, 2002.
R. Widyanto, Megawati, K. Kawamoto, and K. Hirota, "Clustering analysis using a self-organized network inspired by immune algorithm", In Proceedings of the IASTED International Conference on Artificial and Computational Intelligence, Tokyo, Japan, ACTA Press, pp. 197-202, 2002.
Other Presentation & Research ReportOther Presentation & Research Report R. Widyanto, Megawati, Y. Takama and K. Hirota, "Quality prediction of food
product based on time-temperature data using SONIA neural network", Final Research Report, Japan Ministry of Agriculture, Japan, 2002.
PublicationsPublications
22/21
SONIA SONIA [proposed][proposed]
Hidden Unit
w1
Input Vector
s1
s2
x1
w2
Local Response (SONIA) x1 = f( ((s1 - w1)2+ ( s2 - w2) 2)1/2 ) f : tangent sigmoid function
Global Response (BP-NN) x1 = g( s1w1+ s2w2 + ) g : log sigmoid function
23/21
PC Pentium IV 2 Ghz 128 MB Memory Matlab 6.1 C/C++ Library on Windows 2000 SONIA : Stimulation = 0.05, Level = 0.05,
Threshold = 2 SONIA without mutation : 10 hidden units SONIA without mutation : 27 hidden units BP and BP with regularization : 27 hidden units 1000 iterations for 10 trials
ExperimentsExperimentsSinusoidal ProblemSinusoidal Problem
24/21
Method Learning Time (seconds)
BP-NN 8.42
BP-NN with Regularization 19.72
SONIA without Mutation 11.07
SONIA with Mutation 27.83
ExperimentsExperimentsLearning TimeLearning Time
25/21
Depends on random initialization
2 of 10 trials fail for correct approximation
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
ExperimentsExperimentsBayesian Regularization Bayesian Regularization [MacKay, 92][MacKay, 92]
26/21
PC Pentium III 600 Mhz 64 MB MemoryMatlab 6.1 C/C++ Library on Windows
2000SONIA : Stimulation = 0.03, 9 neuronsSONIA with mutation cannot be appliedBP-NN : 9 neurons2600 iterations
ExperimentsExperimentsReal Food Quality Control DataReal Food Quality Control Data
27/21
Region SONIA
(seconds)
BP-NN (seconds)
TOP 60.69 34.72
MIDDLE 38.95 43.77
BOTTOM 54.65 44.21
Average 51.43 40.9
ExperimentsExperimentsLearning TimeLearning Time