數值方法 2008, applied mathematics ndhu 1 nonlinear recursive relations nonlinear recursive...
TRANSCRIPT
數值方法 2008, Applied Mathematics NDHU 1
Nonlinear Recursive relations
Nonlinear recursive relationsTime series predictionHill-Valley classificationLorenz series
Neural Recursions
數值方法 2008, Applied Mathematics NDHU 2
Nonlinear recursion
Post-nonlinear combination of predecessors
z[t]=tanh(a1z[t-1]+ a2z[t-2]+…+ az[t-])+ e[t],
t= ,…,N
數值方法 2008, Applied Mathematics NDHU 3
Post-nonlinear Projection
tanhy
數值方法 2008, Applied Mathematics NDHU 4
Data generation
z[t]=tanh(a1z[t-1]+ a2z[t-2]+…+ az[t-])+ e[t],
t= ,…,N
Taa ],...,[ 1 a
tanh
][tz
數值方法 2008, Applied Mathematics NDHU 5
Nonlinear Recursion
LM learning
數值方法 2008, Applied Mathematics NDHU 6
Prediction
Use initial -1 instances to generate the full time series (red) based on estimated post-nonlinear filter
Post-nonlinear filter
數值方法 2008, Applied Mathematics NDHU 7
UCI Machine Learning Repository
Nonlinear recursions: hill-valley
Classify hill and valley
數值方法 2008, Applied Mathematics NDHU 8
Noise-free Hill Valley
數值方法 2008, Applied Mathematics NDHU 9
Noise-free Hill Valley
數值方法 2008, Applied Mathematics NDHU 10
Hill-Valley with noise
數值方法 2008, Applied Mathematics NDHU 11
Hill-Valley with noise
數值方法 2008, Applied Mathematics NDHU 12
Hill-Valley with noise
數值方法 2008, Applied Mathematics NDHU 13
Nonlinear recursion
LM_RBF: z[t]=F(z[t-1], z[t-2], …,z[t-])+ e[t],
t= ,…,N
][tzLM_RBF 1
數值方法 2008, Applied Mathematics NDHU 14
][tzLM_RBF 1
][tzLM_RBF 2
數值方法 2008, Applied Mathematics NDHU 15
Nonlinear Recursion
LM learning
MLPotts 1 (H)
數值方法 2008, Applied Mathematics NDHU 16
Nonlinear Recursion
LM learning
MLPotts 2 (V)
數值方法 2008, Applied Mathematics NDHU 17
Approximating error
LM_RBF 1 (H)
Green: Original HillBlue: Approximated HillRed: Approximating error
數值方法 2008, Applied Mathematics NDHU 18
Approximating error
LM_RBF 2 (V)
Green: Original HillBlue: Approximated HillRed: Approximating error
數值方法 2008, Applied Mathematics NDHU 19
Separable 2D diagram of Hill-Valley
Noise-free data set 1
LM_RBF 2 (V)
LM_RBF 1 (H)
Mean Approximating Errors ofTwo Models
數值方法 2008, Applied Mathematics NDHU 20
Separable 2D diagram of Hill-Valley
Noise-free data set 2
Mean Approximating Errors ofTwo Models
LM_RBF 2 (V)
LM_RBF 1 (H)
數值方法 2008, Applied Mathematics NDHU 21
Separable 2D diagram of Hill-Valley
Dataset 1 (Noise)
Mean Approximating Errors ofTwo Models
LM_RBF 2 (V)
LM_RBF 1 (H)
數值方法 2008, Applied Mathematics NDHU 22
Separable 2D diagram of Hill-Valley
Dataset 2 (Noise)
Mean Approximating Errors ofTwo Models
LM_RBF 2 (V)
LM_RBF 1 (H)
數值方法 2008, Applied Mathematics NDHU 23
Chaos Time Series
Eric's Home Page
數值方法 2008, Applied Mathematics NDHU 24
Lorenz Attractor
數值方法 2008, Applied Mathematics NDHU 25
Lorenz
>> NN=20000;>> [x,y,z] = lorenz(NN);>> plot3(x,y,z,'.');
3
8:
28:
10:
lz
ly
lx
數值方法 2008, Applied Mathematics NDHU 26
3
8:
13:
10:
lz
ly
lx
13
數值方法 2008, Applied Mathematics NDHU 27
Nonlinear Recursion
Neural nonlinear recursive relations
數值方法 2008, Applied Mathematics NDHU 28
Prediction
Use initial -1 instances to generate the full time series (red) based on derived neural recursions
數值方法 2008, Applied Mathematics NDHU 29
ρ=14, σ=10, β=8/3 ρ=13, σ=10, β=8/3
ρ=15, σ=10, β=8/3 ρ=28, σ=10, β=8/3
Rayleigh number
數值方法 2008, Applied Mathematics NDHU 30
Sunspot time series
1700 1750 1800 1850 1900 19500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
sunpot data
load sunspot_train.mat;n=length(Y);plot(1712:1712+n-1,Y);
數值方法 2008, Applied Mathematics NDHU 31
Sunspot time series
1700 1750 1800 1850 1900 19500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
sunpot data (testing)
load sunspot_test_1.mat;n=length(Y);plot(1952:1952+n-1,Y);
數值方法 2008, Applied Mathematics NDHU 32
1700 1750 1800 1850 1900 19500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
S[1]S[2]S[3]..S[-1]S[]..S[t-]S[t- +1]..S[t-1]S[t]S[t+1]...
x[t]
y[t]
x[-1]
y[-1]
Time-series 2 paired data
數值方法 2008, Applied Mathematics NDHU 33
aM
a2
a1
x
Network
tanh
tanh
tanh
…….
1 b2
b1
bM
1
r1
r2
rM
rM+1
y
數值方法 2008, Applied Mathematics NDHU 34
Paired data : tau=2
x=[];len=3;for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len);endfigure;plot3(x(2,:),x(1,:),y,'.');hold on;
00.2
0.40.6
0.81
0
0.5
10
0.2
0.4
0.6
0.8
1y
x
數值方法 2008, Applied Mathematics NDHU 35
Learning and prediction
M=input('keyin the number of hidden units:');[a,b,r]=learn_MLP(x',y',M);y=eval_MLP2(x,r,a,b,M);hold on;plot(1712+2:1712+2+length(y)-1,y,'r')
MSE for training data 0.005029ME for training data 0.052191K
數值方法 2008, Applied Mathematics NDHU 36
Prediction
1700 1750 1800 1850 1900 1950-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
數值方法 2008, Applied Mathematics NDHU 37
Sunspot
Source codes
數值方法 2008, Applied Mathematics NDHU 38
Paired data: tau=5
x=[];len=5;for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len);end
數值方法 2008, Applied Mathematics NDHU 39
Learning and prediction
M=input('keyin the number of hidden units:');[a,b,r]=learn_MLP(x',y',M);y=eval_MLP2(x,r,a,b,M);hold on;plot(1712+2:1712+2+length(y)-1,y,'r')
???
數值方法 2008, Applied Mathematics NDHU 40
Example
Repeat numerical experiments in slide #34 with
Summarize your numerical results in a table
20,15,10,5, 2 M
20,15,10,5, 5 M
20,15,10,5, 10 M
數值方法 2008, Applied Mathematics NDHU 41
RBF
y
exp exp exp exp
x
2
1
2
1
2μx
2
2
2 i
i
μx
2
1
2
1
2
M
M
μx
2
2
2 M
M
μx .. ..
0w1w iw 1Mw
Mw
數值方法 2008, Applied Mathematics NDHU 42
addpath('..\..\..\Methods\FunAppr\LM_RBF');% x : Nxd% y : 1xNN=200;d=2;x=rand(N,d)*2-1;y=tanh(-x(:,1).^2-x(:,2).^2);eval(['load data\sunspot_train.mat']); %([1 2 4 6 7 10],:)tx = X'; ty = Y;Net=LM_RBF(tx,ty);[yhat,D]=evaRBF(tx,Net);mean((ty-yhat).^2)/2
數值方法 2008, Applied Mathematics NDHU 43
eval(['load data\sunspot_test_1.mat']); tex1 = X'; tey1 = Y; [yhat,D]=evaRBF(tex1,Net); mean((tey1-yhat).^2)/2
One_step_look_ahead prediction