prediction of exchange rate using deep neural network
TRANSCRIPT
Prediction of Exchange Rate Using Deep Neural Network
名古屋大学情報科学研究科
武田研究室
林知樹
1
Agenda
1. Background
2. Outline of Deep Learning
3. Proposed method
The structure of proposed model
features
4. Experiments
5. Conclusion
2
Background
Earning without working is a dream for humans.
3
What is the shortest way
to achieve this?
The answer is FX
Background
FX is money exchange game.
The shortest way to achieve our dream.
How to win?
This is very simple.
4
All you have to do is
only predicting up or down.
When the value is high, sell.
When the value is low, buy.
How to predict
My Hypothesis is
Prediction Using Deep Neural Network : DNN
State-of-the-art machine learning method
5
Future exchange rate
consists of past information.
Deep Neural Network (DNN)
6
Input
layerMiddle
layers
Output
layer
Layered Neural Network has a lot of middle layers
Deep Neural Network :
In general,
#middle layer > 3
The difference is only #middle layer.
Deep Neural Network (DNN)
The structure of DNN doesn’t look new
We can’t train DNN with conventional method.
Initial parameters : randomization
→ Fall into bad local solution
Appropriate initialization method appeared.
Pre-training by RBM or Auto-Encoder
7
We can prevent the
disappearance of gradient.
but
Disappearance of gradient problem
EX. Image recognition Before appearance of DNN
Appearance of DNN
8
Raw data Vector expressionFeature
ExtractionDiscriminator
Recognition
result
Raw dataFeature
Extraction Recognition
Deep Learning
Human-made
Training~
Learning comprehensively
from feature extraction to discriminative system
EX. Image recognition Before appearance of DNN
Appearance of DNN
9
Raw data Vector expressionFeature
ExtractionDiscriminator
Recognition
result
Raw dataFeature
Extraction Recognition
Deep Learning
Human-made
Training~
Learning comprehensively
from feature extraction to discriminative system
EX. Image recognition Before appearance of DNN
Appearance of DNN
10
Raw data Vector expressionFeature
ExtractionDiscriminator
Recognition
result
Raw dataFeature
Extraction Recognition
Deep Learning
Human-made
Training~
Learning comprehensively
from feature extraction to discriminative system
Achieved highest score
using only raw data.
Proposed method
2 kind of approach
1. Direct prediction of the exchange rate
Like Regression
Next time value is used as supervised data.
2. Binary option
2 Class Classification problem
• In next time, the value become high → Class 1
• In next time, the value become low → Class 0 I used these value as supervised data.
11
Regression by DNN
12
Middle layer
Output layer
xxh )(
)( bhy Wz
y
z
Regression→no range
Output identity mapping
OutputReal value
Identity function
W
2 Class Classification by DNN
13
]1,0[
)exp(1
1)(
xxh
z
2 class → 0 or 1
Output is prob.→ [ 0, 1 ]
Sigmoid function
yWMiddle layer
Output layer
)( bhy WzOutput
Input Features We used 10 kind of features as inputs.
Raw value
Exchange value
Top price
Low price
Closing value
Moving Average (9 points)
Relative Strength Index (RSI)
Stochastics RSI
Slow stochastics
Fast stochastics
Williams %R
14
Total 10 dim.
DNN input
15
𝐷 dim. feature
・・・
𝐷 dim.
N frame
DNN can deal with high dimension features and many frames.
tim
e
Total (𝑁 + 1) × 𝐷 dim.
Concatenated feature
Flowchart
16
Calcu
latin
g fe
atu
res
Raw
valu
es
10 d
im. fe
atu
re
Featu
re co
nca
ten
atio
n
Train
ing
DN
N u
sing
featu
res
Training
phase
Testing
phase
100 d
im. fe
atu
re
Train
ed
DN
N
Inp
ut co
nca
ten
ate
d fe
atu
re
for te
sting
Pre
dicte
d va
lue
Flowchart
17
Calcu
latin
g fe
atu
res
Raw
valu
es
10 d
im. fe
atu
re
Featu
re co
nca
ten
atio
n
Train
ing
DN
N u
sing
featu
res
Training
phase
Testing
phase
100 d
im. fe
atu
re
Train
ed
DN
N
Inp
ut co
nca
ten
ate
d fe
atu
re
for te
sting
Pre
dicte
d va
lue
Exchange value
Top price
Low price
Closing value
Flowchart
18
Calc
ula
ting
featu
res
Raw
valu
es
10
dim
. featu
re
Featu
re co
nca
ten
atio
n
Train
ing
DN
N u
sing
featu
res
Training
phase
Testing
phase
100 d
im. fe
atu
re
Train
ed
DN
N
Inp
ut co
nca
ten
ate
d fe
atu
re
for te
sting
Pre
dicte
d va
lue
Raw values
Moving Average
RSI
Stochastics RSI
Williams %R
Flowchart
19
Calcu
latin
g fe
atu
res
Raw
valu
es
10 d
im. fe
atu
re
Featu
re c
on
cate
natio
n
Train
ing
DN
N u
sing
featu
res
Training
phase
Testing
phase
10
0 d
im. fe
atu
re
Train
ed
DN
N
Inp
ut co
nca
ten
ate
d fe
atu
re
for te
sting
Pre
dicte
d va
lue
tim
e
Flowchart
20
Calcu
latin
g fe
atu
res
Raw
valu
es
10 d
im. fe
atu
re
Featu
re co
nca
ten
atio
n
Tra
inin
g D
NN
usin
g fe
atu
res
Testing
phase
100 d
im. fe
atu
re
Tra
ined
DN
N
Inp
ut co
nca
ten
ate
d fe
atu
re
for te
sting
Pre
dicte
d va
lue
1. Pre-training
RBM
2. Fine-tuning
back propagation
Training
phase
Flowchart
21
Calcu
latin
g fe
atu
res
Raw
valu
es
10 d
im. fe
atu
re
Featu
re co
nca
ten
atio
n
Train
ing
DN
N u
sing
featu
res
Training
phase
100 d
im. fe
atu
re
Train
ed
DN
N
Inp
ut c
on
cate
nate
d fe
atu
re
for te
sting
Pre
dic
ted
valu
ePredicted value
100 dim. feature
Test data
Testing
phase
An Exchange rate data
22
including data from 1991 to 2014.
Time interval is 1 hour.
Date TimeExchange
rate
Top
price
Low
price
Closing
value
Total
transaction
Experiment
Experimental conditions
DNN training conditions
23
Dataset $-¥ Exchange rate
# data1991/01/04 ~ 2015/01/5
Total 97362 points
# DNN layer 5 layers
# middle layer node 256 nodes
Pre-training Fine-tuning
Learning rate 0.002 0.00006
Momentum 0.9 0
Batch size 128 128
Epoch 30 50
Dividing dataset
24
Training data ①
Training data ②
Training data ③
Test data ①
Test data ②
Test data ③
Each test data has 24 points(24 hours). In this time, I made from ① to ㉛.
Direct prediction
Input :
Features calculated by presence and past signal
Output :
the next time closing value
26
Direct prediction result
27
Direct prediction result
28
Closed test
Close up
Direct prediction result
29
Closed test
Prediction could capture characteristics of answer line.
Direct prediction result
30
Open test
Close up
Direct prediction result
31
Open test
Predicted signal fluctuates.
There is no information about that
in the next time the value will become up or down.
Direct prediction result
32
Open test
Predicted signal fluctuates.
There is no information about that
in the next time the value will become up or down.
Binary option
Input :
Features calculated by presence and past signal
Output :
In the next time, up(Class 1) or down(Class 0)
33
Binary option result
Closed test
Open test
34
)96366/51516([%]46.53Acc.
)744/375([%]40.50Acc.
Using dice is better than this method.
Using dice is better than this method.
Binary option result
Closed test
Open test
35
)96366/51516([%]46.53Acc.
)500/252([%]40.50Acc.
Why we couldn’t predict? Small fluctuation prevents us from predicting.
36
Why we couldn’t predict? Small fluctuation prevents us from predicting.
37
Similar to white noise
Why we couldn’t predict? Small fluctuation prevents us from predicting.
38
Similar to white noise
Another approach
Prediction of trend transition
Trend transition means
The value will become up or down for Moving average in the past 𝑁 hours.
We can ignore the effect of small fluctuation.
39
Prediction of trend transition
40
Input :
Features calculated by presence and past signal
Output :
In the next time, trend will become up(Class 1) or down(Class 0)
Prediction of trend transition
Closed test
Open test
41
)97338/81516([%]75.83Acc.
)744/678([%]63.87Acc.
42
Prediction of trend transition
Predicted value is [0,1].
The closer to 1 or 0 predicted value is,
the more reliable the prediction is.
We can set the threshold
to make the prediction more reliable.
Open test (Setting Threshold as 0.8 and 0.2)
)515/487([%]61.94Acc.
43
Prediction of trend transition
Predicted value is [0,1].
The closer to 1 or 0 predicted value is,
the more reliable the prediction is.
We can set the threshold
to make the prediction more reliable.
Open test (Setting Threshold as 0.8 and 0.2)
)515/487([%]61.94Acc.
Conclusion and future works
Conclusion
We try to predict exchange rate using DNN.
3 kind of approach
Direct prediction
Binary option
Trend transition
We could predict trend transition
with 83%(Closed) and 87%(Open).
Future problem
Considering another kind of feature
Prediction of more long term change44
Failed…
Failed…
Succeeded!!
Thank you for your attention!
45
Pre training
RBMによる貪欲学習
1層目と2層目をRBMとみなして学習
2層目の出力をサンプリング
2層目と3層目をRBMとみなして学習
以下繰り返し
(1)1段目のRBM学習 (2)2段目のRBM学習 (3)3段目のRBM学習