prediction of exchange rate using deep neural network

Prediction of Exchange Rate Using Deep Neural Network

名古屋大学情報科学研究科

武田研究室

林知樹

1

Agenda

1. Background

2. Outline of Deep Learning

3. Proposed method

The structure of proposed model

features

4. Experiments

5. Conclusion

2

Background

Earning without working is a dream for humans.

3

What is the shortest way

to achieve this?

The answer is FX

Background

FX is money exchange game.

The shortest way to achieve our dream.

How to win?

This is very simple.

4

All you have to do is

only predicting up or down.

When the value is high, sell.

When the value is low, buy.

How to predict

My Hypothesis is

Prediction Using Deep Neural Network : DNN

State-of-the-art machine learning method

5

Future exchange rate

consists of past information.

Deep Neural Network (DNN)

6

Input

layerMiddle

layers

Output

layer

Layered Neural Network has a lot of middle layers

Deep Neural Network ：

In general,

#middle layer > 3

The difference is only #middle layer.

Deep Neural Network (DNN)

The structure of DNN doesn’t look new

We can’t train DNN with conventional method.

Initial parameters : randomization

→ Fall into bad local solution

Appropriate initialization method appeared.

Pre-training by RBM or Auto-Encoder

7

We can prevent the

disappearance of gradient.

but

Disappearance of gradient problem

EX. Image recognition Before appearance of DNN

Appearance of DNN

8

Raw data Vector expressionFeature

ExtractionDiscriminator

Recognition

result

Raw dataFeature

Extraction Recognition

Deep Learning

Human-made

Training~

Learning comprehensively

from feature extraction to discriminative system

EX. Image recognition Before appearance of DNN

Appearance of DNN

9

Raw data Vector expressionFeature

ExtractionDiscriminator

Recognition

result

Raw dataFeature

Extraction Recognition

Deep Learning

Human-made

Training~

Learning comprehensively

from feature extraction to discriminative system

EX. Image recognition Before appearance of DNN

Appearance of DNN

10

Raw data Vector expressionFeature

ExtractionDiscriminator

Recognition

result

Raw dataFeature

Extraction Recognition

Deep Learning

Human-made

Training~

Learning comprehensively

from feature extraction to discriminative system

Achieved highest score

using only raw data.

Proposed method

2 kind of approach

1. Direct prediction of the exchange rate

Like Regression

Next time value is used as supervised data.

2. Binary option

2 Class Classification problem

• In next time, the value become high → Class 1

• In next time, the value become low → Class 0 I used these value as supervised data.

11

Regression by DNN

12

Middle layer

Output layer

xxh )(

)( bhy Wz

y

z

Regression→no range

Output identity mapping

OutputReal value

Identity function

W

2 Class Classification by DNN

13

]1,0[

)exp(1

1)(

xxh

z

2 class → 0 or 1

Output is prob.→ [ 0, 1 ]

Sigmoid function

yWMiddle layer

Output layer

)( bhy WzOutput

Input Features We used 10 kind of features as inputs.

Raw value

Exchange value

Top price

Low price

Closing value

Moving Average (9 points)

Relative Strength Index (RSI)

Stochastics RSI

Slow stochastics

Fast stochastics

Williams %R

14

Total 10 dim.

DNN input

15

𝐷 dim. feature

・・・

𝐷 dim.

N frame

DNN can deal with high dimension features and many frames.

tim

e

Total (𝑁 + 1) × 𝐷 dim.

Concatenated feature

Flowchart

16

Calcu

latin

g fe

atu

res

Raw

valu

es

10 d

im. fe

atu

re

Featu

re co

nca

ten

atio

n

Train

ing

DN

N u

sing

featu

res

Training

phase

Testing

phase

100 d

im. fe

atu

re

Train

ed

DN

N

Inp

ut co

nca

ten

ate

d fe

atu

re

for te

sting

Pre

dicte

d va

lue

Flowchart

17

Calcu

latin

g fe

atu

res

Raw

valu

es

10 d

im. fe

atu

re

Featu

re co

nca

ten

atio

n

Train

ing

DN

N u

sing

featu

res

Training

phase

Testing

phase

100 d

im. fe

atu

re

Train

ed

DN

N

Inp

ut co

nca

ten

ate

d fe

atu

re

for te

sting

Pre

dicte

d va

lue

Exchange value

Top price

Low price

Closing value

Flowchart

18

Calc

ula

ting

featu

res

Raw

valu

es

10

dim

. featu

re

Featu

re co

nca

ten

atio

n

Train

ing

DN

N u

sing

featu

res

Training

phase

Testing

phase

100 d

im. fe

atu

re

Train

ed

DN

N

Inp

ut co

nca

ten

ate

d fe

atu

re

for te

sting

Pre

dicte

d va

lue

Raw values

Moving Average

RSI

Stochastics RSI

Williams %R

Flowchart

19

Calcu

latin

g fe

atu

res

Raw

valu

es

10 d

im. fe

atu

re

Featu

re c

on

cate

natio

n

Train

ing

DN

N u

sing

featu

res

Training

phase

Testing

phase

10

0 d

im. fe

atu

re

Train

ed

DN

N

Inp

ut co

nca

ten

ate

d fe

atu

re

for te

sting

Pre

dicte

d va

lue

tim

e

Flowchart

20

Calcu

latin

g fe

atu

res

Raw

valu

es

10 d

im. fe

atu

re

Featu

re co

nca

ten

atio

n

Tra

inin

g D

NN

usin

g fe

atu

res

Testing

phase

100 d

im. fe

atu

re

Tra

ined

DN

N

Inp

ut co

nca

ten

ate

d fe

atu

re

for te

sting

Pre

dicte

d va

lue

1. Pre-training

RBM

2. Fine-tuning

back propagation

Training

phase

Flowchart

21

Calcu

latin

g fe

atu

res

Raw

valu

es

10 d

im. fe

atu

re

Featu

re co

nca

ten

atio

n

Train

ing

DN

N u

sing

featu

res

Training

phase

100 d

im. fe

atu

re

Train

ed

DN

N

Inp

ut c

on

cate

nate

d fe

atu

re

for te

sting

Pre

dic

ted

valu

ePredicted value

100 dim. feature

Test data

Testing

phase

An Exchange rate data

22

including data from 1991 to 2014.

Time interval is 1 hour.

Date TimeExchange

rate

Top

price

Low

price

Closing

value

Total

transaction

Experiment

Experimental conditions

DNN training conditions

23

Dataset $-¥ Exchange rate

# data1991/01/04 ~ 2015/01/5

Total 97362 points

# DNN layer 5 layers

# middle layer node 256 nodes

Pre-training Fine-tuning

Learning rate 0.002 0.00006

Momentum 0.9 0

Batch size 128 128

Epoch 30 50

Dividing dataset

24

Training data ①

Training data ②

Training data ③

Test data ①

Test data ②

Test data ③

Each test data has 24 points(24 hours). In this time, I made from ① to ㉛.

Direct prediction

Input :

Features calculated by presence and past signal

Output :

the next time closing value

26

Direct prediction result

27

Direct prediction result

28

Closed test

Close up

Direct prediction result

29

Closed test

Prediction could capture characteristics of answer line.

Direct prediction result

30

Open test

Close up

Direct prediction result

31

Open test

Predicted signal fluctuates.

There is no information about that

in the next time the value will become up or down.

Direct prediction result

32

Open test

Predicted signal fluctuates.

There is no information about that

in the next time the value will become up or down.

Binary option

Input :

Features calculated by presence and past signal

Output :

In the next time, up(Class 1) or down(Class 0)

33

Binary option result

Closed test

Open test

34

)96366/51516([%]46.53Acc.

)744/375([%]40.50Acc.

Using dice is better than this method.

Using dice is better than this method.

Binary option result

Closed test

Open test

35

)96366/51516([%]46.53Acc.

)500/252([%]40.50Acc.

Why we couldn’t predict? Small fluctuation prevents us from predicting.

36

Why we couldn’t predict? Small fluctuation prevents us from predicting.

37

Similar to white noise

Why we couldn’t predict? Small fluctuation prevents us from predicting.

38

Similar to white noise

Another approach

Prediction of trend transition

Trend transition means

The value will become up or down for Moving average in the past 𝑁 hours.

We can ignore the effect of small fluctuation.

39

Prediction of trend transition

40

Input :

Features calculated by presence and past signal

Output :

In the next time, trend will become up(Class 1) or down(Class 0)

Prediction of trend transition

Closed test

Open test

41

)97338/81516([%]75.83Acc.

)744/678([%]63.87Acc.

42

Prediction of trend transition

Predicted value is [0,1].

The closer to 1 or 0 predicted value is,

the more reliable the prediction is.

We can set the threshold

to make the prediction more reliable.

Open test (Setting Threshold as 0.8 and 0.2)

)515/487([%]61.94Acc.

43

Prediction of trend transition

Predicted value is [0,1].

The closer to 1 or 0 predicted value is,

the more reliable the prediction is.

We can set the threshold

to make the prediction more reliable.

Open test (Setting Threshold as 0.8 and 0.2)

)515/487([%]61.94Acc.

Conclusion and future works

Conclusion

We try to predict exchange rate using DNN.

3 kind of approach

Direct prediction

Binary option

Trend transition

We could predict trend transition

with 83%(Closed) and 87%(Open).

Future problem

Considering another kind of feature

Prediction of more long term change44

Failed…

Failed…

Succeeded!!

Thank you for your attention!

45

Pre training

RBMによる貪欲学習

1層目と2層目をRBMとみなして学習

2層目の出力をサンプリング

2層目と3層目をRBMとみなして学習

以下繰り返し

(1)1段目のRBM学習 (2)2段目のRBM学習 (3)3段目のRBM学習