david verstraeten
DESCRIPTION
RC tutorial. David Verstraeten. Reservoir Computing. Random and fixed. Trained. Steps in training and testing. Create random weight matrices Input to reservoir Reservoir to reservoir (Output to reservoir) Scale weight matrices globally to the desired value - PowerPoint PPT PresentationTRANSCRIPT
RCT tutorial 1
RC tutorialDavid Verstraeten
RCT tutorial 2
Reservoir Computing
Random and fixedTrained
RCT tutorial 3
Steps in training and testing
• Create random weight matrices – Input to reservoir– Reservoir to reservoir– (Output to reservoir)
• Scale weight matrices globally to the desired value• Feed all input samples to the reservoir• Collect reservoir states• Train readout weight using linear regression or
ridge regression
RCT tutorial 4
Scaling reservoir matrix:Spectral radius
€
x[k +1] = Ax[k]+ Bu[k +1]
y[k +1] =Cx[k +1]+Du[k +1]
€
x[k +1] = f Wresresx[k]+Wres
inu[k +1]( )
y[k +1] =Woutoutx[k +1]+Wout
in u[k +1]
LTI system
RC system
€
ρ =max eig Wresres
( )( )Spectral radius
RCT tutorial 5
Training
M N P
Inputs Reservoir Outputs
T1
T2
TS
1
2
S
…
1
2
S
…
1
2
S
…
A B
RCT tutorial 6
Linear/ridge regression
A BWX =
€
wopt = (ATA)−1ATB
€
wopt = (ATA + λ I)−1ATB
Linear regression
Ridge regression€
wopt = argminwAw −b
2
€
wopt = argminwAw −b
2+ λ w
2
RCT tutorial 7
Pattern generation tasks: training
Fixed connections
Trained connections
RCT tutorial 8
Pattern generation: generation/testing
WarmupFreerunFixed connections
Trained connections
RCT tutorial 9
Leaky integrator neurons
RCT tutorial 10
Global optimization parameters
• Input scaling• Reservoir scaling (spectral radius)• Leak rate• Feedback scaling • Regularization parameter (through cross-
validation)• Not:
– connection fraction (only for spiking neurons)– reservoir size
RCT tutorial 11
RC toolbox
RCT tutorial 12
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 13
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 14
Rationale
• Reservoir Computing toolbox:• Box of tools for
– Quick-and-dirty experiments– Large scale parameter sweeps and optimizations
• Modular setup: function hooks for customization– Reservoir types– Learning rules– Scoring and evaluation methods
RCT tutorial 15
Getting started
• Download from http://snn.elis.ugent.be/rct
• Unpack, go to directory $RCT_ROOT and type:
>> install• For ‘Hello world’ experiment, type
>> rc_simulate
RCT tutorial 16
Usage
• Default settings– Are ok as starting point for many experiments– Contained in $RCT_ROOT/default_settings/default_*.m
– Look at these scripts! They give an overview of RCToolbox functionality
RCT tutorial 17
Custom settings• Two ways of overriding defaults:
– Command line>> dataset_generation_function=… @my_dataset_function();>> rc_simulate;
– Custom configuration file
>> custom_configuration=‘my_conf’>> rc_simulate;
– Example: analog_speech.m• Other examples: $RCTROOT/examples/configurations
RCT tutorial 18
Flow diagram for basic experiment
RCT tutorial 19
Structure of rc_simulate
RCT tutorial 20
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 21
Standard topology for classification/regression
InputInput BiasBias
ReservoirReservoir
ReadoutReadout
Mandatory
Optional
RCT tutorial 22
Connectivity matrix between layers
From/to Input Bias Reservoir Readout
Input 0 0 1 0/1
Bias 0 0 0/1 0/1
Reservoir 0 0 1 1
Readout 0 0 0 0
InputInput BiasBias
ReservoirReservoir
ReadoutReadout
RCT tutorial 23
Standard topology for classification/regression
Mandatory
Optional BiasBias
ReservoirReservoir
ReadoutReadout
RCT tutorial 24
Connectivity matrix between layers
From/to Bias Reservoir Readout
Bias 0 0/1 0/1
Reservoir 0 1 1
Readout 0 1 0/1
BiasBias
ReservoirReservoir
ReadoutReadout
RCT tutorial 25
Topology datastructure in the toolbox
• topology.– layer(1,n)– conn(n,n)– with n = number of layers
• Default settings: – $RCTROOT/default_settings/default_topology.m– Layer 1 : input– Layer 2 : bias– Layer 3 : reservoir– Layer 4 : readout
RCT tutorial 26
• topology.layer(1:n).– size: integer– is_trained_offline: boolean– is_trained_online: boolean– node_type: string– training_function:function pointer– nonlin_functions: array of function pointers
Topology datastructure in the toolbox
RCT tutorial 27
• topology.layer(1:n).– init_simulation: function pointer– finish_simulation: function pointer
– dt: float– regul_param: float– scoring: function pointer
Topology datastructure in the toolbox
RCT tutorial 28
• topology.conn(1:n,1:n).– is_active: boolean– creation_pipeline: array of function pointers
– scaling_factor: float– conn_frac: float [0,1]– discrete_set: float array
Topology datastructure in the toolbox
RCT tutorial 29
Topology creation pipeline
• Array of function pointers which pass weight matrices
• Generation functions– @gen_rand, @gen_1D, @gen_load
• Assignment functions– @assign_rand, @assign_discrete
• Scaling functions– @scale_specrad, @scale_constant
RCT tutorial 30
Topology creation pipeline
• Example (default for reservoir to reservoir connection):– topology.conn(3,3).creation_pipeline= {@gen_rand,
@assign_randn, @scale_specrad}
• Can be customized! Function hooks
RCT tutorial 31
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 32
Dataset generation
• Function hook• Default
– dataset_generation_function=@dataset_narma_10(input_dt, input_maxtime, n_samples)
• Signature– function [inputs outputs] = my_dataset_function(…)– inputs = cell(1,nr_samples)
• inputs{:} = matrix (M,:);• M = input dimensionality
– outputs = cell(1,nr_samples)• outputs{:} = matrix (P,:) or [] for signal generation tasks.• P = output dimensionality
• Example: 10th order NARMA
RCT tutorial 33
Data struct
• data.layer(1:n).– r : required/teacher forced states– s : simulated states
RCT tutorial 34
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 35
Training and testing
• cross-validate – Training and testing using cross-validation
• cross_validate_grid- Training and testing using cross-validation with
optimization of regularization parameter
RCT tutorial 36
Cross-validate• Example: three-fold cross-validation with
three samples.
11
22
33
TrainingTraining
TestingTesting
ResultsResults11
22
33
error fold 1
error fold 1
Fold 1
RCT tutorial 37
Cross-validate• Example: three-fold cross-validation with
three samples.
11
22
33
TrainingTraining
TestingTesting
ResultsResults11
22
33
error fold 1
error fold 1
Fold 2
error fold 2
error fold 2
RCT tutorial 38
Cross-validate• Example: three-fold cross-validation with
three samples.
11
22
33
TrainingTraining
TestingTesting
ResultsResults
11
22
33
error fold 1
error fold 1
Fold 3
error fold 2
error fold 2
error fold 3
error fold 3
RCT tutorial 39
Cross-validation
• Function hook:– train_params.cross_val_set_function– Default: @random_cross_val– Other possibilities:
• cross_val_only_training: no test set• no_cross_val: simple train and test set• random_freerun_cross_val : cross-validation for freerun tasks
• Freerun cross-validation
11
22
TrainingTraining
TestingTesting
11
22
33 33
3
3 3 Fold 1
…
RCT tutorial 40
Cross-validate with gridsearch• For optimization of regularization parameter
TrainingTraining
TestingTesting
ResultsResults
1133
error fold 1
error fold 1
Fold 1
ValidationValidation
22
11 22 33
Val. results
Val. results
error error
error error
error nerror n
…
Subfold 1
22
11
Subfold 2
n opt error fold 2
error fold 2
error fold 3
error fold 3
Fold 2Fold 3
RCT tutorial 41
Outline
• Rationale, usage and structure
• Topology
• Datasets
• Cross-validation
• Parameter sweeps and custom scripts
RCT tutorial 42
Parameter sweeps• parameter_ranges = struct( 'name', {}, 'range', {});
• Example– 1D sweep:parameter_ranges = struct(
'name', {‘topology.layer(3).size’},
'range’, {10:10:100});
– 2D sweep (all combinations!):parameter_ranges = struct(
'name', {‘topology.layer(3).size’, ‘topology.layer(1).scale_factor’},
'range', {10:10:100, .1:.1:1});
RCT tutorial 43
Parameter sweeps
• Results are saved in output_directory• Set save_results=true!
• Plot results of sweep – plot_error(output_directory)
• Get variables from simulation– get_saved_data(output_directory, variables)
RCT tutorial 44
Custom scripts
• Toolbox is: box of tools you can use to build your own experiments
• Good starting points:– $RCTROOT/examples/tutorial/tut1.m– $RCTROOT/examples/framework/rc_simulate_job.m
RCT tutorial 45
Questions