Fully Online Multicommand Brain-Computer

Interface with Visual Neurofeedback Using SSVEP

Paradigm

이명춘

lmspring@sclab.yonsei.ac.kr

P. Martinez, H. Bakardjian, and A. Cichocki, Computational Intelligence and Neuroscience,

Vol. 2007, Article ID 94561, 9 pages, 2007.

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Outline

• Motivation

• BCI system based on SSVEP paradigm

– Stimulator design

– Analysis system overview

– Artifact rejection by blind source separation

– Bank of band-pass filters and features extractions

– ANFIS classifier

• Operating modes

– Training mode

– Evaluation mode

– Free race (unsupervised) mode

• Experiment results

• Conclusion

• Hybrid BCI

1

S FT COMPUTING @ YONSEI UNIV . KOREA 16

BCI & BMI

• BCI & BMI

– BCI (Brain-computer interface) & BMI (Brain-machine interface)

– A system that control the computer or machine from brain signals

• Different methods to extract the user’s intentions

– Primary motor cortex • Results from imaginary limbs and tongue movements

• Based on temporal/spatial changes

• Based on the spectral characteristics of the SMR (Sensorimotor rhythm) oscillations

– SSVEP (Steady-state visual evoked potentials) • Elicited by multiple flashing light sources

– ERP (Event-related potential) • Follow an event noticed by the user

• Ex) p300: peak waveforms after a flash of a character the user focused attention

2

Motivation

S FT COMPUTING @ YONSEI UNIV . KOREA 16

SSVEP (Stead State Visual Evoked Potential)

• Definition

– Exogenous responses of the brain under visual stimulations at specific

frequencies

– Noticeable in the occipital (visual) cortex

• Advantages

– High performance with minimal training time and low requirements

– Robust in respect to noise and artifacts

– Easy to extend to more commands

• Disadvantages

– Fatigue or tiredness from the flickering visual stimuli

– Depend on muscular control of gaze direction for their operation

• Proposition

– Blind source separation (BSS) algorithm for artifact rejection

– Tuned microbatch filtering to estimate the features 3

Motivation

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Stimulator design

• A visual stimulation unit design

– An array of four small checkerboard images

– Flickering with different frequencies • Low-frequency range

(UP: 5Hz, LEFT: 6Hz, DOWN: 7Hz, RIGHT: 8Hz)

• Medium-frequency range

(UP: 12Hz, LEFT: 13.3Hz, DOWN: 15Hz, RIGHT: 17Hz)

– Moving along with the controlled object

• Meaningful electronencephalogram extraction

– Attention or gaze on a specific flickering image

– Weak quasi-periodic component is elicited mainly over the occipital cortex

– Signal attenuation in a large noise

4

BCI system based on SSVEP paradigm

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Analysis system overview

• Conceptual scheme

– BSS (Blind source separation): module for automatic rejection of artifacts

and noise

– Bank of filters: To enhance the first harmonics of the SSVEP responses

– Feature extraction • S-G (Sawitzky-Golay) smoothing and energy normalization

– ANFIS (adaptive network fuzzy inference system)

• Implementation

– Labview, C/C++, Matlab 5

BCI system based on SSVEP paradigm

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Analysis system overview (cont.)

• Platform overview

• Data acquisition

– Biosemi, Neuroscan commerical EEG devices

– Six EEG channels sampled at 256 Hz

– Five: CPZ, PZ, POZ, P1, P2

– One: FZ

6

BCI system based on SSVEP paradigm

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Artifact rejection by blind source separation

• Blind source separation

– The separation of a set of signals from a set of mixed signals

– Assumption that the source signals do not correlate with each other

• Several methods of blind signal separation

– PCA (Principal components analysis)

– SVD (Singular value decomposition)

– ICA (Independent component analysis)

• Purpose

– To enhance the signal

– To attenuate artifacts

• High pass filtering

– A cut off frequency of 2 Hz

7

BCI system based on SSVEP paradigm

S FT COMPUTING @ YONSEI UNIV . KOREA 16

AMUSE algorithm

• Definition and characteristics

– Algorithm for multiple unknown signal extraction

– Fast and reliable

– Estimation of the independent components

• Method

– Two consecutive PCA • First PCA: applied to the input data

• Second PCA (SVD): applied to the time-delayed

covariance matrix of the output from the previous stage

– Enhancement of EEG via AMUSE • Decomposed and ranked as independent or spatially

decorrelated components

• Projection of useful components to the scalp level

while artifacts and noise are removed from the signals

8

BCI system based on SSVEP paradigm

S FT COMPUTING @ YONSEI UNIV . KOREA 16

AMUSE Algorithm (cont.)

• Online preprocessing module

– Left: raw EEG data

– Center • Automatically ranked independent components

• The first and the last components were rejected as artifacts

– Right • Back-projected EEG signals

• Input for the bank of band-pass filters

9

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Bank of band-pass filters and features extractions

• IIR (Infinite impulse response)

– Bandwidth 0.5 Hz

– Center frequencies corresponding to the flickering frequencies of the

checkerboard

• SSVEP responses

– Estimation of the power of the output signals

• Smoothing filter

– Moving average (MA filter)

– Savitzky-Golay filter (S-G filter) • A filter that performs a local polynomial regression on a series of values

• Preservation of fundamental features such as relative maxima, minima, width of the peaks

10

BCI system based on SSVEP paradigm

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Bank of band-pass filters and features extractions

• Standard normalization of the smoothed energy

– M: The number of the electrodes

– N: The number of band-pass filters

– eij: Estimated energy of electrode i and band-pass filter j

– Ej: The relative energy per band, Input parameters for the ANFIS

• Normalized multiband signals

11

BCI system based on SSVEP paradigm

S FT COMPUTING @ YONSEI UNIV . KOREA 16

ANFIS classifier

• Hybrid learning procedure

– ANFIS (Adaptive network based fuzzy inference system)

– Based on fuzzy inference system

– Membership function parameter: Tuned using a back propagation

• Two membership functions

– A constant membership function

– Bell membership function

12

BCI system based on SSVEP paradigm

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Three separate modes

• Training mode

– To find the optimal parameters for each specific subject

– SSVEP responses (six seconds) • Attention on each checkerboard

• Voice-message requests

– non-SSVEP responses (six seconds) • Non-stimulus

• Removing all checkerboard patterns

• Evaluation (testing) mode

– To evaluate the BCI performance (time responses, percentage of success)

– Generation of random order requests for movement

– At intervals of Nine seconds in 32 trials (eight trials per direction)

– Success if the car moves properly between 1~6 seconds after voice request

• Free racing (unsupervised) mode

– Complete at least on lap

– Without any external voice commands

– Lap time: 90 ~ 150 seconds 13

Operating modes

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Results

• Subject-specific results

• Mean results

14

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Conclusion and discussions

• Three novel points

– The integrated moving checkerboard patterns • To maximize selective attention

• To minimize eye movements

– Online BSS module • To reduce automatically artifacts and noise

– Improved feature selection algorithm • Smoothing, filtering, ANFIS

• The performance

– High accuracy both medium-frequency range flicker and low-frequency

range

– FFT-based method: Require the usage of the higher harmonics

15

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Hybrid BCI

• Definition

– Composed of two BCIs, or at least on BCI and another system

• Four criteria

– Direct: The system must rely on activity recorded directly from the brain

– Intentional control: At least one recordable brain signal, which can be

intentionally modulated, must provide input to the BCI

– Real time processing: The signal processing must occur online and yield a

communication or control signal

– Feedback: The user must obtain feedback about the success or failure of

his/her efforts to communicate or control

16

S FT COMPUTING @ YONSEI UNIV . KOREA 16

Hybrid BCI

• Examples

• Two cases

– Process their inputs simultaneously

– Operate two systems sequentially, where the first system can act as a

“brain switch” • Brain switch: a BCI system designed to detect only one brain state in the ongoing brain

activity

• Reference

– Gert Pfurtscheller et al., “The hybrid BCI,” Frontiers in Neuroscience,

vol.4, no. 3, DOI=10.3389/fnpro.2010.00003, 2010. 17