fully online multicommand brain-computer interface with visual...

of 18/18
Fully Online Multicommand Brain-Computer Interface with Visual Neurofeedback Using SSVEP Paradigm 이명춘 [email protected] P. Martinez, H. Bakardjian, and A. Cichocki, Computational Intelligence and Neuroscience, Vol. 2007, Article ID 94561, 9 pages, 2007.

Post on 27-May-2020

2 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Fully Online Multicommand Brain-Computer

    Interface with Visual Neurofeedback Using SSVEP

    Paradigm

    이명춘

    [email protected]

    P. Martinez, H. Bakardjian, and A. Cichocki, Computational Intelligence and Neuroscience,

    Vol. 2007, Article ID 94561, 9 pages, 2007.

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Outline

    • Motivation

    • BCI system based on SSVEP paradigm

    – Stimulator design

    – Analysis system overview

    – Artifact rejection by blind source separation

    – Bank of band-pass filters and features extractions

    – ANFIS classifier

    • Operating modes

    – Training mode

    – Evaluation mode

    – Free race (unsupervised) mode

    • Experiment results

    • Conclusion

    • Hybrid BCI

    1

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    BCI & BMI

    • BCI & BMI

    – BCI (Brain-computer interface) & BMI (Brain-machine interface)

    – A system that control the computer or machine from brain signals

    • Different methods to extract the user’s intentions

    – Primary motor cortex • Results from imaginary limbs and tongue movements

    • Based on temporal/spatial changes

    • Based on the spectral characteristics of the SMR (Sensorimotor rhythm) oscillations

    – SSVEP (Steady-state visual evoked potentials) • Elicited by multiple flashing light sources

    – ERP (Event-related potential) • Follow an event noticed by the user

    • Ex) p300: peak waveforms after a flash of a character the user focused attention

    2

    Motivation

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    SSVEP (Stead State Visual Evoked Potential)

    • Definition

    – Exogenous responses of the brain under visual stimulations at specific

    frequencies

    – Noticeable in the occipital (visual) cortex

    • Advantages

    – High performance with minimal training time and low requirements

    – Robust in respect to noise and artifacts

    – Easy to extend to more commands

    • Disadvantages

    – Fatigue or tiredness from the flickering visual stimuli

    – Depend on muscular control of gaze direction for their operation

    • Proposition

    – Blind source separation (BSS) algorithm for artifact rejection

    – Tuned microbatch filtering to estimate the features 3

    Motivation

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Stimulator design

    • A visual stimulation unit design

    – An array of four small checkerboard images

    – Flickering with different frequencies • Low-frequency range

    (UP: 5Hz, LEFT: 6Hz, DOWN: 7Hz, RIGHT: 8Hz)

    • Medium-frequency range

    (UP: 12Hz, LEFT: 13.3Hz, DOWN: 15Hz, RIGHT: 17Hz)

    – Moving along with the controlled object

    • Meaningful electronencephalogram extraction

    – Attention or gaze on a specific flickering image

    – Weak quasi-periodic component is elicited mainly over the occipital cortex

    – Signal attenuation in a large noise

    4

    BCI system based on SSVEP paradigm

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Analysis system overview

    • Conceptual scheme

    – BSS (Blind source separation): module for automatic rejection of artifacts

    and noise

    – Bank of filters: To enhance the first harmonics of the SSVEP responses

    – Feature extraction • S-G (Sawitzky-Golay) smoothing and energy normalization

    – ANFIS (adaptive network fuzzy inference system)

    • Implementation

    – Labview, C/C++, Matlab 5

    BCI system based on SSVEP paradigm

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Analysis system overview (cont.)

    • Platform overview

    • Data acquisition

    – Biosemi, Neuroscan commerical EEG devices

    – Six EEG channels sampled at 256 Hz

    – Five: CPZ, PZ, POZ, P1, P2

    – One: FZ

    6

    BCI system based on SSVEP paradigm

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Artifact rejection by blind source separation

    • Blind source separation

    – The separation of a set of signals from a set of mixed signals

    – Assumption that the source signals do not correlate with each other

    • Several methods of blind signal separation

    – PCA (Principal components analysis)

    – SVD (Singular value decomposition)

    – ICA (Independent component analysis)

    • Purpose

    – To enhance the signal

    – To attenuate artifacts

    • High pass filtering

    – A cut off frequency of 2 Hz

    7

    BCI system based on SSVEP paradigm

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    AMUSE algorithm

    • Definition and characteristics

    – Algorithm for multiple unknown signal extraction

    – Fast and reliable

    – Estimation of the independent components

    • Method

    – Two consecutive PCA • First PCA: applied to the input data

    • Second PCA (SVD): applied to the time-delayed

    covariance matrix of the output from the previous stage

    – Enhancement of EEG via AMUSE • Decomposed and ranked as independent or spatially

    decorrelated components

    • Projection of useful components to the scalp level

    while artifacts and noise are removed from the signals

    8

    BCI system based on SSVEP paradigm

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    AMUSE Algorithm (cont.)

    • Online preprocessing module

    – Left: raw EEG data

    – Center • Automatically ranked independent components

    • The first and the last components were rejected as artifacts

    – Right • Back-projected EEG signals

    • Input for the bank of band-pass filters

    9

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Bank of band-pass filters and features extractions

    • IIR (Infinite impulse response)

    – Bandwidth 0.5 Hz

    – Center frequencies corresponding to the flickering frequencies of the

    checkerboard

    • SSVEP responses

    – Estimation of the power of the output signals

    • Smoothing filter

    – Moving average (MA filter)

    – Savitzky-Golay filter (S-G filter) • A filter that performs a local polynomial regression on a series of values

    • Preservation of fundamental features such as relative maxima, minima, width of the peaks

    10

    BCI system based on SSVEP paradigm

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Bank of band-pass filters and features extractions

    • Standard normalization of the smoothed energy

    – M: The number of the electrodes

    – N: The number of band-pass filters

    – eij: Estimated energy of electrode i and band-pass filter j

    – Ej: The relative energy per band, Input parameters for the ANFIS

    • Normalized multiband signals

    11

    BCI system based on SSVEP paradigm

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    ANFIS classifier

    • Hybrid learning procedure

    – ANFIS (Adaptive network based fuzzy inference system)

    – Based on fuzzy inference system

    – Membership function parameter: Tuned using a back propagation

    • Two membership functions

    – A constant membership function

    – Bell membership function

    12

    BCI system based on SSVEP paradigm

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Three separate modes

    • Training mode

    – To find the optimal parameters for each specific subject

    – SSVEP responses (six seconds) • Attention on each checkerboard

    • Voice-message requests

    – non-SSVEP responses (six seconds) • Non-stimulus

    • Removing all checkerboard patterns

    • Evaluation (testing) mode

    – To evaluate the BCI performance (time responses, percentage of success)

    – Generation of random order requests for movement

    – At intervals of Nine seconds in 32 trials (eight trials per direction)

    – Success if the car moves properly between 1~6 seconds after voice request

    • Free racing (unsupervised) mode

    – Complete at least on lap

    – Without any external voice commands

    – Lap time: 90 ~ 150 seconds 13

    Operating modes

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Results

    • Subject-specific results

    • Mean results

    14

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Conclusion and discussions

    • Three novel points

    – The integrated moving checkerboard patterns • To maximize selective attention

    • To minimize eye movements

    – Online BSS module • To reduce automatically artifacts and noise

    – Improved feature selection algorithm • Smoothing, filtering, ANFIS

    • The performance

    – High accuracy both medium-frequency range flicker and low-frequency

    range

    – FFT-based method: Require the usage of the higher harmonics

    15

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Hybrid BCI

    • Definition

    – Composed of two BCIs, or at least on BCI and another system

    • Four criteria

    – Direct: The system must rely on activity recorded directly from the brain

    – Intentional control: At least one recordable brain signal, which can be

    intentionally modulated, must provide input to the BCI

    – Real time processing: The signal processing must occur online and yield a

    communication or control signal

    – Feedback: The user must obtain feedback about the success or failure of

    his/her efforts to communicate or control

    16

  • S FT COMPUTING @ YONSEI UNIV . KOREA 16

    Hybrid BCI

    • Examples

    • Two cases

    – Process their inputs simultaneously

    – Operate two systems sequentially, where the first system can act as a

    “brain switch” • Brain switch: a BCI system designed to detect only one brain state in the ongoing brain

    activity

    • Reference

    – Gert Pfurtscheller et al., “The hybrid BCI,” Frontiers in Neuroscience,

    vol.4, no. 3, DOI=10.3389/fnpro.2010.00003, 2010. 17