3 차원 안구모델의 적응적 제어

서울대학교 뇌과학협동과정2002-20618

Introduction

Robotic eye 연구의 의의와 현황

- 최근 휴머노이드 및 로봇의 연구가 활성화되고 있슴

- 로봇에 장착될 인공안구의 필요성 역시 대두됨 .

- 자연스러운 안구의 움직임은 의사 및 감정의 소통에 매우 중요함 .

- 양질의 시각정보 제공을 위해선 빠르고 정확한 안구운동이 필요 .

- 소형화 및 경량화 요구 .

Eye robots

KIST 의 HECtor

MIT 의 Kismet

ATR 연구소의 Infanoid ATR 연구소의 DB

MIT 의 Cog

Social role of eye-movement

Communication using eye movement

: pointing, mimic, social gesture

Human-like Impression

x

z

y

3 DOF manipulator

Human eye

Linear motor 2dof eye

Spherical motor

Agile eye : new type 3D gear

3D eye models

Hard ware scheme

CCD camera

Image capture board

computer

D/O board

D/I board

LM629

LM629

LM629

LMD18200

LMD18200

LMD18200

Motor 1

Motor 2

Motor 3

Encoder input

motor input

Image input

Problems

Calibration : initial point fixation

Motor control

Velocity modification

XZ

Y

k

j

i

Mechanics of 3D eye

6

1

2

2

3

16

1

2

2

3

13

20

3

1

k

j

i

X: 시선방향 Y: 좌우방향축 Z: 상하방향축

k

j

i

v

w

u

o

o

o

1

1

1

i

k

j

v

w

u

o

o

o

2

2

2

j

i

k

v

w

u

o

o

o

3

3

3

skew symmetric matrixweR

w

0

0

0

12

13

23

ww

ww

ww

w

iou

i wew iio

회전축 w 를 중심으로 θ 만큼 회전

Mechanics of 3D eye 2

Motor property

Motor 1,2,3 position relationship

Motor equation :

Motor rotation to x-axes movement

Motor space

Test boardTarget board

Detect salient target

|(R-G)| + |(R-B)|

Center fixation

Psychological inspired neural net

Stress release, emotion

Memory : LTM, STM and Emotion

Hebb’s Rule

Stress

Stress : be arisen from sensory stimulus & pain.

Can be released to actuator

Informational energy

Drive force of action

Motivate movement

Movement uses the Stress as Energy

Emotion

Happy : Maintain this situation

Unhappy(Pain) : Change this situation

Happy : reducing Stress

Pain : Increasing Stress

Emotion makes Memory :

Strengthen the links of activate cells in pool

Brain metaphor

Sensory Cortex

WorldNeo-cortex

Thalamus

AmygdalaHippocampus

Motor cortex

Data Interaction

Emotion evaluator

Data compression

Data derivation

Making stress & Data

transfer

Make Memory

Psychological inspired neural net

Motor Cortex

Sensory Cortex

Stress generator

Emotion generator

memory

Neo cortex

Input cells

Output cells

Animal analogy

foodRange of smell

LTM & STM model

LTM

STMtime

W

Sensory cortex

Memory

Sensory cortex

Activate!

Memory

Sensory cortex

Activate!

Activation decay

Memory

Sensory cortex

Activation inhibited

Memory

Sensory cortex

Memory

Emotion!

All Links strengthened!

Memory

STM duration

Memory

Motor Cortex

Bnew = W * Bold

W new = W old(activation > threshold) * T(1.2)

M = W * (Bold + Snew)

B = S + I + MS = sensor cellI = inter cellM = motor cellW = weight matrix

Step by step movement

target

Problem space

Learning! New target Random search

Known situation Association & learning

Learned actionAssociative memory

Learning Process

Trial-and-error learning

Mnew = Mold * Vstep * R(error)

M = motor value

Vstep = modifying size(5 ~ 15%)

R = probability function(20~80%)

Accept = 0.5 + D(distance)

Result

47% = Error_before – Error_after

Error_before * 100

Result

Result

Velocity of Motor 1

Velocity of Motor 2

Velocity of Motor 3

Before learning After learning

Rotation value of 3 motors

Differences between before and after of motor 1,2,3

Weight change of Neural net

before after

Discussion

Initial point fixation

Effect of gravity

Saccadic suppression

Circular CCD

Psychological inspired neural net vs traditional neural net

Emotion evokes memory

STM

Stress

Auto weight decay

Time serial associative memory

Run & Learn

Mixed layers

Application

Navigation robot’s learning rule

Motion correcting of robot

Interactive controller

Conclusion

For spherical parallel 3D eye model :

1. Initial point fixation using visual input

2. Modify acute motor value by trial-error learning

3. Step-by-step movement by psychological inspired neural networks

Error reduced about 53% less than before learning.

Eye movement after learning

Old version

Circuit for eye

Motion detect

Motion = previous image – present image

Row image Black & white image Motion detect

Motion detect 2

Motion detect 3

Life game

1st noise remove 2nd, 3rd noise remove

Visual trajectory

30 degree

5 degree

Real visual trajectory

Untrained trajectory 1

Untrained trajectory 2

Introduction

robotic eyes

biology/psychological apply

human visual pathway

frued’s mind theory

agile eye

Experiment

Purpose of research

System ( H/W )

Problems

System ( S/W )

Result

calibration

motion-detect

saccade / pursuit

motion

visual cortex / visual filed

2-step saccade

Discussion

Human vision

Human visual pathway

Untrained trajectory 3

Eye Movement

1. Saccade

2. Pursuit

3. Vergency

4. Vestibular optokinesis

Evolution of eye- movement

Saccade – pursuit

Monotone – color

Motion – shape

Eye of frog, cat, human