1.#産総研人間情報 2.京都大学学際融合教育研究推進センター...we measured...

Vergence Eye Movements Measured by An Advanced Real-Time Binocular Eye Tracking System新しいリアルタイム両眼眼球運動計測システムによる輻輳開散運動の計測

Keiji Matsuda1,Aya Takemura1,Kenji Kawano2

松田圭司1、竹村文1、河野憲二2

Human Informatics Research Institute, AIST, Tsukuba, Japan, 2. Center for the Promotion of Interdisciplinary Education and Research (C-PIER), Kyoto University1.  産総研人間情報 2.京都大学学際融合教育研究推進センター

Summary We measured human vergence eye movements using a newly developed binocular eye tracking system. The system adopts a wide-field, hi-resolution (2048x2048 pixels), and hi-frame-rate, USB-3.0 digital camera that provides high sensitivity, resolution, and frame rate. Infrared light illuminates both eyes and the reflected image of the illumination on the cornea and the black image of the pupil of each eye are captured by the camera. The center of the pupil is calculated by fitting an ellipse and tracked over time. The reflected image of the illumination on the cornea is used to compensate head-movements. The positions of both eyes are estimated at each frame and can be read out on line via computer network and DAC (Digital Analog Converter). The adoption of the WINDOWS 7/8/8.1 x 64 as the operation system makes this binocular eye tracking system user-friendly. Because of the high frame rate of the digital camera, the sampling rate of the system can be as high as 334Hz. 　To assess the quality of this system, we developed a new "real 3-D visual display system" that presents two wide-field visual images placed at different distances but at the same visual angle from the subject. The visual images are alternatively presented on two liquid crystal displays (LCDs), either "near LCD" or "far LCD". A half mirror is suspended at 45deg in the light path of the "far LCD" so as to allow the substitution of images on the "near LCD". The sizes of the near and far LCD screens are selected so that the angles of the view are the same. 　By using the binocular eye tracking system and the real 3-D visual display system, we characterized vergence eye movements of humans when ocular fixation shifted between two visual stimuli of the same view angles placed at different distances in 3-D space.

Supported by KAKENHI (24650105).

theore*cal  depth[mm]  mean  error[mm]  maximum  error[mm]  maximum  err  [mm]  1718.7  52.2  131.3  

75.1  1145.7  29.9  60.5  39.2  859.1  12.9  28.9  1.6  687.1  1.7  15.1  6.3  572.4  8.0  16.5  11.5  490.5  1.4  6.9  3.2  429.0  1.3  5.7  3.1  342.9  0.4  2.8  1.3  285.4  2.0  4.2  2.8  213.5  0.2  1.4  0.5  170.1  0.3  1.0  0.5  

a,  b,  c:  the  three  angle  of  the  triangle.      A,  B,  C:  the  three  sides  of  the  triangle.  

Results  

Conclusion By using this system, we succeeded in characterizing vergence eye movements of humans when ocular fixation shifts between two targets placed at different distances in 3-D space.

Camera

IR illumination

Stimulus display 1 (far)

USB3.0

Windows  7,  8,  8.1  x64

Output DA  Converter  TCP/IP  File

Stimulus display 2 (near)

half mirror

virtual screen

　 model Company Option/Memo

CameraGrasshpeopr3 GS3-U3-41C6NIR-C

Point Grey Research, Inc.

USB 3.0 Cable

LensMP7528-MP CBC 　

Infrared filter

R-72 M30.5 X 0.5 Edmund optics

Infrared illumination

TLN233 x 8 Toshiba

PC Windows7, 8, 8.1 x64 　 　

eye

Methods      We  calculated  the  horizontal/ver*cal  gaze  angle  of  each  eye  by  processing  its  video-­‐image.    The  origin  was  set  on  the  center  of  the  eyes.    The  horizontal  gaze  angle  of  the  leW  eye  (HL)  and  that  of  the  right  eye  (HR)  were  calculated  by  our  original  soWware.    “HL”  is  posi*ve  and  “HR”  is  nega*ve  in  the  figure.    C  corresponds  to  the  inter-­‐ocular  distance  (~60mm),  which  can  be  adjusted  for  each  subject.    According  to  the  law-­‐of-­‐sines,  the  target  posi*on  of  the  gaze  in  the  X-­‐Z  plane  can  be  described  as  follows.        Then  we  calculated  the  ver*cal  posi*on.    The  ver*cal  gaze  angle  of  the  leW  eye  (VL)  and  that  of  the  right  eye  (VR)  were  calculated  by  our  original  soWware.    We  used  average  of  them.  

depth  Z

（0,-­‐C/2)

HL HR

（C/2,0)

a b

c

A B

+

Horizontal  X

-­‐ + -­‐

（0,0)

C

gaze  posi*on

z

ver*cal  Y

（0,0)

B

depth  Z -­‐ +

gaze  posi*on

V L+ V R

2

y

A

sin a=

B

sin b=

C

sin c

a =⇡

2�HL

b =⇡

2+HR

c = ⇡ � a� b

(x, z) = (B cos a�C

2

, B sin a)

= (Ccos a sin b

sin c�

C

2

, Csin a sin b

sin c)

y = z tan(V L+ V R

2)

The  subject  gazed  near  s*mulus  reflected  by  the  half  mirror.

eye

monitor 2 (near)

half mirror

virtual screen

monitor 1 (far)

half mirror

The  subject  gazed  far  s*mulus  through  the  half  mirror.

Temporal  average  of  gaze  depth  points  aligned  with  ver*cal  saccadic  eye  movements.

Temporal  average  of  ver*cal  eye  movements  aligned  with  ver*cal  saccadic  eye  movements.

Temporal  average  of  vergence  (horizontal  right  eye  movements  –  leW  eye  movements)  aligned  with  ver*cal  saccadic  eye  movements.

Temporal  average  of  vergence  velocity  aligned  with  ver*cal  saccadic  eye  movements  applied  20  points  (60ms)  moving  average  filter.

−5

−4

−3

−2

−1

0

1

2

3

4

5

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

ertic

al e

ye m

ovem

ents

[deg

]

far to nearnear to far

Large stimulus

−5

−4

−3

−2

−1

0

1

2

3

4

5

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

ertic

al e

ye m

ovem

ents

[deg

]

far to nearnear to far

Small stimulus

−5

−4

−3

−2

−1

0

1

2

3

4

5

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

ertic

al e

ye m

ovem

ents

[deg

]

far to nearnear to far

Same stimulus

300

400

500

600

700

800

900

1000

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of g

aze

poin

ts d

epth

[mm

]

far to nearnear to far

Large stimulus

300

400

500

600

700

800

900

1000

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of g

aze

poin

ts d

epth

[mm

]

far to nearnear to far

Small stimulus

300

400

500

600

700

800

900

1000

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of g

aze

poin

ts d

epth

[mm

]

far to nearnear to far

Same stimulus

5

6

7

8

9

10

11

12

13

14

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

erge

nce

[deg

]

far to nearnear to far

Large stimulus

5

6

7

8

9

10

11

12

13

14

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

erge

nce

[deg

]

far to nearnear to far

Small stimulus

5

6

7

8

9

10

11

12

13

14

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

erge

nce

[deg

]

far to nearnear to far

Same stimulus

−100

−50

0

50

100

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

erge

nce

velo

city

app

lied

mov

ing

aver

age

[deg

/s]

far to nearnear to far

Large stimulus

−100

−50

0

50

100

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

erge

nce

velo

city

app

lied

mov

ing

aver

age

[deg

/s]

far to nearnear to far

Small stimulus

−100

−50

0

50

100

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

erge

nce

velo

city

app

lied

mov

ing

aver

age

[deg

/s]

far to nearnear to far

Same stimulus

−10

0

10

20

30

40

50

60

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

erge

nce

velo

city

app

lied

mov

ing

aver

age

[deg

/s]

largesamesmall

Far to near (convergence)

−50

−40

−30

−20

−10

0

10

20

−0.5 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4 0.5Time [sec]

Tem

pora

l ave

rage

of v

erge

nce

velo

city

app

lied

mov

ing

aver

age

[deg

/s]

largesamesmall

Near to far (divergence)

Experiments          We  measured  gaze  posi*ons  in  3-­‐D  space  and  the  sampling  frequency  was  333Hz.  The  subject  moved  his  gaze  between  the  far  and  near  visual  s*mulus  about  20  *mes.      The  far  s*mulus  was  displayed  on  the  monitor-­‐1  and  seen  through  the  half-­‐mirror.    The  near  s*mulus  was  displayed  on  the  monitor-­‐2  and  its  image  reflected  by  the  half  mirror  was  seen  on  the  virtual  screen.    1)  Large  visual  s*mulus  2)  Small  visual  s*mulus  3)  Same  visual  s*mulus

The  posi*ons  of  the  the  visual  s*muli.

far

near

2sec

20  *mes 2sec

The  *me  course  of  the  visual  s*muli.

X  axis

Z  axis

720mm

410mm

Near  vergence  angle  9.2deg  Far  vergence  angle  5.2deg

3deg

2deg

Y  axis

Z  axis

15deg

192mm 109mm

1deg

7mm 12mm

173mm

24deg 13deg

3deg

2deg

3deg

2deg

3deg

2deg

Large  visual  s*mulus  

Small  visual  s*mulus  

Same  visual  s*mulus  

eye

System outline