UX Group: Eye-tracking with quantity

Mária Bieliková

20 eye-trackers in one room (UXI Labs @ Slovak University of Technology)

20x Tobii X2-60 eye

trackers

Screen recording

Face recording

+

Infrastructure for

transferring all the

data to the central

server

2

3D depth

camera60Hz

eyetracker

Mass data collection is excellent for

summative studies

• Same conditions for

every participant

• Saving staff time

3

Group studies need

infrastructure software

Our UX Group Research Software

• Data collection from all UX nodes

• Lifecycle management of user studies projects

• Processing of gathered data

• Annotations

• Cleaning

• Export

• Archivation

5

UXI@FIIT = three laboratories

6

Tobii

TX300

PC,

EEG,

...

Tobii

X2-60

mobiles,

TV, PC20x Tobii X2-60

gaze tracking

PC

Control

room

8

Setup

Data

Streams

9

Start

Session

Annotate recorded data

10

Export recorded data

11

Enjoy your data!

12

Enjoy your data!

13

Enjoy your data!

14

Enjoy your data!

15

UX Group Lab –

first experiences

Twin-experiment(two experiments taken with the same group of 50 participants, ~2,5GB of

raw csv data)

Experiment #1:

Crowdsourcing study demonstrating the use

of eye-tracking for human computation scenario.Participants had to categorize of documentary movies based on their

descriptions.

Experiment #2:

Questionnaire deception detection using eye-trackingParticipants had to fill out the big five personality trait questionnaire.

17

Experiment #1:

Crowdsourcing movie categorizations

Participant’s task:

1. View the description of a documentary movie

2. Pick a primary category for the movie from the list

3. [Optionally] Pick a secondary category

Hypothesis:

We can discover additional classification information,

if we eye-track the workers during the task

18

19

Experiment #1:

Task user interface with example gaze plot.

The gaze reveals, what other options

the workers considered

20

“Saving rhino phila"

[["animals", 100], ["crime", 50]]

[["traveling", 1150.0], ["geography", 1017.0],

["biography", 500.0], ["health", 400.0], ["animals", 367.0],

Title:

Picked categories:

Viewed categories:

Experiment #1:

Observations

Experiment #2:

Questionnaire deception detection

Participant’s task:

Fill out the big five personality trait questionnaire that matches

1. your personality (respond honestly)

2. an ideal worker for job position (faking good instruction)

Hypothesis:

We can detect deception by analysis of gaze data

Van Hooft, E.A.J. & Born, M.P. (2012). Intentional response distortion on personality

tests: Using eye-tracking to understand response processes when faking. Journal of

Applied Psychology, 97(2), 301–316. doi:10.1037/a0025711

21

22

Experiment #2:

Big five personality trait question example

We were looking for fixation order, response

times, pupil dilation, first fixation…

23

Deception cases were characteristic by:

• First fixations at extreme response options

p = 3.538e-14 < 0.001

• Longer reaction times

p = 0.039 < 0.05

• Pupil dilation variance before and after answering

a question

p = 0.022 < 0.05

Experiment #2:

Observations

Methodological

lessons learnt

Have some assistants

There were 3 of us: one experiment conductor and two

assistants. It was enough for 20 people, but barely.

25

Do a pilot study with multiple participants,

don’t just do it with individuals

If the study has to be parallel recording, do the pilot likewise

Pilot allowed us to properly update instructions• No browser window resizing, no zooming

• Make the scenario as simple as possible. The more steps there are, the more mistakes the participants make.

Instructions are crucial

• Don’t just use paper instructions, use slides and presentation as well

26

Group calibration is best done two-step

First, do a presentation (slide) demonstration.

• Do not allow participants to do anything during the

explanation.

• Make them watch you, or they will screw up.

Second, let everyone follow the remembered

instruction from the first step.

• Watch the calibration status on the dashboard

• Attend everyone that raises any problems

• 3 experiment conductors were enough to handle

calibration problems

27

Never let participants to do anything

during the time you speak to them.

Stress to them the necessity of discipline.

Paper instructions won’t always save you.

Participants do not read them properly.

Better communicate important things also

verbally.

28

More group studies

in preparation

Visual search ability for target findability

evaluation

30

Target:

31

32

33

Join us at FB

http://www.facebook.com/groups/UXIsk/

34