hvad er remote usability testing af jan stage, aau
Post on 06-Jul-2015
193 Views
Preview:
DESCRIPTION
TRANSCRIPT
Hvad er Remote Usability Testing?
Jan Stage
Professor, PhD
Forskningsleder i Informationssystemer (IS)/Human-Computer Interaction (HCI)
Aalborg Universitet, Institut for Datalogi, HCI-Lab
jans@cs.aau.dk
Institut for Datalogi 2
• Forhindringer i praksis
• Ideen om remote usability testing
• Realisering af ideen
Oversigt
Institut for Datalogi 3
• Forhindringer i praksis
• Ideen om remote usability testing
• Realisering af ideen
Oversigt
Institut for Datalogi 4
To undersøgelser af forhindringer i praksis
Primært Nordjylland
1. Software-virksom-heder, typisk svar fra mellemledere
2. Deltagere i en-dages konference om usability
• Tallene angiver procent af svarene i hver undersøgelse
2325
15
86
13
0
10
43
118
5 5
1
26
00
5
10
15
20
25
30
35
40
45
50
Resourcedemands
Developermindset
Customerparticipation
Testparticipants
Conductingtests
No problems Low priority No answ er
Første undersøgelse
Anden undersøgelse
Institut for Datalogi 5
• Forhindringer i praksis
• Ideen om remote usability testing
• Definition og baggrund
• Tidligere arbejde
• Synkrone og asynkrone metoder
• Realisering af ideen
Oversigt
Institut for Datalogi 6
Definition and Motivation
• Remote usability testing is a situation where“the evaluators are separated in space and/or time from users”(Castillo, Hartson and Hix, 1998)
• Well-known challenges• Building and setting up usability labs• Getting users to the lab and making them feel comfortable
• New challenges• Users, developers and evaluators are distributed geographically, sometimes
even in different countries and time zones• Outsourcing• Open Source Software development• Global software development
• Remote usability evaluation is becoming increasingly relevant
Institut for Datalogi 7
Earlier Work (1)
• The idea of remote evaluation emerged between 1994 and 1998
• Literature survey on remote usability testing• 21 references• Early: presents a variety of methods• Recently: practical advice and experiences combined with pros/cons
Institut for Datalogi
Earlier Work (2)
8
Institut for Datalogi 9
Earlier Work (3)
• Two types of methods: synchronous / asynchronous (various forms)
• The majority of references focus on synchronous usability evaluation:Simulating a conventional laboratory-based think-aloud test
• Overall:• Few systematic comparisons of remote methods• Mostly comparisons of a single remote method and the conventional lab-based
method
Institut for Datalogi 10
Synchronous Methods
Laboratory testing (LAB)• Conventional method• Benchmark for comparison
Remote synchronous testing (RS)• Spatial separation: User and test monitor in
separate rooms• Remote sharing of desktop• Webcam connection with picture in the
bottom corner of the screen• Audio connection
Data: Video recording of the test monitor’s screen and the audio
Institut for Datalogi
Asynchronous Methods
Remote asynchronous user testing (AU)• Based on the idea of users reporting the critical incidents they experience
(Castillo, Hartson and Hix, 1998; Hartson and Castillo, 1998; Hartson, Castillo, Kelso and Neale, 1996)
• The users solve the tasks with the system• Simultaneously fill in a web-based form each time they encounter a usability problem• The form briefly explains what a usability problem is and how it can be categorized• In the form, they describe each problem and categorize it’s severity• Spatial and temporal separation
Remote asynchronous expert testing (AE)• Same method as AU but with users that have expertise in usability engineering (graduate
students in HCI)
Data: A set of described usability problems and a start and end time for each user
11
Institut for Datalogi 12
Survey of Remote Asynchronous Methods
Method Paper #
Auto logging 20, 28, 29, 30, 34, 35, 36, 37
Interview 10, 26, 29, 30, 35
Questionnaires 10, 19, 25, 28, 29, 30, 32, 34, 35, 36, 37
User-reported critical incident 1, 5, 6, 7, 13, 14, 31
Unstructured problem reporting 10, 19, 38
Forum 21
Diary 31
Institut for Datalogi 13
• Forhindringer i praksis
• Ideen om remote usability testing
• Realisering af ideen
Oversigt
Institut for Datalogi
• Forhindringer og udfordringer• Tekniske: Kan man få det til at fungere• Metodiske: giver det brugbare resultater• Praktiske: kan man gøre det i praksis
• Det skal vi arbejde med resten af dagen …
14
Realisering af ideen
Institut for Datalogi
aa
15
top related