Forskning i forholdet mellem gevinst og omkostning ved usability-arbejde

Jan Stage

Professor, PhDForskningsleder i Informationssystemer (IS)/Human-Computer Interaction (HCI)Aalborg Universitet, Institut for Datalogi

jans@cs.aau.dk

Institut for Datalogi 22

1. Introduktion2. Forskningsresultater3. Kom selv i gang

Oversigt

Institut for Datalogi 33

1. Introduktion2. Forskningsresultater3. Kom selv i gang

Oversigt

Institut for Datalogi

Usability

Definition (ISO 1998):The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use

Antagelse:Dårlig usability koster – så hvis vi fjerner usability-problemer, bliver vores medarbejdere/kunder mere effektive og gladere

4

Institut for Datalogi

Dårlig usability koster

• Evaluering af IKEA’s hjemmeside”Det er vist lettere at køre derned” (til Århus)

• Evaluering af Budget Cars hjemmesideLej en bil på 60 sekunder10 personer: 4-17 min.; snit: 7,5 min.79 usability problemer (5 / 36 / 38)

• Dokumenteret, at det koster kunder

5

Institut for Datalogi 66

A simple login form with two fields (Email Address and Password), two buttons (Login and Register), and one link (Forgot Password) prevented customers from purchasing products from a major e-commerce site. Users would encounter it after they filled their shopping cart with products they wanted to purchase and pressed the Checkout button.A consultant conducted usability tests showing that first-time users resented having to. As one shopper said, "I'm not here to enter into a relationship. I just want to buy something.“The designers saw the form as enabling repeat customers to purchase faster, but they weren't any happier. Except for a very few who remembered their login information, most stumbled on the form. They couldn't remember the email address or password they used. Some would eventually ask the site to send the password to their email address, which is a problem if you can't remember which email address you initially registered with. The number of people who requested passwords was about 160,000 per day. 75% of them never tried to complete the purchase once requested.The designers fixed the problem by simply taking away the Register button. Instead, they put a Continue button with a simple message: "You do not need to create an account to make purchases on our site. Simply click Continue to proceed to checkout. To make your future purchases even faster, you can create an account during checkout.“The number of customers purchasing went up by 45%. The extra purchases resulted in an extra $15 million the first month. For the first year, the site saw additional purchases of $300,000,000. (Spool, 2009)

High score eksemplet

Institut for Datalogi 77

1. Introduktion2. Forskningsresultater

• Fire typer benefits• Empiriske målinger• Costs

3. Kom selv i gang

Oversigt

Institut for Datalogi

Benefits (1)

A. Software Development1. Reduced development costs

2. Reduced amount of irrelevant functionality

3. Savings gained from making changes earlier in design lifecycle

4. Reduced need for producing documentation

8

Institut for Datalogi

A. Software Development

The rule of thumb in many usability-aware organizationsis that the cost-benefit ratio for usability is $1:$10-$100.Once a system is in development, correcting a problemcosts 10 times as much as fixing the same problem in design. If the system has been released, it costs 100 times as much relative to fixing in design. (Gilb, 1988)

In one large commercial client, the time to execute a quality assurance test fully was reduced by 85% because of a decrease in features brought about through the application of formal usability engineering and testing during the early phases of development. This amounted to a savings of approximately $15 million and a reduction in schedule by 18 months over the prior software development iteration. (Mauro, 2002)

Usability techniques allowed a high-tech company to reduce the time spent on one tedious development task by 40% (Bias & Mayhew, 1994). At another company, usability techniques helped cut development time by 33-50%. (Bosert, 1991)

Sun Microsystems has shown how spending about $20,000 could yield a savings of $152 million dollars. Each and every dollar invested could return $7,500 in savings. (Rhodes, 2000)

9

Institut for Datalogi

Benefits (2)

B. User Organization1. Increased user productivity

2. Decreased user errors

3. Decreased training costs

4. Decreased user help and support costs

5. Decreased unofficial support (from colleagues)

6. Increased user satisfaction

10

Institut for Datalogi

Brugere: tidsforbrug og produktivitet

Et stort distribueret system:• 2000 sagsbehandlere• 20 sager pr. bruger pr. dag• 40.000 sager om dagen• Sagsbehandlerens tidsforbrug pr. sag: 5-20 minutter• Besparelse pr. sag på grund af forbedret usability: 30 sekunder• Timeløn for sagsbehandleren: 150 kr.• Besparelse pr. år:

= 2000 brugere * 20 sager/bruger/dag* 230 dage * 0,5 minutter * 1 time/60 minutter * 150 kr./time

= 11.500.000 kr. pr. årUanset hvad et grundigt usability-arbejde koster for dette system, så er gevinsten kolossalInspireret af Rex Hartson (http://courses.cs.vt.edu/~cs5714/fall2000/notes/wrap_up/wrap_up.htm)

11

Institut for Datalogi

B. User Organization

One study at NCR showed a 25% increase in throughput with an additional 25% decrease in errors resulting from redesign of screens to follow basic principles of good design. (Gallaway, 1981)

A major computer company spent $20,700 on usability work to improve the sign-on procedure in a system used by several thousand people. The resulting productivity improvement saved the company $41,700 the first day the system was used.On a system used by over 100,000 people, for a usability outlay of $68,000, the same company recognized a benefit of $6,800,000 within the first year of the system’simplementation. This is a cost-benefit ratio of $1:$100. (Bias & Mayhew, 1994)

When systems match user needs, satisfaction often improves dramatically. In a 1992 Gartner Group study, usability methods raised user satisfaction ratings for a system by 40%. (Bias & Mayhew, 1994)

Design changes due to usability work at IBM resulted in an average reduction of 9.6 minutes per task, with projected internal savings at IBM of $6.8 Million in 1991 alone. (Karat, 1990)

12

Institut for Datalogi

Benefits (3)

C. External Software Vendor1. Increased sales

2. Reduced customer support costs

3. Reduced cost of training provisions (if training is offered by the vending company)

4. Competitive advantage (usability branding)

13

Institut for Datalogi

C. External Software Vendor

Wixon & Jones did a case study of a usability-engineered software product that increased revenue by more than 80% over the first release of the product (built without usability work) (Wixon). The revenues of the usability-enhanced system were 60% higher than projected. Many customers cited usability as a key factor in buying the new system. (Bias & Mayhew, 1994)

Usability also plays a role in the public’s perception of a company, affecting brand value and market share. About 15% of the space in reviews published in trade magazines, journals, and national newspapers is devoted to user friendliness or usability. (Nielsen, 1993)

14

Institut for Datalogi

Benefits (4)

D. Web Site Use1. Enhanced retention rate

2. Reduced shopping cart abandonment

3. Competitive advantage (satisfied customers return)

4. Better information quality (interactive sites)

15

Institut for Datalogi

D. Web Site Use

You can increase sales on your site as much as 225% by providing sufficient product information to your customers at the right time. In our recent research, we found that the design of product lists directly affected sales. On sites that did not require shoppers to bounce back-and-forth between the list and individual product pages, visitors added more products to their shopping cart and had a more positive opinion of the site. (User Interface Engineering, 2001)

One study estimated that improving the customer experience increases the number of buyers by 40% and increase order size by 10%. (Creative Good, 2000)

The magnitude of usability improvements is usually large. This is not a matter of increasing use by a few percent. It is common for usability efforts to result in a hundred percent or more increase in traffic or sales. (Nielsen, July 1999)

In a 1999 study of Web users, respondents were asked to list the five most important reasons to shop on the Web. Pricing was only the third-most important. The top reason was “Easy to place an order” by 83% of the respondents. (Nielson, February 1999)

16

Institut for Datalogi

Costs

Tidsforbrug:• Planlægning og forberedelse• Udførelse• Fortolkning og dokumentation

Brugere:• Rekruttering af brugere• Betaling af brugere

Leje:• Leje af laboratorium og udstyr

Se litteratur: Kohno et al. 2013, Bevan 2005, Wong 2009

17

Institut for Datalogi 1818

1. Introduktion2. Forskningsresultater3. Kom selv i gang

Oversigt

Institut for Datalogi

Sælg ideen til ledelsen

Start i det små:• Forsøg med usability-arbejde på en lille del af et projekt• Prøv først at løse et afgrænset antal usability-problemer• Sæt et lille usability-laboratorium op (symbolværdi)

Relation til ledelsen:• Klargør mål, forventede gevinster og tidsramme• Få de nødvendige ressourcer afsat• Få klar ledelsesopbakning

Udviklingsteamet:• Skaf mindst én person med den relevante kompetence i interaktionsdesign og

usability, og giv ham/hende ansvar og autoritet• Giv teamet den nødvendige træning• Få medlemmerne af teamet til at forpligte sig

19

Institut for Datalogi

Litteratur (1)

Den klassiske bog på området (artikelsamling):Bias, R. G. and Mayhew, D. J. (Eds.) (1994) Cost-Justifying Usability. San Francisco: Morgan

Kaufmann Publishers.

Bias, R.G. & Mayhew, D.J. (2005) Cost-Justifying Usability: An Update for the Internet Age. San Francisco: Morgan Kaufmann Publishers.

Oversigter over costs-benefit cases:Bevan, N. (2005) Cost benefits evidence and case studies. Adapted from Bias and Mayhew (2005)

http://www.nigelbevan.com/papers/Cost_benefits_evidence.pdf

Marcus, A. (2004) Return on Investment for Usable User-Interface Design: Examples and Statistics.

http://www.amanda.com/joomla_uploads/whitepapers/AM+A_ROIWhitePaper_20Apr0%201.pdf

20

Institut for Datalogi

Litteratur (2)

Det mest markante eksempel på benefit:Spool, J. (2009) The $300 Million Button

http://www.uie.com/articles/three_hund_million_button/

Metode til måling af UCD cost-benefit, som har været anvendt til måling på 22 projekter i deres virksomhed:

Kohno, I., Yasu, H., Sugawara, S. and Nishikawa, M. (2013) Pragmatic Approach to Cost Benefit Analysis of User Centered Design. In: Design, User Experience, and Usability. Design Philosophy, Methods, and Tools. Lecture Notes in Computer Science Volume 8012, 2013, pp 525-534.

Konkret eksempel på en cost-benefit analyse med detaljerede tal:Wong, C. Y. (2009) Applying Cost-Benefit Analysis for Usability Evaluation Studies on an Online

Shopping Website.

http://www.hft.org/HFT03/paper03/09_Won.pdf

21

Institut for Datalogi

Vores forskningsaktiviteter

En væsentlig del af vores forskning handler om at finde løsninger, der reducerer eller helt fjerner vigtige forhindringer for usability-arbejde i praksis:

• Pris• Kompetence

Eksempler:• Barfods-evaluatorer• Instant Data Analysis (IDA)• Remote Synkron/Asynkron evaluering

Kontakt:• Jan Stage, Aalborg Universitet, Institut for Datalogi• jans@cs.aau.dk

22