hydra usability testing
TRANSCRIPT
HYDRA USABILITY TESTINGWest Virginia University Libraries
USABILITY TEST INTRODUCTION
Problem Overview:
In an effort to improve and streamline the digital collection user experience for West Virginia
University (WVU) Libraries’ patrons, with both responsive website and web application user
experiences, WVU Libraries implemented a tabbar-based interaction module for mobile
devices on new HYDRA repositories.
The effectiveness of this user interface change from the standpoint of both institutional
research, special collections, and its importance to the HYDRA UX Interest Group needs to
be tested and vetted.
USABILITY TEST STRATEGY
Testing Parameters:
WVU Libraries used TechSmith Morae to conduct usability testing of the Clarysville Civil War Hospital Digital Collection user interface on
both a DELL Windows 7 laptop - using Internet Explorer 11 at a browser width of 1366 pixels - and on various smartphone devices/browsers
as provided by individual users. These devices were used based on a report generated from 37,296 visits to websites that used
W3Counter's free web stats.
Based on the W3Counter report, since 93% of browsing occurs at a 1,366 or less pixel width - with 17% being the largest portion for a 1,366
by 768 pixel laptop resolution – both laptops and various smartphones were identified as being optimal testing devices.
USABILITY TEST TARGET AUDIENCETarget Audience:
Breakdown of target audiences is based on numbers populated by the WVU Office of the University Registrar, qualitative survey polls, and
quantitative server statistics. This round of usability testing was focused on WVU Libraries primary target audience, and internal to WVU
Libraries by utilizing undergraduate student employees.
Primary Target Audience(73% / 107,062 students per month) WVU undergraduate students who are 48% female or 52% male, are primarily Caucasian, African American, Hispanic, or Asian in ethnicity, are United States citizens or are international students, are 21 years in their average age, and are 49% from the state of West Virginia or 51% from other United States, territories, or countries.
Secondary Target Audience(19% / 27,865 students per month)WVU graduate students who 41% male or 59% female, are 83% US citizens, 14% nonimmigrants mostly from India or the People’s Republic of China, and are 31 years in their average age.
Tertiary Target Audience(8% / 11,733 students per month)WVU administration, faculty, staff, and Morgantown, West Virginia community members.
USABILITY TEST PARTICIPANTS
Alex Chris Colin Devyn
Erica Kayla Ryan Taija
USABILITY TEST FULL VIDEOS
https://testing.lib.wvu.edu/hydra/alex_m.wmv
https://testing.lib.wvu.edu/hydra/chris_d.wmvhttps://testing.lib.wvu.edu/hydra/chris_m.wmv
https://testing.lib.wvu.edu/hydra/colin_d.wmv
https://testing.lib.wvu.edu/hydra/devyn_d.wmv
https://testing.lib.wvu.edu/hydra/erica_d.wmv
https://testing.lib.wvu.edu/hydra/kayla_d.wmvhttps://testing.lib.wvu.edu/hydra/kayla_m.wmv
https://testing.lib.wvu.edu/hydra/ryan_m.wmv
https://testing.lib.wvu.edu/hydra/taija_m.wmv
USABILITY TEST TASKS1. Go to the Clarysville Civil War Hospital Digital Collection2. How many records are there for people from Ohio were treated for gunshot wounds? 143. What was the last name of the only record for a man with the first name Anthony? 4. How many records are there for Corporals with typhoid fever, from Pennsylvania? 35. What was the first name on the record of a man from Kentucky with the last name Smith? 6. On the record of Matthew Wilson, was he married?7. Who was the only Orderly treated at Clarysville Hospital?8. For the record of Michael Weaver, in what town did his relatives live?9. When was the Clarysville Hospital Established?10. What is the age of the sergeant from New York who was married?
OVERALL TASK:AVERAGE TIME( Desktop )
* Average time includes delivery of tasks.
OVERALL TASK:AVERAGE TIME( Mobile )
* Average time includes delivery of tasks.
OVERALL TASK:AVERAGE TIME( Comparison )
* Average time includes delivery of tasks.
0
0.5
1
1.5
2
2.5
Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8 Task 9 Task 10
Aver
age
Tim
e (M
inut
es)
Tasks
Desktop Mobile
OVERALL TASK:SUCCESS RATE( Desktop )
Failed to complete
Completed with difficulty
Completed
Completed with ease
OVERALL TASK:SUCCESS RATE( Mobile )
Failed to complete
Completed with difficulty
Completed
Completed with ease
OVERALL TASK: SUCCESS RATE(Comparison )
Dektop
Mobile
Failed to complete
Completed with difficulty
Completed
Completed with ease
Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8 Task 9 Task 10
1. I think that I would like to use this system frequently2. I found the system unnecessarily complex3. I thought that the system was easy to use4. I think that I would need the support of a technical person to be able to use this system5. I found the various functions in this system were well integrated6. I thought there was too much inconsistency in this system7. I would imagine that most people would learn to use this system very quickly8. I found the system very cumbersome to use9. I felt very confident using the system10. I needed to learn a lot of things before I could get going with this system
USABILITY TEST POST-ASSESSMENT
POST-ASSESSMENT RESULTS( Desktop )
I needed to learn a lot of things before I could get going with this system
I felt very confident using the system
I found the system very cumbersome to use
I would imagine that most people would learn to use this system very quickly
I thought there was too much inconsistency in this system
I found the various functions in this system were well integrated
I think that I would need the support of a technical person to be able to use this system
I thought that the system was easy to use
I found the system unnecessarily complex
I think that I would like to use this system frequently
60%
80%
100%
80%
80%
40%
20%
20%
20%
80%
40%
20%
40%
20%
20%
60%
80%
60%
80%
Strongly Disagree Disagree N/A Agree Strongly Agree
I needed to learn a lot of things before I could get going with this system
I felt very confident using the system
I found the system very cumbersome to use
I would imagine that most people would learn to use this system very quickly
I thought there was too much inconsistency in this system
I found the various functions in this system were well integrated
I think that I would need the support of a technical person to be able to use this system
I thought that the system was easy to use
I found the system unnecessarily complex
I think that I would like to use this system frequently
POST-ASSESSMENT RESULTS( Mobile )
80%
80%
60%
40%
60%
20%
20%
40%
60%
20%
40%
20%
40%
40%
20%
60%
60%
20%
20%
20%
60%
40%
40%
40%
Strongly Disagree Disagree N/A Agree Strongly Agree
POST-ASSESSMENT RESULTS( Combined )
0%10%20%30%40%50%60%70%80%90%
100%
N/A Agree Strongly Agree
I think that I would like to use this system frequently
Desktop Mobile
0%10%20%30%40%50%60%70%80%90%
100%
Strongly Disagree Disagree
I found the system unnecessarily complex
Desktop Mobile
POST-ASSESSMENT RESULTS( Combined )
0%10%20%30%40%50%60%70%80%90%
100%
Strongly Disagree Disagree Agree
I think that I would need the support of a technical person to be able to use this system
Desktop Mobile
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Agree Strongly Agree
I thought that the system was easy to use
Desktop Mobile
POST-ASSESSMENT RESULTS( Combined )
0%10%20%30%40%50%60%70%80%90%
100%
Agree Strongly Agree
I found the various functions in this system were well integrated
Desktop Mobile
0%10%20%30%40%50%60%70%80%90%
100%
Strongly Disagree Disagree N/A
I thought there was too much inconsistency in this system
Desktop Mobile
POST-ASSESSMENT RESULTS( Combined )
0%10%20%30%40%50%60%70%80%90%
100%
Agree Strongly Agree
I would imagine that most people would learn to use this system very quickly
Desktop Mobile
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
StronglyDisagree
Disagree N/A Agree
I found the system very cumbersome to use
Desktop Mobile
POST-ASSESSMENT RESULTS( Combined )
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
N/A Agree Strongly Agree
I felt very confident using the system
Desktop Mobile
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
StronglyDisagree
Disagree N/A Agree
I needed to learn a lot of things before I could get going with this system
Desktop Mobile
• 1 out of 10 tasks' average time for completion was longer on a mobile device versus desktop• 8 out of 10 tasks' average time for completion was equal to/shorter on a mobile device versus desktop• 5 out of 10 tasks' average time for completion was shorter on a mobile device versus desktop
ON A MOBILE DEVICE…
50% of tasks’ time for completion ranged 9 to 41 seconds shorter, averaging 20.8 seconds faster.
FINDINGS: COMPLETION TIMES
FINDINGS: SUCCESS RATES
• 2 out of 10 tasks' overall success rate were less on a mobile device versus desktop• 6 out of 10 tasks' overall success rate were equal to/improved on a mobile device versus desktop• 5 out of 10 tasks' overall success rate were improved on a mobile device versus desktop
ON A MOBILE DEVICE…
50% of tasks’ overall success rate improved by a range of 20% to 80%, increased ‘completed with ease’ success rates by 40%, and reduced ‘failed to complete’ rates by 20%.
• 1 out of 5 more of mobile device users strongly disagreed that they found the system complex.• 1 out of 5 more of mobile device users thought inconsistency in this system was not applicable.• 1 out of 5 more of mobile device users thought confidence in using the system was not applicable.• 1 out of 5 more of mobile device users thought needing to learn the system was not applicable.
ON A MOBILE DEVICE…
User assessment improved by 20% in regards to ease of complexity, learning, and consistency.
FINDINGS: USER ASSESSMENT
• 100% of users don't scroll past the media in the document view. Change all full record text to be above or beside media.
• 80% of desktop viewers found the collection through the library, whereas 40% used voice search on their smartphone. Eliminating intermediate pages and instead link to collection directly.
• 60% of users think Blacklight options are menu options. Make the Blacklight header tabs take up less space, and exist on one line in mobile browsers.
• 40% of users thought the WVRHC/Library navigation header was the collection menu. Reduce branding and headers to be minimal and take up less space.
• Add web application hints for searching and faceting on the collection’s first load, then disable hints with browser cookies for return visitors.
• Incorporate web application hints that explain facets can come from both accordions and search terms, and that users can further refine search after searching.
• Test/Fix JavaScript so menus, tabs, and search boxes collapse/expand on search, selection, and page load.
• Add the word 'Menu' next to the hamburger icon to increase menu visibility.
• Increase the size of the ‘X’s to remove facets on mobile devices. User are struggling with the small selection areas.
PROPOSED ENHANCEMENTS
WVU LIBRARIES AND HYDRA
RESPONSIVE DIGITAL REPOSITORIES
MULTI-LAYERED HYDRA HEADS
TWO-PART AUTOMATED INSTALLATION
Michael BondSenior Software [email protected]
Tim BroadwaterUX Designer / Front-End [email protected]
ADDITIONAL INFORMATION
Other Digital Repositories:
• https://civilwarwv.lib.wvu.edu
• https://storercollege.lib.wvu.edu
• https://clarysville.lib.wvu.edu
• https://holt.lib.wvu.edu
• https://rockefeller.lib.wvu.edu
THANK YOU