1. Last update: 2014.3.18 Zhao Wenbin Chen Cheng Wu Tianyi Xu Yihua Zhu Ruoyu Zuo Xingyuan Roomie Usability Testing Report

2. Roomie Usability Testing Report 3. 1. Introduction About Roomie Roomie is an online platform that aims at helping university students find suitable roommates in Hong Kong according to their personality, life style and other daily issues. 4. Target population of Roomie While the primary users of Roomie are students from Mainland China, as most of them cannot apply for the university dormitories, we also prefer to target graduates who determine to work in Hong Kong but lack roommates to share an apartment, as well as local students who prefer to live separately from parents. 5. Goal of usability testing Our goal for the usability testing was to collect both quantitative and qualitative data of new users learning how to use Roomie to search for potential roommates for the first time. We hope that the results of the usability testing can be used by us to understand the strengths and weaknesses of this project and serve as a benchmark for future usability testing. 6. 2. Methodology Overview Usability testing occurred during the one-week span of March 9-16 (the first round on March 9 and the second round on March 16). Ten participants were recruited who had never heard of our project before but yet had a need to look for roommates, or had related roommate-finding experiences. During the half-an-hour testing sessions, each participant was asked to do the tasks of searching for suitable roommates on Roomie. Time spent on the tasks and qualitative comments were collected for each participant. Each session was recorded using QuickTime to allow for later analysis. 7. Participant Selection & Recruitment Since our goal was to find the strengths and weaknesses of this project, we recruited users who had never heard Roomie before but had a need to look for roommates, or had related roommate-finding experiences. They should be future students, present students or graduates of a certain university in Hong Kong. We recruited our participants through our social networks, and offered them a small gift as an incentive for participation. 8. Test Protocol Each test session last for one hour, including 10 minutes for a pre-test background interview and post-test debriefing questions, and 20 minutes for the tasks. We began each test session by asking them some general questions about how they got along with their former roommates, whether they had some good or bad experiences, and what they cared the most when looking for a roommate. Then we gave them two tasks, and asked them to complete each one while talking aloud about their thought process and any confusion they encountered. 9. Test Protocol The two tasks we gave them were organized by the sequential order. The summary of tasks is listed below. Task Number Task Summary 1 Register for Roomie 2 Search for roommates on Roomie 10. Test Protocol After the participant completed the tasks, we conducted a post-test brief, asking them about their interesting or confused points about Roomie. We also asked them to score each page according to their satisfaction (4-point Likert scale, from 1 =dissatisfied to 4=satisfied) and asked if they thought they would be likely to use Roomie, based on their initial impression. 11. Test Environment and Equipment The tests were conducted in Learning Commons of Wu Ho Man Yuen Building. We used QuickTime to record our test sessions. We each took turns moderating one session and observing one session. We set up QuickTime on a Mac, and give this Mac to participant to use during the test. 12. Data Collection We collected both qualitative and quantitative data about the participants behavior. Our quantitative data consisted of the performance data of time spent on tasks for each participant, and their satisfaction (4-point Likert scale) towards Roomie. The qualitative data we collected were quotes, comments and observations noted down by the observer during the test. 13. 3. Results Score analysis 3. 3 3. 6 3. 6 2. 5 3 3. 1 2. 8 0 0. 5 1 1. 5 2 2. 5 3 3. 5 4 Test 1 Test 1 14. 3. Results Score analysis 2. 7 2. 8 2. 9 3 3. 1 3. 2 3. 3 3. 4 3. 5 Test 2 Test 2 15. 3. Results Score analysis 0 0. 5 1 1. 5 2 2. 5 3 3. 5 4 t est 1 t est 2 16. 3. Results Time spent on task Tim e spent on task 0 5 10 15 20 25 1 2 3 4 5 Minutes Test 1 Test 2 17. Test1 Findings Broken experience 1 The design of users own information page is not clean and clear. Mentioned times: 5 I find the table very stressful and the classification of information is not clear. 18. Solution 1: Change table to listed information, add more suctions in users own information page. 19. Broken experience 2 More personal information such as lifestyle is necessary. Mentioned times: 4 I hope to see more personalized information to know what kind of person the potential roommate is. 20. Solution 2: Provide choices related to lifestyle and apartment requirements . 21. Broken experience 3 3 persons points out that The whole process of registration and search is not very smooth, some buttons are missing. I think there should be a next or finish button. I cant go back to the homepage. 22. Solution 3: Adding navigation at the top of each page, and change submit to finish. 23. Broken experience 4 3 persons points out that The picture in registration page cannot clearly represent the function of Roomie. The picture make the website look like a social media. 24. Solution 4: Changed to a more clear and high quality picture, cannot find a suitable picture with Asian faces. 25. Broken experience 5 2 persons points out that The advanced search is confusing, I do not understand the relationship between apartment and roommate. 26. Solution 5: Change boxes to listed options to make it easier to understand, and also add missed buttons. 27. Test2 Findings Recommendation function on the homepage is overlooked by users. See I can search the website. If there are recommended roommates for me, that would be great! Broken experience 1 28. Solution 1 We put recommendation part on the top of homepage to reinforce its function, and put search bar on the right side. If users want to search more information, they can click Advanced Search and go to search page. Test2 Findings 29. Test2 Findings After registration process, user will landing to personal homepage, but there is no instructions for what to do next. OK, Ive registered. So what I goanna do next? Broken experience 2 30. Test2 Solution 2 Besides words to describe the characteristic,we first provide menu for users to choose their character type. 31. Broken experience 3 2 person points out that they can not understand the word and , and actually the type of the building is not that important. Moreover, the type of the apartment is something that they concern more. I dont know how to choose, I have no idea what it is. I Test2 Findings 32. Solution 3 To be more clear, we change and to and . We also add Type of Apartment to provide more details. Test2 Findings 33. Broken experience 4 2 person points out that they cant understand what grouping means when I want to leave a note to other users. I dont know what the grouping here means, will it appear every time I leave message for someone? Will other people see my grouping information? Would it be more appropriate if it shows when I liked someone? Test2 Findings 34. Solution 4 We revise Grouping faction, add a drop list for users to select and add group category. Test2 Findings 35. Broken experience 5 2 person points out that when searching, they put gender in priority. Where is the gender in the search box? I would definitely want that option before all the other choices. NO GENDER! Test2 Findings 36. Solution 5 We rearrange the choices and add so as it can be seen more easily. Test2 Findings 37. In the process of finding search function, user click people I interested in below head portrait instead of search button on the header. However, people I interested in is not the right search button. There is some misunderstanding of people I interested in. Ill click this(people I interested in) to see who is my suitable roommate. Broken experience 6 Test2 Findings 38. Solution 6 We build list page of users who I am interested in after click people I interested in button. Users can check and manage all the list here. Test2 Findings 39. Broken experience 7 2 users pointed out that they dont want to type in the descriptions of their characters, or it took some time to think about how to describe themselves. So there is a high possibility that they will leave in this stage. I dont want to type, I want to choose. Test2 Findings 40. Solution 7 Besides words to describe the characteristic,we first provide menu for users to choose their character type. Test2 Findings 41. Broken experience 8 In the fill-in personal information page, we want to collect more information about the user. But 3 users complained that they dont want to fill in all the information. Some information I dont want to share to others. Its all required? Test2 Findings 42. Solution 8 We add * in front of those mandatory fields to distinguish from optional ones, so as users can clearly figure out which part they must fill in. Test2 Findings 43. Test2 Findings Further demand of renting a house can not be fulfilled in our website Besides finding a roommate, I want to find a flat. If there is only information of roommates, I think thats not enough. 44. 4. Conclusion The tasks are about the basic functions of Roomie. Most users successfully finished all the tasks. All of them experienced some difficulties but all successfully figured out a way to overcome them. From the first-round test we found many weaknesses of Roomie and optimized it according to participants suggestions. And in the second-round test, though Roomie had been optimized, participants still figured out some broken experiences that need to be solved. It means that to satisfied users real needs, we still have a lot to do.