us ability report on mobile commerce app

USABILITY EVALUATION REPORT

Mobile Trip Assistant Project for


Table of Contents

1. Usability Analysis 1.1 Introduction…………………………………………………………………………3

2. Usability Strategy…………………………………………………………………..32.1 Paper Prototype………………………………………………………………...52.2 1st Phase Model………………………………………………………………...52.3 Final Recommendation………………………………………………………...5

3. Users3.1 User Profiles…………………………………………………………………...63.2 Process Description……………………………………………………………83.3 Schedule of Tests……………………………………………………………..11

4. Feedback & Critical Incidence4.1 Introduction…………………………………………………………………...12

4.2 Pre and Post Test Questions…………………………………………………..124.3 Analysis and Incorporated Changes…………………………………………..13

5. Screen Shot………………………………………………………………………………….21 6. References…………………………………………………………………………...28


1. USABILITY ANALYSIS

1.1 INTRODUCTION

Mobile devices have and are becoming extensively important in our daily lives. Explosion in IT and the development of newer software tools has set in a revolution in Mobile Technology. Trained professionals are required who can ideate, innovate and put New Mobile Systems in place and manage them effectively for the benefit of the stakeholders in particular and the society in general. It is important for such professional to evaluate the usability of the system they are implementing.

2. STRATEGY

Our usability report will be focusing on the following development methods:a. Generic Description: Presenting a general idea of our usability approach.b. Purpose: Will highlight the rationale for our analysis in designing the mobile software for PAAC. This analysis involves a deep and extensive study of users performing real tasks in their environment. The methods that are focused are:

a. User Context Logging: Selected users providing the feedback “on the fly”. The factors taken into account are activity, time, location and social situation. We planned our user tests in an environment which would be used by the user himself when interacting with the application. b. Patterns of Activity:Monitoring the technology interface and the user feedback/satisfaction through interviews and questionnaires. This step enables us to evaluate how the mobile technology is integrated in the user’s activities and how effectively the user accepts such integration. The data collected in this stage will be used by the developing team to incorporate the required changes in to the software. This data would also provide an insight for carrying out the usability analysis for other users.


Mobile Tapping: For any mobile application, the first and foremost necessity is to enable an effective user interaction. This pattern of study enables us to perform and analyze how the user will interact with our mobile application while moving or sitting. Since the Mobile Trip Assistant will be used by individual on the move as well as when stationary, we have to ensure that the application does not affect the daily activities of the individual. This study will however be performed through user context logging. We also test our application on individual who are left handed and right handed.

We performed our usability study in 3 steps:

ITERATIVE DESIGN PROCESS [2]

PAPER PROTOTYPE

1st PHASE DESIGN

FINAL PHASE DESIGN

Design Goals Design Guidelines

Prototype Interface

Formative Evaluation

Operational Interface

Summative Evaluation


2.1 Paper Prototype:A user centered design process which involved creation of a throwaway rough sketch of our mobile application interface used for development. This provided a useful feedback for the entire creative team as everyone brainstormed on the design together. The paper prototypes used in our user test are displayed here.

2.2. 1st Phase ModelThis phase led to the development of our initial mobile application. The feedback we got from the 1st user tests were incorporated in this phase.

2.3 Final ModelThis was our last phase of development. The feedback received from 2nd round of user tests were integrated in our final version of mobile application.


3. USERS

3.1 User Profiles:

# Name Gender/Age Category Profession1. USER A M/22 Expert Mobile

UserStudent at Carnegie Mellon University

2. USER B M/32 Mid-Level User

Working Lady

3. USER C F/48 Basic User House wife

Expert Mobile User: This type of user is termed ‘Expert’ on the basis of his/her usage of mobile applications in daily life. This user interacts heavily, and much of his daily activities involve using of mobile applications. Being a student, the expert is a frequent user of the bus service. He commutes to all the places within Pittsburgh using the bus transport. He relies on the bus schedules to a major extent, and plans his trips by using the transport authority bus schedules via the mobile app or websites.

Mid Level Mobile User: This user has moderate or average usage of mobile apps in the daily life. This user interacts moderately, and much of his daily activities don’t involve using of mobile applications. Being a working man, the user is a moderate user of the bus service. He commutes to certain places within Pittsburgh using own transport. He relies on the bus schedules to some extent, and plans his trips by using the transport authority bus schedules via the mobile apps or websites, but the frequency of using them is moderate.

Basic User: This user has very less or no usage of mobile apps. This user hardly uses the mobile apps in his daily activities. Being a house wife, the user is not a major or moderate user of the bus service. She commutes to most places within Pittsburgh using own transport. She relies on the bus schedules to some extent, and plans her trips by using the transport authority bus schedules via the mobile apps or websites, but the frequency of using them is very low.


The user age group was selected on the basis of the following demographic:

We see that the majority of our users are spread over android, iPhones and other Smartphones. Hence we chose our users on this basis.


3.2 Process Description

User evaluation was conducted in a simple classroom with the phone and the web based application given to the user. One of the team members took the responsibility of conducting the interview and another member took the notes. We decide to have a 15min session with each user. We divided our tests with 3 users in each of our stages of development.

Our usability testing was divided into the following tasks:

Test # Test Case Test Case Description Paper Prototype1. Locate Routes a.1“Current Location” as keyword (already

entered in the Source field)a.2 Enter the destinationa.3 Enter date (default will be already entered- System Date)a.4 Enter time (default will be already entered- System Time)

2. Trip Planner a.1 Enter some location other than the current location as the keyword a.2 Enter the destinationa.3 Enter date (will be already entered- System Date)a.4 Enter time (will be already entered- System Time)

3. Favorite Feature We tell the user to select the current route and add it to the “Favorite” Feature in our application. This will allow the user to add/save his favorite, and most used route.


4. Search Routes by Bus Number

We tell the user to search a particular route by entering the bus number the user wishes to search for. We provide the user with a list of buses to select from.

5. Nearby Application Feature

Here we tell the user to interact with out feature “Nearby”. This feature allows the user to know about the nearest hotels and restaurants around the users’ current position.

6. News Feed We tell the user to search for latest news and updates from the PAAC Website news feed we have in our “News Feed” application.


7. Crowd Shout It’s a social media interaction feature which allows users to post comment and photos about their experience with a particular bus. This feature will be addressed as a recommendation in our design.

8. Weather We tell the user to check for weather updates using the weather application

9. Explore In this feature we tell the user to explore various restaurants, hotels and coffee shops, etc around his current location. This feature is different from our “nearby” feature.


SCHEDULE OF TESTS:

Following information provides the date on the various user tests we performed.

TEST #1: Paper Prototype Test (With 2 Users)

Date of Test Name of User24th April- Sunday USER A24th April- Sunday USER B

TEST #2: 1st Phase Development Test (With 3 users)

Date of Test Name of User26th April- Tuesday USER A26th April- Tuesday USER B26th April- Tuesday USER C

TEST #3: 2nd Phase (Final) Development Test (With 3 users)

Date of Test Name of User29th April- Friday USER A29th April- Friday USER B29th April- Friday USER C


4. FEEDBACK &CRITICAL INCIDENCE

4.1 IntroductionWe performed a series of tests to check the functionality and features of our mobile application. Each of the users was required to answer a set of different questions (tasks) briefly explained below in table 2.

4.2 Pre and Post Usability Question Feedback.

a. Pre- Usability

Question User A User B User C1. What phone do you use? iPhone Android Motorola Flip Flop2. How many times a day do you use

the apps on your phone (1-10)?8 5 2

3. What is your data plan?4. Do you travel by bus? If yes, then

how often (on a scale of 1-5)5 3 2

5. Do you use Google Maps to get the information of the bus schedules?

5 4 2

b. Post- Usability

Question User A User B User C1. How much score would you give to

our application (1-5)?4.5 4 4.5

2. How much score would you give to our application’s design (1-5)?

4 4.5 4

3. How much score would you give to our application’s functionality (1-5)?

5 4.5 5

4. Is our application difficult to use/ is 4.5 4.5 3.5


our application complex?5. Do you like our application? 5 5 56. Would you recommend the

application to your friends?5 4 5

4.2 Analysis and Incorporated Changes:The following tables provide a detailed feedback of our user tests and our the changes we incorporated by analyzing the user tests.

Feedback Table (Table 1): a. Paper Prototype Feedback:

# Screen User’s Feedback Reason for Feedback Scope Severity

1. Tripplan1-Basic

The radio buttons were uncomfortable to be used on touch screen. The use also questioned as to how they will enter the date and time.

The users were not able select the radio buttons effectively.

Tripplan1 screen

High

2. Tripplan2-enter dest

The user was not sure of his current position, and also the map we provided was not a real map.

The users wanted that their current position would be entered by default and the user only had to enter the destination.

Tripplan2 screen

High

3. Weather The weather option should also provide the user to select the time of the day he wants to known the weather. The use also questioned as to how they will enter the date and time.

The users wanted not only the date but also the option to select the time in order to know the temperature on that day

Weather Screen

Medium

The C/F option is good The users liked the C and F option for displaying the temperature

Weather Screen

Medium

4. Favorite There should be option to search for buses which are not mentioned.

The users could not find the bus he wanted to add as favorite. So he wanted to a search option that will help the user add a bus which is not mentioned in the given list.

Favorite Screen

High

5. Nearby There should be option to The users wanted to find more Nearby Medium


search for more places (specific) which are not mentioned.

options like libraries, etc. So he wanted to a search option that will help the user with the search.

Screen

6. All Screen Logo of the application must be on each and every screen.

The users pointed this out because while he was using the app, he forgot the name of the app.

All screen High

7. Explore There should be option to search for more places (specific) which are not mentioned.

The users wanted to find more options like libraries, etc. So he wanted to a search option that will help the user with his search

Explore Screen

Medium

The names of the places should also have a comment mentioned by one of the customers

The users was comparing the ‘Explore Feature’ of our app with that of FourSquare. The user wanted a comment from a customer either telling about the place, or some specialty of that place

Explore Screen

Medium

8. All Screen The user was not able to go back from a particular page without selecting the tabs we had provided below for navigation.

The users wanted a ‘Back’ button to be on all the sub-pages

All Screens High

b. 1st Phase Development Feedback:

# Screen User’s Feedback Reason for Feedback Scope Severity1. All Screens All the features must be

enlisted on each page of the application.

The users were told about the Favorite, weather, and explore, but was not able to find the tabs/link on the main screens. The users wanted all the tabs to be present on all the screens, so that the navigation to other features becomes easy.

All concerned screens

High

The heading “Port Authority of Allegheny County should be larger and clearer.

The users were not able to clearly read the heading of our application.


High

Icon image on “Map” and “Feeds” tab were not correct.

The users were confused with image on the tab map and feeds. They wanted a map image on the “Map” tab and news image on


High


the “Feeds” tab.The “options” button on the top right corner was not working correctly.

The options button was not displaying the required content.


High

2. Tripplan1-Basic

The font of the header “Find Route By” was small.

The users were not able to read the font of the heading on the main screen easily. Moreover, the other headings had font greater than the heading.

Tripplan1 High

3. Tripplan1 – Select Date and Time

The font of the header “Choose a Route” was small.

The users were not able to read the font of the heading on the main screen easily. Moreover, the other headings had font greater than the heading.

Tripplan1 – Select Date and Time

High

Icon image on “Select a date and time” was not correct.

The users were confused with image on the “Select a date and time” tab. He wanted a time image on the tab.


Medium

4. Tripplan1- date time entered

The date and time format was not readable.

The users were not able to clearly understand the date and time format as the they were not clearly spaced (no commas)


High

Icon image on “Select a date and time” was not correct.

The user was confused with image on the “Select a date and time” tab. He wanted a time image on the tab.

Tripplan1- date time entered

Medium

The font of the header “Choose a Route” was small.

The users were not able to read the font of the heading on the main screen easily. Moreover, the other headings were clear (had font greater than the heading).

Tripplan1- date time entered

High

5. Tripplan1-enter curr and dest

The date and time format was not readable.

The users were not able to clearly understand the date and time format as the they were not clearly spaced (no commas)

Tripplan1-enter curr and dest

High

Current and Destination entering feature is good.

The users liked our interface of entering the current and destination location.


High

6. Tripplan1-enter date and time

Date and time entering method is good

The users liked our interface of selecting date and time

Tripplan1-enter date and time

High


7. Tripplan1-Results Page

The page is dull. The users were not impressed with the look and feel of the page

Tripplan1-Results Page

High

8. Tripplan2-enter dest

The users wanted to know their current location.

The map that only wanted the users to enter their destinations did not show where the user was currently.

Tripplan2- enter dest

High

The destination feature is good.

The users liked the idea of displaying the current location and only allowing the users to enter their destination details only.


High

9. Favorite The font of the header “Choose a Bus Number” was small.

The users were not able to read the font of the heading (“Choose a Bus Number”) on the main screen easily.

Favorite High

“Filter items” is a confusing phrase.

The users were confused on the phrase “Filter Items”. The users though new that they had to write on the tab, but the phrase were unclear to them.

Favorite High

10. Favorite- with filter

The font of the header “Choose a Bus Number” was small.

The users were not able to read the font of the heading (“Choose a Bus Number”) on the main screen easily.

Favorite High

11. New Feed Just the twitter feeds are not helpful.

The user wanted more options/features for the news feed.

News-Feed High


c. Final Phase Development Feedback:

# Screen User’s Feedback Reason for Feedback Scope Severity1. Tripplan1-

BasicUsers liked the interface for the main screen.

The users were easily able to understand what the functionality of the main page is. The users also felt that the phrase used: “how can we help”, is a good way to interact.

Tripplan1-Basic

High

Good way to represent favorite routes

The users liked the representation of our favorite feature tab- which had a different icon.

Tripplan1-Basic

High

2. Tripplan1-Results Page

Good look and feel. The users liked our design of representing the results page and also providing options to select a particular bus stop.


High

3. Tripplan1-Route Results Page

The selected stop result was nicely displayed.

The users liked our route result map page which displayed the nearest stop address and location (marked by red pin) selected by the user on the Trip Planner Stop Results Page.

Tripplan1-Route Results Page

High

4. Nearby Stops

Good way of providing list of all nearby stops.

The users liked our idea and interface of displaying all the stops near to the users’ current location.

Nearby Stops

High

5. User Feedback

Feedback Interface was intuitive and simple.

The users liked our feature of providing “Feedback” for a given route/trip. The slider bar which provided them a way to rate their trip was effective and easy to use. The users also liked the idea of expressing their comments on the feedback column.

User Feedback

High

6. Tripplan1- enter dest

An effective and simple design to help locate nearby

The users liked our map interface which tells the routes to user from


High


or required stops. his current location.7. Select Bus-

1,2Interactive Bus Selection System

The users liked our Bus Select Feature in our application.

Select Bus-1,2

High

8. Bus Schedule -1,2

Interactive way of getting information about the bus stops.

The users liked our feature of selecting the bus stops and then having a drop down menu for getting the information about the bus stop timings.

Bus Schedule -1,2

High

Implemented Changes (Table 2):

a. Paper Prototype Changes:

Screen Name Way(s) to RectifyTripplan1 1. We provided the user with a way (used in iPhone) to select the date and time. A

submit button was also provided to the user to submit their date and time requests. The selected date and time appear on the screen of the user.

2. After the date and time is entered, the same way the user can enter their locations (source and destinations) as mentioned above.

3. Once the user was given with the list of the available buses, we provided a “Filter Option” to them. For Ex: Suppose a user doesn’t want to see all the buses, so in the “Filter Item” tab, the user can type the number of the bus, and all the buses with the similar number like 61A,61B,61C,61D will show up.

Favorite 1. We added the list of all the buses available in Pittsburgh. This way we provided the user the option to select whichever bus they want to.

Tripplan2 1. Here we provided the user with the option of entering the destination via the method


mentioned in point 3. Our GPS system locates the current position of the user and shows that position on the map. From there the user can select the destination.

2. We also provided the user with a real time map, with the pin pointing to the user’s current position.

Nearby 1. This feature will be implemented once the basic operation of our mobile application is completed. This is an added feature, which we intend to implement before our final development phase.

Explore 1. This feature will be implemented once the basic operation of our mobile application is completed. This is an added feature, which we intend to implement before our final development phase.

All Screens 1. We added a ‘Back’ Button on each of the sub pages.2. We introduced the application Logo on each of our main category pages.

b. 1st Phase Development Changes:

Screen Name Way(s) to RectifyAll Screens 1. All the features – Route, Nearby Stops and News Feed tabs were added on all the

screens to enable easy navigation. Also their icon images were changed keeping the feature name in consideration. Ex: For Routes- we have a map tab.

2. The heading “Port Authority of Allegheny County” was resized to make it clear.Tripplan1-Basic 1. The font for the heading: header “Find Route By” was changed to “How can we

help?” and also the following titles were rephrased. Ex:

a. Address Trip Plannerb. Bus Number Bus Schedulesc. Favorite Favorite Routesd. Maps Nearby Stops

2. We also added an advertisement bar on the same page.


Tripplan1- Select Date and time

1. The font for the heading: header “Choose a Route” was resized to make it clear.2. Icon image for “Select Date and Time” was changed and relevant one was put.

Tripplan1- date and time entered

1. The date and time format was corrected with commas and proper spacing added where required.

2. Icon image for “Select Date and Time” was changed and relevant one was put.3. The font for the heading: header “Choose a Route” was resized to make it clear.


1. The date and time format was corrected with commas and proper spacing added where required.


1. The design for the results page was improved by adding the stop information in bold and color box.


1. We added a red pin indicating users of their current location.

Favorite 1. The font for the heading: header “Choose a Bus Number” was resized to make it clear.

2. We feel that the phrase “Filter Items” was apt for the given screen. Too many words were not providing a good interface.

Favorite-with filter

1. The font for the heading: header “Choose a Bus Number” was resized to make it clear.

News Feed 1. New Options were added into the news feed feature. They are a. PAAC Announcementsb. Crowd Shoutc. Trip Review

c. Final Phase Development Changes

Instead of the design changes for screens in the 1st phase development, in the below table we describe the extra features that were added into our mobile trip application.

Screen Name Way(s) to RectifyRoutes Result Page

1. We added another page to display the address of the nearest bus stop.

Nearby Stops 1. We improvised the design and functionality for the nearby stops feature in our application by providing a list of all the stops near the users’ current location.

User Feedback 1. We added another feature of “User Feedback” into our application.Select Bus 1. We provided the users an effective way of selecting a particular bus from the “Bus

Schedules” feature on the main screen and then get the route for the selected bus.Tripplan1-Basic 1. The home screen was revised keeping in mind the functionality and features

required.


Tripplan1- Current and Destination Secletion

1. We provided a link called “Current Location” on this screen helping the user enter his current location by selecting the link.

Bus Schedule -1,2

1. We added a drop down menu on the list of all the bus stops and the timings of the stops in the respective bus routes.

5. Screen Shots:Below are the screen shots of our mobile application from the first phase development to the last and final version.

The following table represents the first phase development and the second side the final mobile application. There are certain pages in which there are no 1st phase development screen shots. This is because we included certain features in our application in the final phase development, which were not present in the 1st phase.

Screen Name 1st Phase Development Final Phase Development

Tripplan1- Basic


Tripplan2- Departing and

Destination Selection

Tripplan2 – Time and Date

selection


Bus Schedule -1

Bus Schedule -2 (Expanded)


Tripplan-1: Results Page

Tripplan1- Route Result

Screen


Favorites

Favorites- with Filter

Option


News Feed

Map and Destination

Screen


Nearby Stops 1

Nearby Stops 2


User Feedback

Select Bus-1


Select Bus -2


6. REFERENCES:

1. Mobile Computing:http://en.wikipedia.org/wiki/Mobile_computing

2. Usability Checklist for the Usability Evaluation of Mobile Phone User Interface

http://en.wikipedia.org/wiki/Mobile_computing

us ability report on mobile commerce app

Documents