Project - Paper Prototype
- Testing session: March 31 (Wednesday)
- Writeup for pilot testing: April 5 (Monday)
- Writeup for real testing: April 12 (Monday)
In this project assignment, you will build and test a paper prototype of the application you’ve been designing. The paper prototype should be able to handle at least 3 scenarios. These scenarios will probably be the scenarios you described in the previous project assignment.
On March 31, we will use class time to allow you to conduct pilot testing on the paper prototype you’ve built on your classmates. The session will be divided into three periods, and the projects will likewise be divided into three groups. During each period, two-thirds of the class will run their prototypes, while the remaining third serve as users.
Since your classmates are similar to you (CS or EE students) and do not necessarily represent your target users, they are considered pilot users. Testing on pilot users is a convenient way to identify the most obvious usability problems and help you practice running your paper prototype with people other than your team members.
After the in-class testing session, you should find some realistic users. First, revise your paper prototype to address the critical usability problems and explore possible design alternatives. Then, test it on at least 3 users from your target population (or user representatives if recruiting real users is not possible). These users can not be your classmates in this class.
Preparing for the Testing Day
Before testing your prototype, you should:
Develop 3 scenario tasks
Identify the risky parts of the UI (refer to lecture notes). Choose 3 tasks that allow you to test these parts. Ideally you can use the tasks you wrote scenarios for in the previous project assignment with some modifications. Write the concrete goal(s) of the tasks you’ve chosen (e.g. "buy a ticket” instead of “clicking on buy button”). Don't write the specific steps to follow, since that's for your users to figure out (so you can evaluate the learnability of your design). The tasks should be brief, roughly 5 minutes to run.
Review the content of the lecture on paper prototyping and recall your experiences from the in-class paper prototyping exercises. Based on the design sketches and UI storyboards your team created in the previous project assignment, consider what layout components can be static and pre-drawn, what contents need to be dynamically generated, what controls need to allow test users to interact with, and how all these components should be integrated together. Hand-sketching is highly encouraged. Make sure your prototype can handle the 3 scenario tasks you developed.
A paper prototyping session typically involves four roles: computer, facilitator, observer, and user. Decide which role each team member will play for each round of the testing. It may be useful for you to swap roles after every user on Testing Day, so that each of you gets a chance to try each role, but decide how you'll do it in advance.
Prepare briefing notes
Each session with a test user should start with a briefing to inform the user the following:
- The purpose of your application
- Any background of the domain (e.g., medical) needed to understand what’s going on
- The tasks the user needs to perform
Prepare notes to help the facilitators remember what to tell the user, for example, writing down key points on index cards.
Practice each role beforehand, including playing the computer, briefing test users (facilitator), writing down observation notes (observer). Every team member should learn the steps involved in making the prototype functional, such as rearranging pieces and writing responses.
On the Testing Day
The testing session is divided into three periods. The session will be divided into three periods. Teams will be likewise be divided into three groups. During each period, two-thirds of the class will run their prototypes, while the remaining third serve as users.
Each period will then be divided into two rounds. In each round you will have a different user. Thus, your team will be able to test at least 4 users (2 rounds x 2 periods). You should aim to test all 3 tasks in each round. If there's time left in the round, talk with the user to get feedback.
Below is the schedule showing which teams will run their prototypes in each period and which teams will serve as pilot users in each round. There is a 2 minute and 3 minute break between rounds and periods respectively.
- Period 1: (Teams: 2, 3, 5, 6, 7, 9, 10, 11)
- Round 1: 2:03 - 2:13 (Pilot users: 1, 4)
- Round 2: 2:15 - 2:25 (Pilot users: 1, 8)
- Period 2: (Teams: 1, 2, 4, 6, 8, 9, 10)
- Round 1: 2:28 - 2:38 (Pilot users: 3, 5)
- Round 2: 2:40 - 2:50 (Pilot users: 7, 11)
- Period 3: (Teams: 1, 3, 4, 5, 7, 8, 11)
- Round 1: 2:53 - 3:03 (Pilot users: 2, 6)
- Round 2: 3:05 - 3:15 (Pilot users: 9, 10)
When you run your prototype on a user, you should do the following things:
- Brief the user. Use the briefing you wrote up to describe orally the purpose of the application and background information about the domain. Don't waste too much time on this: 1 minute should be enough.
- Present one task. Hand the index card to the user and let them read it. Make sure they understand the task.
- Watch the user do the task. Take notes from your observations on the discussion page of your team Wiki page.
- Repeat with the other tasks. Run as many tasks on the user as you have time for.
Bring extra materials on Testing Day. Having extra parts such as Post-it notes, correction tape, and index cards on hand will help you improvise on the spot if a user does something unexpected, or help you make small fixes to your prototype between users.
Serving as a Pilot User
When you serve as a pilot user, you will be given a sheet to write down the team for which you have served as a user and rate your experience. The sheet will be handed to you in the beginning of the class and collected from you at the end of the class.
Below are some tips for being a good pilot user:
- Relax and enjoy yourself. Remember it is the interface that's being tested, not you. The experience of being a pilot user allows you to get a feel of what it's like to be users in a user test, so that you can empathize with them.
- Be cooperative. Interact with the interface as you would if you were really using it instead of trying to break it.
- Think aloud. Help the observers understand what you're thinking by verbalizing your thought process. "Let's see, I want to order a bottle of shampoo, so where's the button for adding the shampoo to a shopping cart, hrm. is there a shopping cart, oh there is, oh there is an "add to cart" button there ... " You get the idea.
Update your team's wiki page by adding the following subsections under the paper prototype section.
The first five subsections below are due on April 5 (the Monday after the testing day).
- Risk assessment. List the parts of your interface that you consider risky and state which scenarios will test each one.
- Prototype photos. Digital photos of the pieces of your prototype. Show the prototype in interesting states; don't just show a blank window. A digital camera will be available during Testing Day for you to take photos of your prototype, if you need it. (Although you will iterate your paper prototype during this assignment, the photos only need to show one iteration.)
- Briefing. The briefing you gave to users.
- Scenario Tasks. The tasks you gave to users, as you wrote them on the cards.
- Observations on the testing day. Usability problems you discovered from pilot testing.
The following parts are due on April 12 (which gives your team a week to improve the prototype and find real users to test it)
- Prototype iteration. Describe how your prototype changed between your Testing Day users and the real users.
- Observations on the real users. Usability problems you discovered with the new prototype from testing with real users. You should include at least 3 users from outside the class.
- Risk resolution. What did you learn about the risky parts of your interface from this prototype? Propose design solutions for the usability problems you found.