IA05: User Observation

Posted: Monday, Sep 12
Deadlines:
  • Partnerships Formed: Thurs, Sept 15, 11:59PM (You must form your group on Canvas by this deadline)
  • Final Report: Tues, Sept 27, 11:59PM

Assignment Overview

Formative inquiry is often performed at the beginning of a design project to better understand a given problem space and associated human behaviors, social contexts, and environmental constraints therein. In this assignment, you will practice using formative inquiry methods--namely, in-situ observation--to understand how people currently use the DC Metro ticket machine and turnstiles.You will analyze this data, identify where and how people struggle, and sketch out alternative designs. Note: although this is under "individual assignments," you are to do this assignment with a partner.

IA05-ObservationSteps.png



Assignment Parts
There are four parts to this assignment:
  1. User Research Ethics. Read the UMD Institutional Review Board (IRB) documentation and determine whether this assignment would require a minimal risk review, an exempt review, an expedited review, or a full board review if it were a proposed research project at the university. Why did you make this selection? Do you think this sort of observational research requires informed consent? Why or why not? What are some ethical issues with observational research? To help answer these questions and learn more about working with participants in HCI, please read Section 14.2 from Lazar, Feng, Hochheiser's book Research Methods in Human-Computer Interaction (2010).

  2. Performing the User Observation. Together, you and your partner must go to a Metro station of your choosing (the College Park station and Greenbelt station are the closest) and collect observational data about how people interact with various Metro human-computer interfaces (primarily the ticket machines and electro-mechanical turnstiles). You should identify two tasks in particular (one for the ticket machine, and one for the turnstiles) and focus on the steps necessary to accomplish those tasks. For example, for the ticket machine, a task might be "buying a ticket or smart trip card for the first time." Another example might be "refilling a smart trip card with $15.50." For the turnstile, there are of course fewer tasks but be creative: one task may be "getting through the wheelchair accessible turnstile" and another might be "using a smart card with the turnstile" and finally "using a paper ticket with the turnstile." For each task that you decide upon, I want you to observe how people accomplish that task--see if you can identify "expert" users vs. "first-time" users. What are some mistakes that "first-time" users seem to make? What are the confusing parts of the interfaces (e.g., with the turnstile, how does someone know if their smartcard scanned correctly--what feedback does the machine give them?)

    Data Collection. Your observational data should include notes, pictures, and, in some cases, video (e.g., recorded via your smartphone). Each report must include at least three pictures of the interface. Bear in mind, however, that you should not be intrusive. You do not want your presence to affect how people ordinarily behave. IDEO refers to this as the Fly on the Wall method--a type of Contextual Inquiry. The goal is to observe and record behavior within its context, without interfering with people's activities. The data you collect should be anonymized. For example, your pictures/videos should not have identifiable information--you should avoid recording faces. If you observe a behavior but do not want to take pictures/videos of a stranger, you can have your partner takes pictures/videos of you replicating this behavior (for the purposes of demonstration).

  3. Data Analysis. Then, analyze the information you've collected to identify problems, patterns, and insights.

  4. Sketching Alternative Designs. Finally, sketch out some alternative designs that help fix the problems you've identified.

Your Report

Your report should be structured as follows:
  1. Provide background on your observational method: how did you approach this assignment? Which Metro station(s) did you choose? How long were you there for? How many people did you observe? Why did you choose the approach that you did? What would be an alternative approach?

  2. Discuss the UMD IRB documentation and your answers to the IRB related questions above.

  3. Briefly explain the two tasks that you decided to study and why.

  4. For each task, create a top-level header ("Heading 1" style) with the name of the task. Underneath the header, describe the steps necessary to accomplish this task (e.g., the first step to recharging a smarttrip card is to hold the card up to the RFID reader, which is a circular shape on the front of the machine). Use supporting images (with annotations) to help provide a clear breakdown of the task. Alternatively, if you recorded a video demonstrating these steps, you can refer to the video (with link) in the text rather than using supporting images.

  5. Create a sub-heading ("Heading 2" style) called "Challenges, Errors, and Successes." In this subsection, enumerate the challenges, errors, and successes that you observed for this specific task. Categorize the challenges and errors into low, medium, and high severity (where high severity is an issue that would likely prevent the Metro user from successfully getting on a ticket or through the turnstile). Use supporting images/figures to help describe these issues.

  6. Create another sub-heading ("Heading 2" style) called "Recommendations for Design." In this section, suggest specific design fixes/alternatives that DC metro could implement to fix your identified problems. Be creative but practical. Provide at least one sketch with your recommendations. Update (2/13): we strongly prefer that these sketches are on paper in your sketchbook and scanned/photographed into your report.

  7. Loop back to step #4 above to describe your second task.

  8. Create a top-level header ("Heading 1" style) called "Concluding Remarks." Here, provide at least one sentence describing what you learned from this assignment, one sentence describing what you didn't like from this assignment, and one sentence explaining how this assignment could be improved in the future. :)

As with many assignments in this class, Matt and I reserve the right to award bonus points for excellence and creativity.

Example Reports

Matt and I have selected a few high-scoring reports from previous semester. Note: we improve/change the assignment every semester, so make sure you base your reports on the above sections rather than strictly these examples.
  • Example 1. : This example scored highly and demonstrates the type of visual style we are looking for in this assignment.
  • Example 2. : Additionally, this example demonstrates how basic photo editing tools can be used to present recommendations.
  • Example 3. : This example demonstrates all of these concepts and good use of bulleted lists with prose.
  • Example 4. : Another example of a highly scoring submission.
  • Example 5. : Finally, Example with very good photos and clear writing.

Deliverables

Please upload your report as a PDF or Word Docx including inline images taken during your observational study and your sketches to Canvas. If you took video that you think highlights an issue particularly well, please upload this to YouTube and include a link in your report. Make sure both names are on the report. There is a hard limit of 9 pages (including images/figures and references).

Rubric

We want to emphasize a few things:
  • Presentation matters. An excellent report will not only incorporate lots of images to help communicate but carefully layout these images, provide on-image annotations/labels as necessary (with, for example, arrows), and include well-written captions
  • Prose preferred to lists. While bulleted lists are often useful in enumerating points, your primary text should be prose.
  • Figures. You should appropriately reference figures in your text, which means your figures should be appropriately numbered, titled, and captioned.

You can download the rubric as an Excel file below:


Click on the rubric to enlarge:
IA05-Observation-GradingRubric_v1b_MLM_cropped_650w.png

Best Of (Fall 2016 Edition):