Usability Testing: Setting Objectives and Questions

Setting Research Objectives and Questions for Usability Tests

• Before conducting usability tests, it's crucial to define research objectives and questions.

• Regular testing is beneficial, but it should always be purposeful.

• Each test should have a clear objective and aim to answer specific research questions to help focus efforts.

Focusing Efforts with Specific Objectives and Questions

• Consider an airline website as an example to illustrate the importance of focus.

• Potential tasks include:

  • Booking one-way, return, or multi-city trips.

  • Booking for various passenger configurations (adults, children, infants).

  • Checking in bags or not.

  • Selecting specific seats or not.

• Secondary tasks:

  • Adding passengers to a group.

  • Canceling a booking.

  • Changing the itinerary.

  • Checking frequent flyer points.

• It is impossible to cover all tasks and scenarios in a single session, so it is important to narrow the focus.

Example: Improving an Airline Website

• Objective: Improve the website user experience and increase revenue.

• Research Questions:

  • Why are the airline's conversion rates below the industry average?

    • Conversion rate: the percentage of visitors to a website that perform a specific action.

    • If 3 out of 100 visitors purchase a flight, the conversion rate is 3%.

    • $Conversion\ Rate = \frac{Number\ of\ Conversions}{Total\ Number\ of\ Visitors} * 100$

    • A higher rate means the website is more effective.

  • Why are people dropping out along the purchase funnel?

    • Usability testing can uncover reasons behind user behavior.

  • Does the airline's booking process take longer than competitors, and are there unnecessary roadblocks?

    • Faster processes improve conversion rates.

  • Why are upsell and cross-sell modules underperforming?

    • Airlines generate revenue by upselling and cross-selling (e.g., seat upgrades, insurance, hotels, rental cars).

Focusing on Specific Primary Tasks

• Focus on tasks relevant to answering the research questions.

• Exclude irrelevant secondary tasks (e.g., canceling bookings, changing itineraries).

• Focusing on a narrow set of tasks (e.g., booking a return trip for multiple adults with checked bags) suffices because the booking process is similar across different scenarios.

Benefits of a Focused Approach

• A focused approach will:

  • Reveal why users are not converting to bookings.

  • Determine why users are dropping out of the purchase funnel.

  • Identify and eliminate unnecessary roadblocks in the booking process.

  • Show how users respond to upsell and cross-sell efforts.

Key Takeaways

• Set clear objectives and define specific research questions before planning usability tests.

• Keep the focus as narrow as possible when selecting tasks.

• Cover only the tasks that help answer research questions and achieve objectives.

• Avoid trying to cover everything at once. Doing so dilutes the quality of data.

Addressing the Scarcity Mindset

• UX designers often try to pack too much into a single usability test because they perceive it as a one-off event.

• Instead, usability tests should be scheduled regularly to facilitate continuous iteration and improvement.

• Regular testing supports iterative design improvements for a better user experience.