WEEK 10 Usability Engineering Evaluation Concepts

CS2003 - Usability Engineering

UE Evaluation

• Quickfire usability evaluation

• Instructor: Dr. Monica Pereira

• Email: monica.pereira@brunel.ac.uk

Lecture Overview

Today’s Lecture Topics:

• Evaluation: Definition and Purpose

  • What it is and isn’t

  • Why we do it

  • Fit with User-Centered Design (UCD) and Usability Engineering (UE)

• Role of Evaluation:

  • In/Formal evaluation

  • Goals and factors to consider

• Methods of Evaluation:

  • Discount usability testing

  • Heuristic evaluation and cognitive walkthrough

  • Strategies for quick and cheap evaluations

• Timing of Evaluation:

  • When and how often to evaluate in the UCD process

Technical Software Evaluation

Overview

• Formal Technical Reviews: Include non-developers

• Technical Software Testing:

  • White-box testing: Basis path and control structure testing

  • Black-box testing: Focus on functional requirements

• Software Testing Strategies:

  • Unit Testing

  • Integration Testing

  • Validation Testing

  • System Testing: Recovery, Security, Stress, Performance

• Note: Focus on software, not user interaction in this module

Understanding Usability Evaluations

Significance of Usability Evaluations

• Shift focus from "Does the software achieve its task?" to "How well and in what way does the software assist the user?"

• Key question: How to assess and measure usability meaningfully?

Definitions of Evaluation

• Evaluation: Process to gather information about system usability to improve or assess the system. (Preece, 1994)

• Evaluation Method: Procedure for collecting relevant data on the usability of a computer system. (Preece, 1994)

• Key questions addressed:

  • How to choose between interface alternatives?

  • How to improve an existing design?

Rationale for Evaluation

Goals of Usability Evaluation

• Compare designs and assess engineering toward goals

• Check conformance to standards

• Understand real-world efficiency and effectiveness

Reasons for Conducting Usability Evaluations

Key Questions to Address:

• How effectively does the design facilitate task completion?

• Are users able to manage emergency scenarios?

• Usability metrics for success and complexity:

  • Example: How easy is a ticket machine?

  • Size of screen correlated with selection errors?

  • Escape mechanisms for erroneous commands?

  • Rate of user retention and organizational adoption impact?

Establishing Goals for Evaluation

Key Evaluation Goals

• Assess system functionality and appropriateness

• Evaluate the effect of the interface on user experience

  • Learning ease, usability, and expectations

• Identify specific design problems, including contextual aspects

Key Considerations for Selecting Evaluation Approaches

Factors to Consider

• Characteristics of users

• Types of user activities

• Environment where the study takes place

• Nature of the artifact being evaluated

Role of Usability Evaluator

Responsibilities

• The evaluator acts as a critical friend to provide constructive criticism to designers

The Trunk Test: Website Evaluation

Concept

• Imagine being blindfolded and locked in a trunk. After navigating a website, can you answer these questions?:

  • What site is this?

  • What page am I on?

  • What are the key sections available?

  • What are my local navigation options?

  • How can I search?

Common Mistakes in Usability Testing

Common Pitfalls

• Overreliance on 'common sense'

• Incorrect assumption about typical users (testing only on self)

• Failure to involve representative users

• Delaying testing until too late

HCI Design Process and Evaluation

Life Cycle of HCI Design

• Star life cycle includes:

  • Requirements specification

  • Conceptual design

  • Prototyping

  • Evaluation (ISO 13407)

Timing of Evaluation

Phases of Evaluation

• Formative: Helps in decision-making and identifying issues early

• Summative: Conducted at the end of the design process to ensure goals are met

• Iterative testing favored over late summative testing

Forms of Usability Evaluation

Three Main Types

1. Analytical Testing: Expert review, involving opinions of usability experts

2. Abstract Testing: Drawing from existing data, such as cognitive analysis

3. Empirical Testing: Direct user testing through qualitative studies

Expert Review Techniques

Quickfire Approaches

• Heuristic Evaluation: Scorecard approach assessing against usability principles

• Cognitive Walkthrough: Step through user actions to simulate tasks

• Formal Usability Inspections: Collaborative meetings to discuss strengths and weaknesses

Analytic Method: Heuristic Evaluation

Process

• Experts evaluate the user interface against usability principles

• Duration: Typically 1-2 hours

• Output: List of usability issues identified

Problem Reporting

Guidelines for Reporting Issues

• Each problem should include:

  • Description

  • Anticipated user difficulties

  • Context of the problem

  • Assumed causes

Key Heuristic Principles

1. Visibility of System Status

• Feedback mechanisms based on response time

2. Match Between System and Real World

• Use user-friendly terminology and conventions

3. User Control and Freedom

• Features like Cancel and Undo should be readily available

4. Consistency and Standards

• Maintain user expectations through standard practices

5. Error Prevention

• Design to prevent errors from occurring initially

6. Recognition Rather than Recall

• Minimize memory usage through visible options

7. Flexibility and Efficiency of Use

• Provide shortcuts for experienced users

8. Aesthetic and Minimalist Design

• Avoid overloading users with unnecessary information

9. Recognition, Diagnosis, and Recovery from Errors

• Clear error messages without technical jargon

10. Help and Documentation

• Provide ample resources that are easily accessible and relevant

Evaluation Prioritization

Rating Issues

• Problems should be rated based on frequency and impact to prioritize corrective action

Heuristic Evaluation Process Continuation

Obtain Feedback

• Experts should combine findings to build a comprehensive issue map

Limitations of Expert Reviews

• Lack of ecological validity and potential biases

• Discuss the necessity of combining expert reviews with user testing

Observational Testing Approaches

Empirical/User Testing

• Evaluate through observing usage patterns

• Strategies like 'break it' testing, think-aloud methods, and software logging with attention to user privacy

Conclusion

Summary of Content

• Focused on heuristic evaluation and methodologies to catch errors early in usability processes

• Acknowledge that real user feedback is critical in validating the design's effectiveness and adaptability