Human–Computer Interaction: Heuristic Evaluation & Usability Evaluation

Usability Evaluation – Core Ideas

• Goal: Systematically assess and improve the usability of an interactive system.

• Two broad families of approaches

  • Testing (real‐user based): direct observation of representative users; expensive but high ecological validity.

  • Inspection (expert based, or “discounted usability”): experts simulate user behaviour using guidelines, personas, scenarios; cheaper and quicker.

• Timeline keywords often used in literature

  • Exploratory → before design begins (only testing).

  • Predictive → during conceptual design (inspection only).

  • Formative → during implementation (testing + inspection; iterative improvement).

  • Summative → after implementation / before or after release (benchmarking & comparison).

Purpose-Based Typology (4 E/F/P/S)

• Exploratory — “How is / will the product be used?”

• Conducted before interface development.

• Collect statistical summaries, observational data on current practice.

• Predictive — “How good will it be?”

• Uses engineering models of human cognition.

• Abstract, quantitative, approximate; estimated from previous experiments.

• Advantages: $\text{No prototype} \Rightarrow \text{compare alternatives early; no real users required}$; identifies not just what but why usability problems may occur.

• Formative — “How can we make it better?”

• Continuous feedback loop while building the UI.

• Captures process data (qualitative observation of what & why something went wrong).

• Summative — “How good is it?”

• Completed (or near-complete) interface.

• Bottom-line quantitative metrics: task time, success rate, error counts.

The Modified Soup Analogy

• "When the cook tastes other cooks’ soups ➜ exploratory."

• "When the cook assesses a recipe ➜ predictive."

• "When the cook tastes while cooking ➜ formative."

• "When the guests/critics taste ➜ summative."

Method-Based Typology

Additional split

• Design-oriented (early): heuristic inspection, cognitive walkthrough.

• Implementation-oriented (later): observation, interview/survey, controlled experiment.

Heuristic Evaluation (Nielsen)

• Definition: Small group of experts evaluate an interface against a set of recognized usability heuristics.

• Suitability: Early design stages when any form of prototype (even paper) exists. Quality is bounded by evaluator expertise.

Three-Phase Process

1. Briefing session

   • Evaluators + project manager.

   • Share identical information: goals, prototype, personas, scenarios.

2. Independent evaluation

   • Each expert works alone for $1\text{–}2\,\text{h}$.

   • Two passes through UI
     1) Global flow, scope, navigation
     2) Detailed element‐by‐element check against heuristics.

   • Notes & screen-shots captured.

3. Debriefing session

   • Aggregate & discuss findings, prioritise, propose fixes.

   • Deliverables: problem list, severity ratings, suggested solutions.

Number of Evaluators vs. Problems Found

• Empirical curve (Nielsen): ≈$5$ evaluators uncover ≈$75\%$ of issues; diminishing returns after $10$.

Severity Scale (0–4)

Averaging ratings from $3\text{–}5$ evaluators yields stable priorities.

Nielsen’s Ten Usability Heuristics (with Examples)

1 Visibility of System Status – “Provide Feedback”

• Users know what is happening.

• Response-time guidelines
  • <0.1\,\text{s} ⇒ perceived instantaneous • $1\,\text{s}$ ⇒ flow intact but delay noticed • $10\,\text{s}$ ⇒ attention drifts; provide progress indicator • >10\,\text{s} ⇒ user will multitask; supply cancel or background.

• Short delays: busy cursor; long: percent-done + time remaining.

2 Match Between System & Real World – “Speak Users’ Language”

• Terminology, icons, metaphors map to user mental model.

• Avoid cryptic messages (e.g., "Rstrd Info").

• Use task language ("Withdraw cash" instead of "Debit transaction").

3 User Control & Freedom – “Clearly Marked Exits”

• Undo/redo, Cancel, Escape, Interrupt, Quit, Defaults.

• Prevent feeling trapped (e.g., $20\,\text{s}$ countdown with no exit ⇒ bad).

4 Consistency & Standards – “Be Consistent”

• Internal, external, metaphorical consistency.

• Same widgets look & act the same; identical commands yield identical effects.

5 Recognition Rather Than Recall – “Minimize Memory Load”

• Visible menus/icons instead of memorised commands.

• Provide examples, default values, tooltips, calendars.

• Short-term memory ~$7\pm2$ items; UI should not exceed.

6 Flexibility & Efficiency – “Provide Shortcuts”

• Accelerators unseen by novices: $\text{Ctrl+C}$, macros, type-ahead, MRU lists.

• Web: $\approx60\%$ of page visits are revisits; history/jumps save time.

7 Aesthetic & Minimalist Design – “Keep It Simple”

• Every extra element competes with relevant information.

• Present exactly what user needs, in natural order; cluster related info; avoid unnecessary modes/windows.

8 Error Prevention

• Design so errors cannot occur: constrained widgets, validity checks (
  e.g., SSN mask), confirmation before destructive acts (Format $C:$).

9 Help Users Recognise, Diagnose & Recover – “Good Error Messages”

• Plain language, precise, constructive fix.

• Defensive phrasing (no blame); multilevel; link to help.

• Example: Error: Cannot create story card before creating backlog. ➜ Offer button “Create backlog”.

10 Help & Documentation

• Necessary when discoverability insufficient.

• Should be searchable, task-focused, step-oriented, concise.

Types of Help

1. Tutorial / Getting Started – conceptual intro & exploration.

2. Reference Manual – exhaustive lookup (experts).

3. Reminders – quick cards, tooltips.

4. Wizards – task walkthrough; risky if user diverges.

5. Tips – context-aware suggestions, learning path.

Chunking & Alternative Principle Sets

• "Meet expectations" chunk: Visibility, Real-World Match, Consistency, Help.

• "User is Boss" chunk: Control/Freedom, Flexibility/Efficiency.

• "Handle Errors" chunk: Prevention + Recovery.

• "Keep it Simple" chunk: Aesthetics/Minimalism.

Other well-known sets
• Shneiderman’s 8 Golden Rules: consistency; shortcuts; informative feedback; dialog closure; error prevention/handling; easy reversal; locus of control; memory load.
• Norman’s 7 Principles: knowledge in world & head, task simplification, visibility, correct mapping, constraints, design for error, standardize.

Performing a Full Heuristic Evaluation – Practical Checklist

1. Obtain design artefact (verbal description ⇢ paper mock-up ⇢ prototype ⇢ running system).

   • For paper mock-ups, watch for missing dialogue elements.

2. Prepare materials

   • Persona & scenario brief.

   • Heuristic checklist (+ any domain-specific nuances).

   • Logging template / spreadsheet.

3. Individual review (each evaluator)

   • Pass 1: global look & navigation.

   • Pass 2: itemised scrutiny vs. heuristics.

   • Capture screenshots; annotate issues & positive findings.

4. Aggregation

   • Evaluation manager merges independent lists into master sheet (spreadsheet recommended).

   • Columns: Serial #, Problem, Violated Heuristic(s), Suggestion, Severity.

5. Severity rating

   • Distribute master list back to evaluators; each rates $0$–$4$ alone.

   • Compute mean per issue; sort descending.

6. Reporting

   • Prioritised fix list, rationale, quick wins vs. high-cost fixes.

   • Include illustrative screenshots & examples.

Example Spreadsheet Skeleton (adapt, extend)

Severity key reproduced earlier (0–4). Use to prioritise remediation.

Advantages & Limitations of Predictive Evaluation (Model-Based)

• No UI prototype required → compare many alternatives quickly.

• No real users needed → cheaper; early identification.

• Theory explains why a problem arises ⇢ targeted redesign.
  − Abstract & approximate → may overlook context-specific nuances.
  − Requires validated cognitive models & analyst expertise.

Additional Resources (Links)

• usabilitygeek.com/heuristic-evaluation-introduction

• uxgorilla.com/heuristic-evaluation

• prototypr.io/10-usability-heuristics-with-examples-4a81ada920c

• UX case study example: uxdesign.cc/…yuppiechef-com-c92098052ce4

(End of study notes)