EECS 3461 - Week 8A: Expert Evaluation

Week 8A

What are the steps for the interaction design process (DraPE)?

1. Discovering Requirements

2. Designing Alternatives

3. Prototyping

4. Evaluating

What are the 3 Evaluation methods? (SUE)

Stand-Alone Critiques - designers evaluate their own work

Usability Tests - real users test the system

Expert Reviews - usability experts evaluate the system

What is the definition of Expert Evaluation?

An inspection method where usability experts evaluate a system using usability principles or heuristics

What are the benefits of Expert Evaluation?

Cost-effective

Independent

Early detection

Saving of time and money

List the purpose of expert evaluation at each stage

To identify potential usability issues

To assess the effectiveness of the design changes

To identify areas for improvement

To conduct a final assessment

For future product updates

What are the two expert evaluation types?

Heuristics and Cognitive Walkthrough

When can a Heuristic Evaluation be done in the design process?

At any stage

Who developed the Heuristic Evaluation?

Jacob Nielson and Rolf Molich

What is the Heuristic Evaluation used and not used for?

Used for identifying usability problems in prototypes/designs

Not used for coming up with new design solutions

What are the disadvantages of Heuristic Evaluation?

It may reinforce existing design assumptions and it is not always consistent between evaluators

What are the steps in Heuristic Evaluation?

Select 3–5 evaluators, brief them with context and process, independent evaluation, consolidation

Which step in the heuristic evaluation produces a list of justified, rated problems and involves going through the interface at least twice?

Independent Evaluation

Which step in the heuristic evaluation sets the scene for the evaluation by explaining the target users, what users are expected to know, the context, and what tasks are most interested?

Briefing session

What are the Heuristic Evaluation (Interface-Based) Steps? (Hint: SIR)

Select a set of principles or guidelines, inspect design against each principle, and record where design violates each principle

What are the 10 of Neilsen’s Heuristics?

1. Visibility of system status

2. Match Between the System and the Real World

3. User Control and Freedom

4. Consistency and Standards

5. Error Prevention

6. Recognition Rather than Recall

7. Flexibility and Efficiency of Use

8. Aesthetic and Minimalist Design

9. Help Users Recognize, Diagnose, and Recover from Errors

10. Help and Documentation

Which of the 10 of Neilsen’s heuristics explains that the system should always keep users informed about what’s going on through appropriate feedback?

Visibility of system status

Which of the 10 of Neilsen’s heuristics explains that the system should speak the users’ language, with words, phrases, and concepts familiar to the user?

Match between the system and the real world

Which of the 10 of Neilsen’s heuristics explains that error messages should be expressed in plain language (no codes), precisely indicating the problem, while suggesting a solution?

Help users recognize, diagnose, and recover from error

Which of the 10 of Neilsen’s heuristics suggests making objects, actions, and options visible, while explaining that the user should not have to remember information from one part of the dialogue to another?

Recognition rather than recall

Which of the 10 of Neilsen’s heuristics explains that the system should allow users to tailer frequent actions with alternative interaction styles such as function keys?

Flexibility and efficiency of use

What is a Cognitive Walkthrough?

An evaluation where experts step through a prototype simulating user problem-solving at each step

What is a Cognitive Walkthrough used and not used for?

Used for assessing ease of learning for new users by identifying detailed problems; not used for evaluating performance of highly skilled or frequently performed tasks

At what phase can a Cognitive Walkthrough be done, and what is the evaluation group?

It can be done at any phase after detailed UI specifications exist, and the evaluation group is usually design or development team members acting as evaluators

What are the four roles in a Cognitive Walkthrough? (FPUS)

Facilitator

Presenter

User

Scribe

What are the components in a Cognitive Walkthrough? (PETSQ)

Prototype, Evaluators (Users), Tasks, Scenario, Questions

What are the Scenario Components for CW/Usability Testing? (GOATS)

Goal, Outcomes, Actions, Tasks

What are the steps for Cognitive Walkthrough (Interaction-Based)? (Hint: De CAR)

Define a task, Convert task into a process, Ask a set of questions per step, Record issues found

Between Heuristic Evaluation and Cognitive Walkthrough, which of the two is provides a flexible and informal assessment?

Heuristic Evaluation

Between Heuristic Evaluation and Cognitive Walkthrough, which is best suited for interaction design?

Cognitive Walkthrough

What is the aim of a Heuristic Evaluation and Cognitive Walkthrough?

The aim for a HE is to see whether the system interface design meets a set of principles, while the aim for a CW is to determine whether a new user can carry out specified tasks