Quiz 8 - PSYC4191, System Design

Explain an example of a situation where human performance improvement is needed but training is arguably not a good solution?

Epi-Pen, people assume the pointy end is the one where the cap is removed, and stab themselves. High rates of misuse in Epi-Pens, even among medical professionals. The problem is argued to be due to the EpiPen having a counter-intuitive design (people intuitively but incorrectly apply a “pen” analogy), where they assume the pointy end is the one where the cap has been removed.

Give two reasons why training may be a weak solution to the misuse of EpiPens.

1. These events are rare – so patients are likely to have forgotten any training by the time they need it (training may have been decades earlier).

2. The patient may not be able to self-administer – so the user may be an untrained bystander.

What is the definition of human factors, according to Wickens et al. (1998)?

“The application of psychological and physiological principles to the design and engineering of ((products, processes, and systems.))”

((A scientific discipline that involves studying the interactions between people and the other elements of a “system.”))

Human factors ≈ psychology + engineering, as applied to “systems”

List four examples of the type of system that might be studied by human factors researchers.

• physical device (e.g., laparoscope)
• computer system (e.g., electronic health records)
• workplace (e.g., operating theatre)
• organisation (e.g., hospital)

Can just list the examples here, don’t need to explain how they operate.

What is the key strategy employed by human factors to solve problems (according to Russ et al., 2013)?

Address problems by modifying the design of the system to better aid people.

• Take human capabilities and limitations into account when designing things. Consider capabilities and limitations that are:

  • ((Physical

  • Cognitive

  • Perceptual))

Describe two key goals of human factors.

1. ((Minimising human error))

2. Increasing safety.

Describe three (out of the five) situations when training IS likely to be an appropriate strategy for improving patient safety (according to Russ et al., 2013)?

REWORD:

1. When the goal is to familiarise users with new technologies (including training on strengths and limitations)

2. When it “allows individuals to ((develop and test new techniques or practice)) evidence-based techniques in a safe, low risk environment”

3. When “other system components are considered ((first, redesigned, and addressed using human factors expertise and principles and no other system changes can possibly be made.”))

4. When it allows “individuals to gain experience with specialised techniques that involve sensorimotor skills” (e.g., surgical skills)

5. When used to practice or test knowledge in realistic scenarios.

Describe four (out the five) situations when training is likely to be an inappropriate strategy for improving patient safety (according to Russ et al., 2013)?

1. “The goal is for individuals to stop using technologies, tools or devices in ‘the wrong way’” (blaming the user when maybe the system usability can be improved)

2. “It is an attempt to change innate human characteristics or imperfections” (e.g. telling people to “pay more attention” or “be more careful” is unlikely to improve things)

3. “It is intended to address a type of error that is occurring across multiple people” (suggests the problem is with the system not the individuals using it)

4. ((“Individuals have been previously trained about the safety issue(s) and the problem persists” (suggests training may not be an effective solution)))

5. “Training is the only safety intervention or the primary intervention used, especially when other system components have not been carefully considered and modified first” (i.e., we can’t just assume training is the only solution)

Describe the structure of a system design research program.

Phase 1: understand the task, its problems, and its context

• Task analysis, then

• Heuristic evaluation, then

• User surveys

Phase 2: design the product or system, using iterative feedback (or provide suggested improvements to existing designs)

• Iterative design stage with rapid prototyping

Phase 3: Evaluate the new product

• Experimental studies, ((then))

• Clinical evaluation

List three research methods we can use to understand a system and its problems, as might be used as part of a human factors system design research program

1. Task analysis and cognitive task analysis (Christofidis et al., 2012).

2. Heuristic evaluation of existing and proposed products to identify design problems (Preece et al., 2013). Involves getting panel of experts to look at previous products to identify problems with them.

3. Stakeholder surveys to understand beliefs about the task (Preece et al., 2012). Important to understand this as stakeholder belief may moderate uptake. Need to win them over.

What is a task analysis?

“…a qualitative evaluation of a task in order to understand how it is accomplished.”

What is a cognitive task analysis?

“…a type of task analysis that focuses on understanding the cognitive aspects of the task, such as ((situation assessment, decision making, response planning, and response execution)).”

Give three examples of methods that could be used in a task analysis.

• Review literature (maybe someone has already dissected the task – e.g. in a textbook)

• Interviews with task experts and other relevant stakeholders (e.g. novices, supervisors, those affected by the task outcomes)

• Observational studies (go and watch people doing the task for real)

• Get people to do simulations of the task if appropriate.

What is a heuristic evaluation?

• “…involves a team of experts trained in human factors conducting a qualitative evaluation of existing products against a pre- determined set of design criteria (Preece et al., 2013).”

• Their goal is to identify as many design problems with the products as possible.”

Give three examples of design criteria that might be evaluated in a heuristic evaluation.

1. Information layout
2. Appropriateness of included information
3. Methods of representing decision-making information
4. Use of decision-support systems
5. Effect on cognitive workload and memory

Why might it be important to know what key stakeholders think about a particular task – even if they’re incorrect?

• We can design a better product – but maybe no one will use it because they have some attachment to what they’ve already got.

• This might be for valid reasons (experience, something human factors re-designers haven’t thought of) or invalid reasons (they believe what they have is better even if it can be objectively demonstrated to be worse – or some self-serving vested interest).

• These surveys might help us understand what we need to do to get people to use the product – and what training and support materials might be needed (Preece et al., 2012).

Describe how you might undertake the design phase of creating a new product in order to increase the chances that it will have good usability?

• Consider using the best aspects of pre-existing products
  

• Rapid prototyping:

  • Create a prototype product that can be easily modified

  • Obtain “quick and dirty” feedback on the prototype – and modify it

  • Obtain more feedback – and modify it again

  • Be prepared to go through dozens of design iterations
    before moving on (Horswill et al., 2015)

List three types of study that could be used to evaluate the usability of a product.

1. Objective controlled experiments, comparing performance with the product with other products (is it actually better?)

2. Field studies – does the product lead to better outcomes in the real world

3. Obtaining opinions of stakeholders on the product (does the product have face validity?), using surveys etc (e.g. SECATS).