M8 - M11: Performance, Issue-Baased, Self-Reported, and Behavioral Metrics

true

In collecting task time, we need to log the start and end time of performing a task.

Task success

It is the most widely used performance metric.

true

Another way to analyze usability issue is to get the number of issues per participant should decrease per design iteration.

true

Social desirability bias is a tendency to provide more positive feedback in person or on the phone.

false

We do not need to identify the phenomenon like being happy because it is already obvious and common to us.

false

Giving partial points to participants in completing their tasks is under binary success.

Efficiency

Defining the when is the start and end of action are under what performance metrics.

true

Different moderators will also influence the issues that are observed.

false

Ease of use gives participant a reference for good and bad design before performing a task.

true

Capturing brain signals allows us to understand cognitive behavior during interaction.

Efficiency

It is the amount of effort required to complete a task.

true

Performance metrics are used to determine the magnitude of a specific usability issue.

true

Issues can also be analyzed at a task level.

true

Post-study ratings are the most common way to collect self-reported data.

false

We do not need to operationalize the identified phenomenon for human behavior.

true

The level of success considers how close the participant is in completing their task.

errors

Incorrect action on the part of the user.

Low severity

It results in annoyance or frustration but does not lead to task failure.

true

Likert scale allows the participant to rate a statement based on the level of agreement he or she experienced.

true

Microexpressions are facial expressions that occur within 1/25th of a second.

true

Learnability measurement is important in a continuous learning setup.

Task success

It measures how effectively users are able to complete a given set of tasks.

true

Bias on participant exists since they have different level of knowledge, one issue from a participant may not be an issue to another.

Design a sampling technique

Getting several participants to be observe is under \_______________.

Validate

Define/operationalize the phenomenon

Design a sampling technique

Identify the phenomenon

true

The most common way to analyze and present binary success rates is by individual task.

false

High severity is any issue that directly leads to task failure.

true

Self-reported measures are measures in which respondents are asked to report directly on their own behaviors, beliefs, attitudes, or intentions.

true

The oral method is the easiest method for quick ratings.

true

Validating collected behavioral metrics can be changeling for biometrics.

true

Observers may have different observations.

EFFICIENCY

\__________________ can be a ratio of task completion rate to mean time on task

true

Binary success is appropriate to use when the success of the product depends on users completing a task or a set of tasks.

LOW GRANULARITY

It is good for examining the overall design or organization but not good for specifics.

HIGH GRANULARITY

It is good for identifying very specific problems but not good for big picture thinking.

EASE OF USE

It is the most common type of post-task ratings.

true

Behaviors can be verbal and non-verbal.

FALSE

NONVERBAL BEHAVIORS ARE HARDER

Verbal behaviors are harder to observe that non-verbal.

true

It is a best practice not only to report issues but also the findings.\

true

To analyze self-reported data, we can assign numeric value and compute for its average.

false

We cannot user self-report to assess specific attributes on the product.

true

We can analyze behaviors base on the participant comments.

false

Another common way of looking at binary success is by user or type of user.

AUTOMATED STUDIES

Collecting a verbatim comment is to require the participant to provide a comment at the conclusion of each task.

false

We do not need rules in turning off or on the clock in measuring time on task.

true

We can simply examine each issue and then categorize it into a type of issue.

ORAL

It is the easiest method for quick ratings.

OPEN ENDED

Assessing a product by asking the participant on what you like most do is an example of \____________.

true

Capturing brain signals allows us to understand cognitive behavior during interaction.

true

Microexpressions are facial expressions that occur within 1/25th of a second.

true

Trial is a test of the performance, qualities, or suitability of someone or something.

true

Issues should be specific and actionable.

true

Not everyone believes in severity ratings.

true

Not all behavior can be captured by our naked eye, with this, we need tools to be able to capture them.

true

Post-task ratings are the most common way to collect self-reported data.

true

Task failure and error might look the same.

LOW GRANULARITY

It is good for identifying very specific problems but not good for big picture thinking.

High granularity

Impact on user experience

Low granularity

Multidimensional

true

Making a survey anonymous may avoid bias in self-reported metrics.

true

Checking whether certain details stood out is an example of awareness and comprehension.

false

These facial expressions are universal, meaning they occur on everyone around the world.

Levels of Success

The participant receives some value from partially completing a task.

High severity

Any issue that directly leads to task failure.

false

The Likert scale involves presenting pairs of bipolar, or opposite, adjectives at either end of a series of scales.

Expectation Measure

This type of post-task ratings assesses the participants whether how easy or difficult a task is in comparison to how they thought it is.

true

To collect behavioral metrics, first we need to identify the phenomenon.

45179

45179

true

Binary success is the simplest and most common way of measuring task success.

true

Usability issues are used to improve the product.

Usability magnitude estimation

It gives the participant a reference to the good and bad design before performing a task.

Post-task

Collecting data after each task may help us determine problematic tasks and parts of the product.

false

Verbal behaviors are harder to observe that non-verbal.

false

Learnability is a test of the performance, qualities, or suitability of someone or something.

false

A large impact on user experience and many users experiencing the problem will lead to medium severity.

false

To avoid bias, the person collecting self-report can also be the test monitor.

true

Checking whether certain details stood out is an example of awareness and usefulness.

true

Motion is one of the behaviors that need to have equipment while to capture.

true

An error opportunity is basically a chance to make a mistake.

true

One of the ways to analyze the usability issue is to count the frequency of unique issues.

Oral

One of its disadvantages is that the observer must take notes.

Rating Scale

Online

Written

Oral

true

We may user self-report to assess specific attributes on the product.

FALSE

LIKERT

Semantic differential allows the participant to rate a statement based on the level of agreement he or she experienced.

true

Asking the participant if he or she is aware of a certain functionality about the product is an example of awareness and usefulness.

true

Body language can also show the behavior of a participant towards the product.

response bias

It is a tendency to provide more positive feedback in person or on the phone.

true

Anyone who uses technology interacts with some type of interface to accomplish his or her goal.

true

A small impact on user experience with few users experiencing the problem will give us low severity ratings.

false

One of the issues in considering time data is when do you need to report all tasks or just the successful ones.

true

You should measure errors when an error will result in significant loss of efficiency.

FALSE

MEDIUM SEVERITY

Low severity of user experience contributes but does not directly prevent failure.

Granularity

It pertains to how detailed an issue is.

After-scenario questionnaire

It is answered by the participant after a certain situation.

true

Not all behavior can be captured by our naked eye, with this, we need tools to be able to capture them.

true

To measure efficiency, we need to identify the actions to be measured.

true

According to task bias, the nature of the prototype or product you are evaluating has a huge impact on your findings.

false

Brain signals can be observed by our naked eye.

Time on task

It measures the time it takes to perform a task.

real issue

If a problem is likely to be repeatable with a larger population, then it is considered as \______________.

true

Not seeing something that should be noticed is an example of an issue.

true

The semantic differential technique involves presenting pairs of bipolar, or opposite, adjectives at either end of a series of scales.

true

Sampling allows you to choose the number of your participant.

true

In levels of success, the participant is either will complete a task successfully or not.