UX-Study

Why do UX evaluations?

- Verifies if we met the requirements
- Saves on the cost of fixing defects
- Tests assumptions
- Bad experiences -\> low customer satisfaction -\> low sales -\> damaged reputation

What is a Mental Model?

What the user believes about the system at hand.

Belief vs facts.

Actions based on predictions.

User success depends on how well the designer has communicated the systems purpose.

What is a Conceptual Model?

Every software application has an underlying conceptual model: How data, functions, tasks, roles and members of a group are organised in a way that is expected and can be easily understood by the user.

What happens when there is a 'good match' between the software's conceptual model and the user's mental model?

The software is usable and intuitive.

What happens when there is a 'mismatch' between the software's conceptual model and the user's mental model?

Creates cognitive friction for the user.

What are the three main settings for evaluation methods?

Controlled settings (labs)

Natural Settings (real world)

Without users

What are some different evaluation methods?

- Observing (experiments & observations)
- Asking users (focus group & accessibility testing)
- Asking experts (field study & heuristic evaluation)
- Testing (cognitive walkthrough)
- Modeling (GOMS)

What are the UI Heuristics?

1. Visibility of system status
2. Match between system and the real world
3. User control & freedom
4. Consistency & standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility & efficiency of use
8. Aesthetic & minimalist design
9. Help users recover
10. Help & documentation

What is Visibility of System Status?

Tells the user what's going on, e.g. a loading bar.

Helps users feel in control, trust the system and take appropriate action.

What is Match the Real World?

Use the user's language -\> this helps when they search.

Use natural mappings and associations.

E.g. for volume buttons, the one on top is the 'louder' button.

What is User Control and Freedom?

User should be able to quickly correct mistakes or backtrack choices.
- encourages exploration
- increases use & sales

E.g. back, cancel, close links and undo & redo options.

What is Consistency & Standard?

Use conventions.

E.g. CTRL+C CTRL+V

E.g. NZ dates vs American dates

E.g. Radio buttons vs Checkboxes

What is Error Prevention?

Avoid unconscious slips -\> when users intend to perform one action but end up doing another (often similar) action.

Mistake prevention is better than cure.
- Don't allow incorrect actions
- Confirmation steps
- Remove error prone situation

What is Recognition Rather than Recall?

Use objects/images rather than long text.

E.g. you can recognise someones face but don't know their name.

E.g. GUI vs Command Line

What is Flexibility and Efficiency?

Consider novices and experts.
- Provide shortcuts for experts
- Make the system learnable for novices

Allow more than one way to achieve a task.
Allow for personalisation and customisation.

What is Aesthetic & Minimalist Design?

- Signal vs Noise
- Make a good first impression
- More memorable experience
- Brand identity

What is Help Users Recover?

Have good error messages.
Clear actions on buttons and suggest solutions.

E.g. wrong passwords -\> Your username and/or password do not match.
Rather than -\> You cannot log in.

What is Help & Documentation?

Few people will read, but when stuck it becomes essential.

What are the two Discount (less work) usability evaluations?

Heuristic Evaluation - Use UI guidelines to assess a system

&

Cognitive Walkthrough - Pretend to be a user/persona and step through the system

How does Heuristic Evaluation work?

Process where 'experts' use rules of thumb to measure the usability of user interfaces using established heuristics, and can reveal insights that can help design teams enhance product usability from early in development.

Can test:
- Paper prototypes
- Working software
- Mobile
- Desktop

What are some pros of Heuristic Evaluation?

- Evaluators can focus directly on specific issues.
- Evaluations can pinpoint faults with individual elements early on and determine their impact on the overall UX
- Compared with testing on users you can get feedback without the ethical and practical dimensions and costs.
- You can combine it with usability testing.
- With the right heuristics, evaluations can help flag detailed issues and lead the way to optimal solutions.

What are some cons of Heuristic Evaluation?

- Evaluators can mark "issues" that aren't actually usability problems.
- You have to choose exactly the right heuristics (and number of them) to make sure evaluators find all usability issues.
- It can be hard and/or expensive to find evaluators who are experts in a certain industry (e.g. banking) and usability.
- As you need several expert evaluators, you may find it easier (and cheaper) to stick to usability testing.
- Its subjective: findings can lack proof and be biased.

What is good practice for the Heuristic evaluation steps?

Decide which heuristics to evaluate.
Choose appropriate evaluators.
Agree on a severity scale.
Use a template.
Debrief after the session.

What are some Mobile-specific heuristics?

Efficiency of Use and Performance: The device must be able to load and display info in a reasonable amount of time and minimise the steps required to perform a task.

Pleasant and Respectful Interaction with the User: The user should not feel uncomfortable while using the application.

Privacy: The application must protect the users sensitive data.

Who is a "double expert"?

Usability experts AND from the relevant domain (finance, education, entertainment etc.) - not end users.

How do you agree on a Severity Scale?

Frequency: Is it common or rare?
Impact: Will it be easy or difficult for the users to overcome?
Persistence: Is it a one-time problem or a repeated one?

Can use a number scale to represent this.

What are the five stages of design thinking?

Empathise, Define, Ideate, Prototype, Test

What occurs in the Empathise Stage?

Understanding user's needs, motivations, and pain points.

What are some methods used in the Empathise Stage?

Interviews, observations and surveys.

What are some outputs from the Empathise Stage?

Empathy maps, personas, user journey maps.

What is involved in the Define Stage?

Analysing and synthesizing user insights.

Crafting problem statements: "How might we...?"

Defining project scope, goals, and success criteria.

What is involved in the Ideate Stage?

Brainstorming creative solutions addressing user needs.

Encouraging open-mindness, collaboration and divergent thinking.

What are some techniques used in the Ideate stage?

Mind mapping
Affinity diagrams
SCAMPER

What does SCAMPER stand for?

Substitute, combine, adapt, modify, put to other uses, eliminate, rearrange

What does it mean by 'divergence and convergence' in the Ideate stage?

Divergence \= remove all constraints, brainstorm like crazy!

Convergence \= start adding back constraints and see what is possible

What is involved in the Prototype Stage?

Building low-fidelity, quick and cheap prototypes to explore ideas and gather feedback.

What are examples of tools used in the Prototype stage?

Sketch, Figma and Adobe XD

What occurs in the Test Stage?

Evaluating prototypes with real users.

Usability testing methods: moderated and unmoderated testing, A/B testing

Iterating and refining designs based on user feedback and insights

When creating personas to evaluate your app, what things do you want to avoid?

- "The Elastic user," i.e. they are okay with anything, agree with everything
- "Self-referential design," i.e. assuming that the end users are themselves
- "Edge cases," i.e. target the most important users (however this has slightly changed over time, e.g. disabilities)

What does 'Metonymy' metaphor mean?

Refers to something by using a sub-part.

E.g. when we are talking about 'The Crown', we know we are talking about the Monarch even though we are only mentioning an object associated with this.

What does 'Orientational' metaphor refer to?

Maps interface concept to a spatial concept.

E.g. "To the right is more"
"Down is less"

What does the 'Structural' metaphor refer to?

Maps abstract system concept to real-world concept.

E.g. Deleting uses a waste bin as its icon

What does the 'Ontological' metaphor refer to?

Maps system concept to basic category of existence.

"Application is an entity" - e.g. the Duolingo Bird

What are the first three categorising scenarios in User Modeling?

As-is
Vision
Evaluation

What does an As-is scenario represent?

Captures existing user behaviour, based on research and data.

E.g. Maria, a third-year university student, currently uses a mix of textbooks and YouTube videos to learn Spanish

What does a Vision scenario represent?

Depict how users might interact with a proposed system or feature.

E.g. Maria discovers a new language learning app that offers interactive lessons and daily practice exercises. She starts using it regularly.

What does an Evaluation scenario represent?

Provide context for usability testing, allowing users to complete tasks in a controlled, measurable way.

E.g. In a usability test, Maria attempts to use the new apps chatbot to practice Spanish conversation

What are the second three scenarios for User Modeling?

Context
Key-path
Validation

What does a Context scenario represent?

Black box design

Focus: goals, not interface specifics

Captures the environment, day in the life aspect of how the system interacts with the personas life

What does the Key-path scenario represent?

Description of interaction with the system.

Uses 'design' vocabulary

More system detail than context scenarios

What does the Validation scenario represent?

"what if" scenarios, used to validate design

Typically not core tasks like key-path scenarios

E.g. edge cases, infrequent usage

What does GOMS stand for?

Goals - "State of affairs to be achieved"
Operators - "Elementary perceptual, motor or cognitive steps"
Methods - "Conditional sequence of goals and operators" to achieve a goal
Selection Rules - Multiple methods per goal?

What are the 'two parts' of User Experience (UX)?

1. A person's perceptions and responses
2. The use of a product, system of service

What are the three different types of mobile users?

Microtasking: When the user interacts with their device for brief but frenzied periods of activity
Local: When the user wants to know whats going on around them
Bored: When the user has nothing better to do and is looking to be entertained or otherwise diverted.

What are the four steps of user attention?

Scan -\> Focus -\> Remember (short term) -\> Remember (long term)

What are the different types of scanning patterns?

- Z-Pattern scanning
- F-Pattern scanning

How many evaluators would you ideally have for heuristic evaluation?

Not 3-4 because they get the most errors (~60%)

Ideally more like 7 or 8, however number of testers relates to the complexity of the interface.

What two categories does Heuristic Evaluation come under in evaluation methods?

Without users & Asking experts

What two categories does Cognitive Walkthrough come under in evaluation methods?

Controlled settings (labs) & Testing

What are the 4 questions you want to answer during cognitive walkthrough?

- Will the user try and achieve the right outcome?
- Will the user notice that the correct action is available to them?
- Will the user associate the correct action with the outcome they expect to achieve?
- If the correct action is performed, will the user see that progress is being made towards their intended outcome?

What are the three Cognitive Walkthrough Roles?

Facilitator - keep on task
Recorder - record everything
Evaluator - who is playing the role of the user

What are some examples of evaluators that you would want for Cognitive Walkthrough?

Product experts - intimate knowledge of the product design e.g. UX designer or Product owner

UX practitioners - knowledge in human-computer interaction or cognitive science

Engineers - who are building the system

Domain experts - knowledge of the realm in which the product operates, e.g. finance, insurance, education

How many evaluators are recommended for Cognitive Walkthrough?

Depends on how many different roles are involved.

Neils Nelson group recommend 2-6 people.

How to conduct action sequences when role playing the user?

Start with the happy path (correct sequence of actions), then think about where the user may fail and why.

What are the best practices for Cognitive Walkthrough?

- Gives a perspective on how new/infrequent users will interact with the system
- Acting as the imagined user/persona is key
- Focus on how quickly/easily tasks are achieved
- Stick to the 4 questions
- Tool for developing the system, not for validating, especially useful in the early stages of development.

What are the four evaluation methods that occur in controlled settings (labs)?

- Experiments
- Focus group (asking users)
- Focus group (asking experts)
- Cognitive Walkthrough

How do you define your testing goals for user testing?

1. List tasks that users must be able to do
E.g. Top task: Apply for a bank account
- Fill in application form
- Verify identity
- Be sure the application process is complete
- Understand when the account is active

2. Are we 'concerned' about any of the above?
E.g.
Goal 1: Discover where people need instructions
Goal 2: Determine the best location to link to the application form
Goal 3: The application process must take less than 30s

For user testing, who are your ideal study participants?

Characteristics of study participants should match those of your ideal end users. E.g:
- Level of experience with computers/devices
- Level of experience with interface
- Quality of domain knowledge
- Experience with similar software
- Demographic (age, location, income etc.)

What are some of the different measurements (performance) you can use to evaluate the user testing?

Successful Task Completion:
- Measure from start to finish
- Measure from start to where the first error occurred

Error rate:
- Critical errors (unable to finish the task)
- Non critical errors (able to recover and finish the tasks)

Task on Time:
- The amount of time it takes the participants to complete the task.

What are some of the different measurements (subjective) you can use to evaluate the user testing?

Levels of satisfaction, ease of use, perceived complexity etc.

Likes, Dislikes and Recommendations

What are the different types of verbal data when user testing?

Think Aloud & Probing

How does Think Aloud work?

- Participants verbalise thoughts and intentions while interacting with the system.
- During the sessions: to get a running stream of consciousness as they work
- After the session: participant retraces their steps when the session is complete

How does Probing work?

When participants do or say something interesting or unique, the moderator asks follow-up questions: "Why did you click here?"

What are different pieces of equipment that can be used during user testing?

- Note paper & pen (or digital versions)
- Camera
- Voice/video recorder
- Eye tracking software
- Hotjar to generate heat maps
- Logging software

What are the relevant ethical codes?

- The Designers Institute of NZ Code of Ethics
- PMI Code of Ethics & Professional Conduct
- NZ Institute of IT Professionals Code of Ethics
- AMC Code of Ethics
- IEEE Code of Ethics

What do you include in your user test report?

Report your quantitative data (measures) and qualitative data (verbal data)
- Include positive and negative findings
- Rank the problems in order of severity
- Include images and/or video

What are the Best Practices for User Testing?

- Do a pilot study before 'real' user testing
- Treat participants with respect and make them feel comfortable
- Take good notes
- Take both performance and subjective measures, these do not always match.

What are the four type of evaluation methods that occur in Natural Settings (real world)?

Observations
Accessibility testing
Field study (Asking experts)
Field study (Testing)

What are some characteristics of testing in Natural Settings?

- Little to no control
- Good for learning the unexpected and understanding real challenges
- What do users actually do with our software
- Places: homes, workplace, street etc.
- Less suitable for performance measures, good for subjective measures

How can you collect data when conducting a Natural Setting study?

- Observations of the participants (take notes/video recording)
- Interviews with the participants (voice/video recording)
- Survey with the participants (digital or paper forms)

What are some ways you can 'observe' participants?

- Users can show you what they normally do
- Body language, gestures, what they're not doing

What are some good practices when interviewing participants?

Users can tell you what they need
Ask open-ended questions (where the answers are not 'Yes' or 'No')

What is a good interview structure to follow?

- Start with a warm-up
- Allow flexibility to explore topics that are important to the user
- End with telling the participant what you're doing next

What is an experiment?

Need to have a Hypothesis

Needs to be Testable

Needs to be Dependant and have independent variables

What are conditions in an experiment?

E.g. Do more users make purchases if the checkout button is blue or red?

2 Conditions:
- Control: blue button
- Treatment condition: red button

What are variables in an experiment?

E.g. Do more users make purchases if the checkout button is blue or red?

- Independent Variable (manipulated by the experimenter, participant can't affect it): checkout button colour
- Dependent variable (outcome/behaviour): number of users making purchases
- Need at least one independent variable (can have more that one)

What is replication in an experiment?

Ensure the same test conditions for all participants
- Tasks
- Device used
- Feedback - text, verbal, vibrations etc.
- Facilitator behaviour - what questions they ask, how they treat the participant etc.

How is statistics used in an experiment?

Need to know that your results are not due to randomness.
- Collect data
- Accept of reject the null hypothesis

H0 \= there is no difference in the number of users making purchases
h1 \= more users will make purchases with the red button

What are the differences between Progressive Web Apps (PWA) and Native apps?

PWA:
- Installable website
- Runs in a browser
- Can access (some) device hardware
- HTML, CSS, JavaScript, Web frameworks (e.g. React)

Native App:
- Installed from an app store
- Can access all device-specific capabilities
- Objective-C and Swift

What is the HTML DOM?

The HTML Document Object Model (DOM) represents the structure of an HTML document as a tree-like structure, where each element, attribute, and text node in the document is represented as an object.

How does React.js speed up performance?

Updates to DOM are slow, have to reload the whole tree.

React separates the 'virtual DOM' from the 'real DOM', minimises the impacts of changes.

What is a React component?

Components are the building blocks of the UI

A React component is a JavaScript function that returns HTML

Components can be as small as a button, or as large as an entire page

Components can be composed of other components

What is JSX?

JSX is a syntax extension for JavaScript that lets you write HTML-like markup inside a JavaScript file

How does the Ionic framework help?

Provides over 100+ prebuilt components.

Includes animations/transitions.

Optimises the build.

What is Ionic?

"Open source app development toolkit for building modern, fast, top-quality cross-platform native and Progressive Web Apps from a single codebase with JavaScript and the Web."

What are Event Handlers?

Run in response to (user-generated) interactions with the system.
E.g. the IonButton component defined in the React library: onClick

What are Props (aka Properties)?

Special react objects that are used to pass data between components.

Data flows one way ONLY
- From parent to child

What two things need to happen in order to update a react component with new data?

1. Retain the data between renders
2. Trigger React to render the component with new data (re-rendering)

React has built in features to manage state.

How do you integrate Design Thinking with Agile Development?

Incorporating design thinking into agile processes: design sprints, user story mapping, iterative releases

What is a design sprint?

Time-boxed, focused efforts to create and test design solutions.