SYST 469 - Exam 3

Describe Likert scales and use example?

Describe Likert scales and use example?

• It measures opinions, attitudes, or beliefs using a range of options.

• For example, to evaluate website colors:

  • "The colors on this website are appealing."

    • 1 = Strongly Disagree

    • 2 = Disagree

    • 3 = Neutral

    • 4 = Agree

    • 5 = Strongly Agree.

• It helps gauge satisfaction or preferences.

Describe semantic differential scales and use example?

• is a survey tool used to measure people's attitudes, opinions, or feelings about a topic by presenting them with a set of opposing adjectives or phrases. Respondents rate their feelings on a scale between these opposites.

• For example:
  Imagine a survey about a new smartphone. One question could be:

  • How would you describe the design?

    • Ugly 1 2 3 4 5 6 7 Beautiful

• In this case, respondents choose a number to show where their opinion falls between "ugly" and "beautiful." These scales help gather nuanced opinions about a product, service, or idea.

When, in general, should you tend to have a small number of scale points (e.g., 3), a medium-size number (e.g., 5), and a larger number (e.g., a 7- or 9-point scale)?

• Use small number when you have limited possibilities (yes, no, DK)

• Use a medium-size number (e.g., 5) when making judgements involving like/dislike, agree/disagree statements

• Use longer range (e.g., 7 or 9) when asking for subtle/precise judgments (e.g., regarding user experience goals)

What is a pilot study (also called a pilot test) and why would you conduct one?

A small trial run to find and fix problems before doing the actual study. The aim is to make sure your evaluation works as planned. The pilot study checks the follow:

• You can conduct the procedures as planned

• interview scripts, questionaires, experiments, usability test, and user experience test works approriately

Describe what five of the six letters stand for in DECIDE?

• Determine the high-level evaluation goals

• Explore specific questions addressing goals

• Choose the evaluation methods to answer those questions and, in turn, the goals

• Identify practical issues in performing evaluation

• Decide how to deal with the ethical issues

Describe the iPad usability test. Make sure to describe one of the questions they were trying to answer, the participants, the test procedures, and the results (i.e., the answer to the question with an example).

• Goal:

  • aimed to evaluate how users performed and preferred interacting with general-purpose internet websites versus iPad-specific apps. The study was conducted in a controlled experimental setting.

• Question:

  • Do user prefer iPad specific apps to general, non-iPad specific websites?

    • Previous study with iPhone showed people preferred using apps than browsing the web because websites were slow using the iPhone.

    • Are user preferences and performance the same for iPad?

• Participants:

  • Total Number: 7 participants.

  • Demographics:

    • Ages ranged from 20s to 60s.

    • Various occupations, such as food servers and paralegals.

    • Three participants were male, and four were female.

  • Experience:

    • All were experienced iPhone users (owned one for at least three months).

    • One participant also owned an iPad.

    • All participants had experience using the web and apps with similar interaction styles.

• Test procedure:

  • Stage 1:

    • Participants explored the iPad independently, using any apps or features they found interesting.

    • They verbalized their thoughts (think-aloud protocol) about likes and dislikes.

    • The moderator recorded observations and video of their interactions.

  • Stage 2:

    • Participants were assigned tasks involving both iPad-specific apps and general websites.

    • Tasks were randomized to avoid order effects, and participants sometimes performed the same task in both conditions (apps vs. websites).

    • Examples of tasks included finding specific information, navigating content, and interacting with features.

• Results:

  • Main Finding: Participants overwhelmingly preferred iPad-specific apps over general websites.

  • Examples:

    • General Websites: Links were often too small to tap reliably, and fonts were difficult to read. Users also struggled to locate navigation buttons and menus, leading to confusion and errors.

    • iPad-Specific Apps: These were optimized for the device, offering larger, easier-to-use buttons and layouts designed for touch interaction.

  • Conclusion: The findings suggested that companies should prioritize developing apps specifically for the iPad, rather than relying on general desktop-style websites.

• What were the null and alternative hypotheses for the 12-point Verdana font versus 12-point Times New Roman font experiment that I described in class (and in my slides)?

• What were the independent and dependent variable?

• Hypotheses:

  • H₀ (Null): No difference in reading speed between 12-point Verdana and 12-point Times New Roman.

  • Hₐ (Alternative): Reading speed differs between the two fonts.

  Variables:

  • Independent Variable: Font type (Verdana vs. Times New Roman).

  • Dependent Variable: Reading speed (time to read text).

Know general, experimental design terms.

• For example:

  • Name and define the three types of experimental designs along with noting one advantage and one disadvantage of each type.

  • Able to use the “font“ experiment to describe different designs.

• Between-Subjects:

  • Definition: Each participant experiences only one condition.

  • Advantage: No order effects.

  • Disadvantage: Needs more participants.

  • Font Example: One group reads in Verdana, another in Times New Roman.

• Within-Subjects:

  • Definition: Each participant experiences all conditions.

  • Advantage: Controls individual differences.

  • Disadvantage: Potential order effects.

  • Font Example: Participants read in both Verdana and Times New Roman, order randomized.

• Matched-Participants:

  • Definition: Participants are paired by traits, then split across conditions.

  • Advantage: Reduces differences between groups.

  • Disadvantage: Difficult to match perfectly.

  • Font Example: Matched pairs; one reads Verdana, the other Times New Roman.

Know statistical testing concepts. For example;

• What do inferential statistical tests estimate?

• When do we typically reject the null hypothesis?

• Describe the difference between Type I vs. Type II errors.

• Inferential Tests: Estimate the probability (p-value) that results occurred by chance if the null is true.

• Reject Null: When p ≤ 0.05 (results unlikely due to chance).

• Errors:

  • Type I: False positive (rejecting true null, α).

  • Type II: False negative (failing to reject false null, β).

• What were the two goals of the Painpad field study?

• Describe two of the three types of data analyzed, and what that data showed (i.e., the study’s results).

• Two goals:

  • Determine if patients liked using the Painpad for monitoring pain.

  • Assess if Painpad improved compliance with the two-hour routine for logging pain compared to nurses.

• Two of the three types of data analyzed:

  • Satisfaction with Painpad based on questionnaire responses

    • Results: The average satisfaction score was 4.63 out of 5.

  • Patients’ compliance with two-hour routine for monitoring pain

    • Results: Some patients appreciated the notifications, while others found them bothersome.

Describe the “evaluation period“ (Stage 2) of a heuristic evaluation process. In addition, name and briefly describe (or give an example of) four of Nielsen’s ten heuristics.

• During the evaluation period, usability experts independently review the system for 1-2 hours by simulating user tasks, identifying heuristic violations, and conducting a two-pass review: first to explore, then to pinpoint issues.

• Four of Nielsen’s ten heuristics:

  • User control and freedom - Allow easy undo/redo (e.g., "Cancel" or "Undo" buttons).

  • Visibility of System Status: Keep users informed with timely feedback (e.g., progress bars during uploads).

  • Error Prevention: Prevent errors with constraints (e.g., disable "Submit" until all fields are filled)​

  • Help and documentation - Ensure support is easy to use and focused on users' tasks. (FAQs)

Identify two advantages (of 3 listed) and two disadvantages (of 4 listed) of heuristic evaluation.

• Advantages:

  • HE should find the most “big” problem with a prototype

  • Few ethical issues because users not involved

• Disadvantages:

  • Misses: HE can miss violations of heuristics

  • False alarms: HE can identify trivial problems sometimes

Describe the typical walk-through procedure (also called cognitive walkthroughs), making sure to describe (a) three of the four things that the developer presents to the usability experts, and (b) the three questions guiding the walkthrough.

• Procedure:

  • Usability experts simulate users' thought processes, evaluating tasks step-by-step.

• Things developers present to usability experts

  • Assumptions: User details, tasks, and context.

  • Prototype: The system or part of it.

  • Sample Tasks: Tasks users are expected to perform.

• Questions:

  • Will the user know what to do?

  • Can they see how to do it?

  • Will feedback confirm correctness?

Describe the additional participants and procedures in a pluralist walkthrough.

• Participants:

  • Team of users

  • Usability experts (evaluators)

  • Developers

• Procedures:

  • Participants identify issues independently.

  • Managed discussion: Users share first, experts follow, and developers prioritize.

  • Team collaborates on solutions​

Using an example, illustrate how analytics are used to evaluate web site (or advertising) effectiveness.

• A retailer uses Google Analytics to track traffic sources, user engagement, and conversion rates.

• For instance, if social media ads drive traffic but few purchases, they might adjust the landing page or ad targeting to improve effectiveness

What is AB Testing? What is it used for? How is it implemented? (Note: make sure to name the type experimental design.)

• What it is:

  • A/B testing is a controlled experimental design comparing two versions of a design (control vs. experimental) to evaluate performance differences.

• Purpose:

  • It is used to assess feature changes, such as design updates or new functionality, and determine which version performs better.

• How is it implemented?

  • Divide users into two random groups.

  • Present one group with the control version and the other with the experimental version.

  • Collect and analyze data (e.g., clicks, conversions) to measure which version meets the desired outcomes

(a) Discuss what Fitts’ Law predict, and (b) what it’s user for?

• Fitts’ Law predicts that the time required to point to or reach a target using a pointing device depends on:

  • The distance to the target.

  • The size of the target.

  The further the target or the smaller it is, the longer it takes to locate and accurately interact with it.

• Fitts’ Law is particularly useful for:

  • Designing interfaces: Ensuring buttons and clickable areas are appropriately sized and positioned.

  • Evaluating device usability: Optimizing key size and sequences for handheld devices.

  • Improving user experience: Making objects easier to select, especially in touchscreens or desktop applications.

Discuss prototyping, answering the following question: (a) Why prototype? (give 2 reasons) (b) What to prototype? (identify 3 aspects), and (c) How to prototype, including a discussion of the differences between low and high fidelity prototyping and the role of evaluation?

• Why prototype:

  • Designers questions in choosing between design alternatives

  • Stakeholders questions so that they can interact with a representation of the product

• What to prototype:

  • Physical appearances (e.g., size, shape, weight)

  • Screen features

  • System inputs and outputs

• How to prototype:

  • Start with low-fidelity prototypes (e.g., sketches, storyboards, or cardboard mockups):

    • Advantages: Quick, cheap, and good for exploring concepts and layouts.

    • Disadvantages: Limited detail and realism.

  • Gradually move to high-fidelity prototypes (e.g., interactive software):

    • Advantages: Looks and feels like the final product; better for usability testing.

    • Disadvantages: More resource-intensive and time-consuming.

  • Role of Evaluation:

    • Iterative testing helps refine designs by identifying strengths and addressing flaws.

    • Feedback from low- to high-fidelity stages ensures the product meets user needs effectively.

What are storyboards? Give an example showing how they can be used with scenarios.

• What are storyboards:

  • Storyboards are visual sequences that depict how users interact with a product, helping designers visualize workflows and user journeys.

• Example: Online Shopping Scenario

  • Scenario: Sarah buys a book online.

    Storyboard:

    1. Sarah visits the website (homepage).

    2. She searches "best science fiction books."

    3. A list appears; she clicks a book.

    4. She adds it to the cart.

    5. Sarah checks out and completes the purchase.

Provide a brief example of an experience map and describe why they are used (2 reasons).

• Example of an Experience Map

  • Scenario: Booking a flight online.

    Experience Map:

    1. Awareness: User searches for flight options online.

    2. Consideration: Compares airlines and reviews prices.

    3. Booking: Selects flight, enters details, and makes payment.

    4. Confirmation: Receives a confirmation email.

    5. Travel: Checks in, boards the flight, and travels.

• Why Use Experience Maps:

  • Identify Pain Points: Helps uncover user frustrations (e.g., confusing checkout processes) to improve usability.

  • Understand User Journey: Provides a holistic view of user interactions, enabling better alignment with user needs.