WEEK 25 Usability Engineering - Surveys and Questionnaires

CS2003 Usability Engineering Lecture Notes

Course and Instructor Information

• Course Code: CS2003

• Instructor: Dr. Faris Alwzinani

• Email: Faris.Alwzinani@brunel.ac.uk

• Office: WLFB-111

Lecture Overview: Surveys and Questionnaires

• Today's focus:

• Definitions of surveys and questionnaires

• Applications and advantages of questionnaires

• Evaluation tools for usability

• Designing effective questions

• Key tips for success

• Common pitfalls to avoid

Terminology

• Survey:

• Research methodology for collecting data from a sample.

• Types:

  • Face-to-face

  • Telephone (personal or computer-administered)

  • Self-administered (paper or online)

• Questionnaire:

  • A set of written questions designed for surveys.

Components of a Questionnaire

• Demographic Questions:

• Age, gender, occupation, income, education.

• Factual Questions:

• Behavioral patterns, ownership.

• Non-Factual Questions:

• Attitudes, beliefs.

Response Types

• Open Questions:

• Free text responses allowing detailed feedback.

• Closed Questions:

• Pre-defined responses, e.g., yes/no, multiple choice, scales.

Purpose and Timing of Questionnaires

• User Requirements Evaluation:

• Early design:

  • Descriptive purposes.

• Mid to late stages:

  • Measurement of usability.

• Advantages:

• Efficient way to gauge user insights.

Questionnaire Design Considerations

• Use caution for open-ended questions in new designs as users may lack clarity on needs.

• Useful for redesigning existing systems to assess strengths, weaknesses, and interest in new features.

Measuring Usability

• Usability Components:

• Effectiveness, efficiency, satisfaction.

• Satisfaction is subjective and needs to be assessed through user feedback.

Validity and Reliability in Measurement

• High Validity:

• Measures what is intended.

• High Reliability:

• Consistency in measurement.

• Achieving Reliable and Valid Measurement:

• Complex design process for psychometric properties; recommend using established questionnaires.

Example: System Usability Scale (SUS)

• A scoring method that yields an overall usability score out of 100

• Scores categorized as follows:

• Not Acceptable

• Acceptable

• Marginal

• Good

• Excellent

Sample Size for Usability Tests

• Aim for 10-15 respondents for quantitative data collection.

• Adjust based on evaluation design complexity and user type.

Crafting Effective Questions

• Demographic Questions:

• Include only necessary personal information; stress data ethics and anonymity.

• Prefer ranges over exact values.

Writing Factual Questions

• Typical queries cover demographics, ownership, behavior, and frequency of actions.

• Address recall challenges and societal pressures.

Writing Non-Factual Questions

• These deal with attitudes and beliefs but are less reliable due to biases.

• Common biases include leading questions, and social desirability effects.

Wording Strategies

• Avoid double-barreled and negative phrasing.

• Use clear and simple language; pilot test for clarity.

Measuring Attitudes

• Attitudes can be quantified using scales.

• Recommended to balance positive and negative items on scales (optimal: 7 points).

Developing Scales

• Use in-depth interviews for exploring constructs.

• Pilot studies to refine questions and conduct factor analysis on collected data.

Key Takeaways

• Utilize existing scales for measuring subjective usability when designing assessments.

• Recognize that designing a questionnaire involves complex considerations and pitfalls.