WEEK 25 Usability Engineering - Surveys and Questionnaires
CS2003 Usability Engineering Lecture Notes
Course and Instructor Information
Course Code: CS2003
Instructor: Dr. Faris Alwzinani
Email: Faris.Alwzinani@brunel.ac.uk
Office: WLFB-111
Lecture Overview: Surveys and Questionnaires
Today's focus:
Definitions of surveys and questionnaires
Applications and advantages of questionnaires
Evaluation tools for usability
Designing effective questions
Key tips for success
Common pitfalls to avoid
Terminology
Survey:
Research methodology for collecting data from a sample.
Types:
Face-to-face
Telephone (personal or computer-administered)
Self-administered (paper or online)
Questionnaire:
A set of written questions designed for surveys.
Components of a Questionnaire
Demographic Questions:
Age, gender, occupation, income, education.
Factual Questions:
Behavioral patterns, ownership.
Non-Factual Questions:
Attitudes, beliefs.
Response Types
Open Questions:
Free text responses allowing detailed feedback.
Closed Questions:
Pre-defined responses, e.g., yes/no, multiple choice, scales.
Purpose and Timing of Questionnaires
User Requirements Evaluation:
Early design:
Descriptive purposes.
Mid to late stages:
Measurement of usability.
Advantages:
Efficient way to gauge user insights.
Questionnaire Design Considerations
Use caution for open-ended questions in new designs as users may lack clarity on needs.
Useful for redesigning existing systems to assess strengths, weaknesses, and interest in new features.
Measuring Usability
Usability Components:
Effectiveness, efficiency, satisfaction.
Satisfaction is subjective and needs to be assessed through user feedback.
Validity and Reliability in Measurement
High Validity:
Measures what is intended.
High Reliability:
Consistency in measurement.
Achieving Reliable and Valid Measurement:
Complex design process for psychometric properties; recommend using established questionnaires.
Example: System Usability Scale (SUS)
A scoring method that yields an overall usability score out of 100
Scores categorized as follows:
Not Acceptable
Acceptable
Marginal
Good
Excellent
Sample Size for Usability Tests
Aim for 10-15 respondents for quantitative data collection.
Adjust based on evaluation design complexity and user type.
Crafting Effective Questions
Demographic Questions:
Include only necessary personal information; stress data ethics and anonymity.
Prefer ranges over exact values.
Writing Factual Questions
Typical queries cover demographics, ownership, behavior, and frequency of actions.
Address recall challenges and societal pressures.
Writing Non-Factual Questions
These deal with attitudes and beliefs but are less reliable due to biases.
Common biases include leading questions, and social desirability effects.
Wording Strategies
Avoid double-barreled and negative phrasing.
Use clear and simple language; pilot test for clarity.
Measuring Attitudes
Attitudes can be quantified using scales.
Recommended to balance positive and negative items on scales (optimal: 7 points).
Developing Scales
Use in-depth interviews for exploring constructs.
Pilot studies to refine questions and conduct factor analysis on collected data.
Key Takeaways
Utilize existing scales for measuring subjective usability when designing assessments.
Recognize that designing a questionnaire involves complex considerations and pitfalls.