Data Gathering and Requirements Analysis

Data Gathering and Requirements Analysis in HCI

Objectives

• Plan and run a successful data gathering program.

• Plan and conduct interviews.

• Design simple questionnaires.

• Plan and execute observations.

• Identify requirements analysis procedures.

Five Key Issues of Data Gathering

1. Setting Goals: Decide how data will be analyzed after collection.

2. Identifying Participants:

   • Population and Sampling (Probability and Non-probability).

   • Probability sampling: Random (using random number generator or every nth person) and Stratified (dividing population into groups).

   • Non-probability sampling: Convenience and Volunteer panels.

3. Relationship with Participants: Maintain clear and professional interactions; obtain informed consent.

4. Triangulation: Investigate a phenomenon from at least two different perspectives.

   • Types: Data, Investigator, Theories, Methodological.

5. Pilot Studies: Conduct small trials before the main study.

Data Recording

• Methods: Notes, audio, video, photographs.

• Notes plus photographs: Least technical, flexible, but can be difficult to manage.

• Audio plus photographs: Useful alternative to note-taking, less intrusive than video, but transcription is time-consuming.

• Video: Captures visual and audio data; consider camera position, focus, and impact on participants.

Interviews

• Definition: Conversation with a purpose.

• Types:

  • Open-ended/Unstructured: Rich but not easily replicable; requires a plan of main topics.

  • Structured: Tightly scripted, replicable, but may lack richness.

  • Semi-structured: Guided by a script, allows for exploration of interesting issues.

  • Group Interviews (Focus Groups): Used in marketing, political campaigning, social sciences.

    • Involve 3-10 people, led by a trained facilitator.

    • Benefit: Raise diverse/sensitive issues.

Planning and Conducting Interviews

• Develop questions/topics.

• Gather documentation.

• Check recording equipment.

• Structure the interview and organize time/place.

Interview Questions

• Types: Closed (predetermined answers) and Open (no predetermined format).

• Avoid:

  • Long questions, compound sentences, jargon, leading questions, unconscious biases.

Running the Interview

• Introduction: Explain goals, ethical issues, ask to record, present consent form.

• Warm-up: Easy, non-threatening questions.

• Main body: Logical question order.

• Cool-off: Easy questions to defuse tension.

• Closure: Thank interviewee, signal the end.

Other Forms of Interview

• Skype, Email, Phone-based, Screen-sharing.

• Enrichment: Use of props like prototypes or scenarios.

Questionnaires

• Questions can be closed or open.

• Administered to large populations (paper, email, web).

• Sampling can be an issue.

Questionnaire Design

• Question order matters.

• Consider different versions for different populations.

• Provide clear instructions.

• Balance white space with compactness.

• Decide on positive, negative, or mixed phrasing.

Question and Response Format

• Yes/No checkboxes, multiple-choice checkboxes, rating scales (Likert, semantic - 3, 5, 7+ points), open-ended responses.

Encouraging Good Response

• Study purpose clear, promise anonymity, well-designed questionnaire.

• Offer short version, include stamped envelope (if mailed), follow-up, provide incentive.

• Response rates: 40% is high, 20% acceptable.

Advantages of Online Questionnaires

• Quick responses, no copying/postage costs, data collected in a database, reduced analysis time, easy error correction.

Problems with Online Questionnaires

• Problematic sampling, preventing multiple responses, question alterations.

Observation

• Direct observation in the field: Fill in details about user behavior.

• Can be complicated, produce much data.

• Direct observation in controlled environments.

• Indirect observation: Tracking users’ activities, diaries, interaction logging.

Structuring Frameworks for Observation

• The Person (Who), The Place (Where), The Thing (What).

• Robson (2011) framework: Space, Actors, Activities, Objectives, Acts, Events, Time, Goals, Feelings.

Ethnography

• Philosophy with techniques like participant observation and interviews.

• Researchers immerse themselves in the culture.

• Requires cooperation, uses informants, continuous data analysis.

• Interpretivisttechnique.

• Refine questions and reports contain examples.

Direct Observation in a Controlled Environment

• Often in usability labs; more formal.

• Users may feel apprehensive.

• Think-aloud technique: Users verbalize thoughts while using interface.

Indirect Observation

• Diaries: Useful for scattered participants.

• Interaction logs: Record user activity on a device.

Choosing and Combining Techniques

• Depends on study focus, participants, technique nature, resources.

Technique Overview

Summary

• Three main data gathering methods: interviews, questionnaires, observation.

• Four key issues: goals, triangulation, participant relationship, pilot studies.

• Interviews: structured, semi-structured, unstructured.

• Questionnaires: paper, online, telephone.

• Observation: direct/indirect, field/controlled.

• Techniques combined based on study focus, participants, technique nature, resources.