Qualitative Research Methods and Data Collection Techniques

Qualitative Analysis

Big goal: let themes emerge from raw data, not test a hypothesis

Open coding

Label data segments with descriptive codes

Axial coding

Group related codes into categories

Selective coding

Identify core themes that explain the data

Codebook

List of codes with definitions and examples

Limits of Introspection

You can't always explain why you acted

Availability Heuristic

You judge by recent or vivid examples

Serial Position Effect

You recall first and last items best

Anchoring Effect

You rely on the first number you see

Framing Effect

You choose differently if info is 'gain' vs. 'loss'

Confirmation Bias

You favor info that matches your beliefs

Halo/Horns Effect

One trait makes you over‑ or underestimate others

Fundamental Attribution Error

You blame personality, not context

Open‑ended vs. closed‑ended questions

Multiple choice, Likert, semantic differential, ranked choice

Good question

One idea at a time; neutral wording; no double negatives

Mutually exclusive and exhaustive response options

Response options that do not overlap and cover all possibilities

Common flaws

Double‑barreled questions; leading wording; presuppositions

Response sets

Non‑differentiation, acquiescence, fence‑sitting

Reduce bias

Mix positive/negative phrasing; vary formats; include attention checks

Validity errors

Coverage, nonresponse, measurement

Structured interview

Fixed script, same questions for everyone

Semistructured interview

Guide plus open probes ('Tell me more')

Unstructured interview

Free‑form, participant‑led conversation

Interview vs. focus group

One‑on‑one vs. small group discussion

Challenges

Memory bias, social desirability, interviewer bias, dominant speakers, groupthink

Diary Studies

Participants record entries over time without researcher present

Entry triggers

Time‑based (e.g., daily) or event‑based (e.g., when frustration occurs)

Pros of Diary Studies

Real‑time data, natural context, reveals inaction

Cons of Diary Studies

High participant burden, risk of missing entries, depends on honesty

Comparison of Diary Studies and Surveys

Diary = longitudinal depth; survey = one‑time snapshot

Case study

In‑depth analysis of one case (tool, event, organization)

Edge case

Extreme or unusual example

Critical case

Pivotal example that can influence broader understanding

Holistic case study

Single unit of analysis (entire organization)

Embedded case study

Multiple subunits within one case (teams within organization)

Ethnography

Researcher participates or observes a group to study culture/practices

Case study vs. ethnography

Case is theory‑driven; ethnography is exploratory and participatory

Pros/cons of Case Studies and Ethnography

Limited generalizability vs. rich cultural insight, researcher bias risk

Automated Data Collection

Interaction logs: clicks, keystrokes, timestamps

Activity‑logging software

Screen recorders, proxy tools, usability tools

Custom software metrics

Built‑in usage tracking in applications

Strengths of Automated Data Collection

Large objective datasets, time‑stamped, minimal participant effort after setup

Weaknesses of Automated Data Collection

Large messy data, lack of context, heavy cleaning needed, privacy/ethical issues

Data analysis

Clean logs, infer high‑level actions, combine with surveys/interviews for context

Motivation strategies

Gamification, micropayments with attention checks