Chapter 1-10 Notes on Self-Report Surveys and Biases

Chapter 1: Introduction

Focus on data collection needs and equipment considerations: video cameras, microphones, recording devices, and other measurement tools may be required depending on what you’re trying to measure.
Contrast between lab-like data gathering and natural environment data gathering: surveys can be completed in natural settings (e.g., in bed, in the kitchen) to observe data in real-world contexts.
Data collection is presented as a linear process, but in practice we usually consider many factors simultaneously and at various stages.
Key takeaway: understanding data collection context is essential for choosing methods that fit the research question.

Self-report data defined: participants describe their thoughts, feelings, and behaviors in a quantitative way.
Typical formats include:
- Surveys or questionnaires with Likert-type items (e.g., answering on a 5-point scale from Never to Always).
- Structured interviews where the same set of questions is asked of everyone and responses are recorded.
Administration details:
- Self-reports are quantified data; can be collected via online surveys.
- Structured interviews may be conducted by a researcher who reads questions aloud and guides the participant through the survey.
Terminology and example items:
- A common setup is a 5-point scale (Likert-type) to measure agreement or frequency.
- Example phrasing might involve a scale from 1 to 5, with 1 = Never and 5 = Always.
Practical note: surveys are a cost-effective way to gather data quickly, especially when delivered online.

Self-report data can be inexpensive and quick when done online.
However, longitudinal or prospective studies require collecting data at multiple time points, which introduces delays and scheduling considerations.
Example setup in a longitudinal study: participants stay with the study for a defined period (e.g., twelve continuous months) and complete surveys at multiple stages.
Trade-off: while online self-report methods are efficient, repeated measures over time increase complexity and potential attrition.

Self-report is often the only feasible way to assess internal states (thoughts, feelings, perceptions) because these states are not directly observable.
Strength: provides access to subjective experiences that can’t be inferred reliably from behavior alone.
Key question for researchers: what are the potential downsides or biases associated with self-report data?
Emphasizes the need to be mindful of limitations and sources of error when interpreting self-reported information.

Introduction to bias in survey data: two primary types discussed.
Self-serving bias:
- Tendency to present oneself in a favorable light and take credit for positive outcomes, while downplaying negative aspects.
Self-image bias (referred to as a related but distinct concept):
- Another bias related to how individuals view and present their own abilities or characteristics.
Important note: there are two main bias categories researchers should recognize and address when designing and interpreting surveys.

Distinction from self-serving bias:
- Social desirability bias involves answering in a way that you think will be viewed favorably by society or by the researcher, rather than accurately reflecting your behavior.
Common context: controversial or sensitive topics (e.g., intimate partner violence).
Example: respondents may underreport undesirable behaviors (e.g., starting conflicts, aggression) to avoid social disapproval, leading to distorted data.
Real-world implication: social desirability can obscure the true distribution of behaviors or attitudes on sensitive topics.

Interpretation bias: different readers may interpret a question differently, leading to inconsistent responses.
Literacy and comprehension issues can contribute to misinterpretation of items.
Researcher takeaway: ensure that questions are understood as intended and consider pilot testing to detect misinterpretations.

Necessity of avoiding jargon and using everyday language to improve comprehension.
Cultural differences can affect interpretation of terms (e.g., “hooking up” may mean different things across age groups or cultures).
Interpretation challenges are heightened by recall biases and differing knowledge bases.
Practical implication: use clear, accessible language and consider cultural/linguistic context when designing surveys.

Recall bias and awareness issues affect how accurately people remember and report past experiences.
Recall inaccuracy can vary across individuals and contexts.
Awareness changes can shift responses retroactively:
- Example from intervention research: asking about thoughts and feelings before and after an intervention can produce apparent declines in a measured skill (e.g., communication) even when real performance has improved.
This phenomenon is tied to changes in interpretation and knowledge gained during the study, not necessarily to actual deterioration in ability.

Key concept: response shift bias – after gaining new knowledge or awareness, respondents reinterpret prior states, leading to artificial changes in self-reported measures.
Response shift can complicate interpretation of pre/post survey data.
Researchers can attempt to control response shift statistically, but it remains a critical consideration when relying on self-report data.
Takeaway: awareness of bias sources (recall, interpretation, social desirability, etc.) is essential for designing robust studies and for accurate data interpretation.

The instructor pauses for questions before continuing, highlighting the interactive nature of the session and opportunities to clarify any points.