3.1 Observational Research

Observational Research Overview

• Observational research involves the systematic monitoring of participants (people or animals) and recording their behavior.

  • Naturalistic Observation: Conducted in natural settings ("in the wild").

  • Controlled Observation: Conducted in a controlled environment, such as a laboratory.

Example of Observational Research

• Case Study: Susan Goldin-Meadow's examination of language and gesture.

  • Monitoring gestures in communication requires a clear definition of what constitutes a gesture.

Operationalizing Behavior

• Defining behaviors is essential for accurate measurement:

  • Gestures (e.g., pointing) must be clearly operationalized to count accurately in observations.

• Exclusions include irrelevant movements such as:

  • Scratching

  • Fixing hair

  • Body movements unrelated to speech (e.g., bouncing).

Observation Choices

• Event Sampling:

  • Count the frequency of specific behaviors during a set time period.

  • Compute the percentage of time spent on the behavior.

• Time-based Sampling:

  • Record behavior presence in pre-selected short periods.

  • Count occurrences within randomly chosen video segments (e.g., 10 segments of 30 seconds).

Observer Bias

• Definition: When an observer's expectations impact their interpretation of participants’ behaviors.

• Minimization Techniques:

  • Develop specific coding criteria for observers.

  • Train observers thoroughly.

Inter-rater Reliability

• Multiple observers are essential for ensuring reliability in ratings.

• Calculation:

  • Inter-rater reliability often assessed via pairing ratings from a subset of data (15-20%).

  • Reliability scale:

  • 0-0.2 = Poor agreement

  • 0.3-0.4 = Fair agreement

  • 0.5-0.6 = Moderate agreement

  • 0.7-0.8 = Strong agreement

  • >0.8-1 = Almost perfect agreement

Settling Disagreements Between Raters

• Approaches include:

  • Averaging ratings

  • Discussion between raters on discrepancies

  • Using an independent rater for third-party resolution

  • All methods should be reported in research methodology.

Observer Effects

• Related to how observations may alter behaviors of participants due to observers' presence.

  • Masked (blind) designs help mitigate these effects by concealing predictions or conditions from observers.

Reactivity

• Reactivity refers to participants altering their behavior due to awareness of being observed.

• Minimization Techniques:

  • Blend in with observations or wait until participants acclimate to observation.

  • Measure outcomes indirectly rather than observing directly.

Comparison of Self-report and Observation

• Surveys/Self-report:

  • Best when the self-perspective is essential.

  • Potential for accurate self-reporting.

• Observation:

  • Offers essential behavioral insights not accessible via self-report but is often complex and time-consuming.

Practice Questions

1. For observational research, it is advisable to always have at least two research assistants conduct ratings in order to establish:

   • A) Observer effects

   • B) Internal reliability

   • C) Test-retest reliability

   • D) Inter-rater reliability

2. Dr. Ewell is concerned children will behave differently due to observer presence. He is concerned about:

   • A) Observer bias

   • B) Reactivity

   • C) Expectancy effects

   • D) Inter-rater reliability

3. Best ICC value for Dr. Ewell to find is:

   • A) -0.92

   • B) 0.02

   • C) 0.89

   • D) 1.15

4. Masked designs deal with:

   • A) Observer bias

   • B) Reactivity

   • C) Unclear coding rules

   • D) Reducing raters to one

5. List of behaviors where participants are not acting normally:

   • A) Observer bias, Observer effects

   • B) Observer effects, Reactivity

   • C) Observer bias, Reactivity

   • D) Observer bias, Observer effects, Reactivity