Research Methods: Descriptive Research and Validity

Revisiting Descriptive Research and Construct Validity

  • Descriptive research focuses on describing what people do without manipulating the variables.
  • Construct validity assesses how well a study measures what it claims to measure.

Creating Reliable and Valid Surveys

  • Important Considerations:
    • Ensure questions are understood clearly.
    • Avoid confusing wording or ambiguous terms.

Making Reliable and Valid Observations

  • Observational methods:

    • Close observation and documentation of behavior.
    • High external validity; useful in natural settings.
    • Low control over extraneous variables.

  • Case Study:

    • Detailed analysis of an individual or specific group.
    • Helps in understanding unique cases but less generalizable.

  • Survey:

    • Data collection via questionnaires or interviews.
    • Good for large samples, but sampling method is crucial.

The Four Big Validities (Table 3.5)

  1. Construct Validity:

    • Measures how well the variables are defined and manipulated.

  2. Statistical Validity:

    • Evaluates how well the study's numerical data supports conclusions.
    • Includes effect size and confidence intervals.

  3. External Validity:

    • Examines the extent to which results can be generalized beyond the sample.

  4. Internal Validity:

    • Focuses on whether changes in one variable (A) can be attributed to changes in another variable (B), free from interference by other variables (C).

Evaluating Claims Using the Four Validities (Table 3.6)

Frequency Claims (e.g., "4 in 10 teens admit to texting while driving")

  • Assess construct validity by evaluating measurement quality.
  • Check statistical validity through confidence intervals and sample representativeness.
  • Evaluate external validity by considering generalizability to other populations/settings.

Association Claims (e.g., "Study links exercise to higher pay")

  • Focus on construct validity for both measured variables.
  • Examine statistical validity through effect size and study estimates.
  • External validity standards apply, especially regarding the populations to which findings can generalize.

Causal Claims (e.g., "Pretending to be Batman helps kids stay on task")

  • Must be substantiated by experimental studies with controlled conditions.
  • Assessment involves all forms of validity, particularly internal due to the need for a cause-and-effect link.

Mistakes to Avoid in Surveys

  1. Questions with ambiguous answers:

    • E.g., "How many times did you feel sad in the past year?"
    • Difficult for participants to quantify experiences accurately.

  2. Leading/loaded questions:

    • E.g., "Do you support the pro-life view that abortion is murder?"
    • Presumes agreement on contentious issues, risking bias.

  3. Double-barreled questions:

    • Explores multiple constructs in one question:
      • E.g., "I am generally a very relaxed and reliable person".

  4. Double negatives:

    • Can confuse participants; simplify wording.

  5. Confusing or ambiguous wording:

    • E.g., "How often do you sometimes feel happy when you're alone?"

Affective Neuroscience Personality Scales (ANPS)

  • Developed to assess human behavior related to six primary neural affective systems.
  • Aim for concise and high-validity questions regarding behavioral traits related to playfulness and enjoyment in social contexts.

Response Sets in Self-Reporting

  • Be aware of tendencies such as:
    • Yea-saying or Nay-saying,
    • Social desirability biases,
    • Responding systematically, which may cloud genuine responses.

Reliability Testing

  • For surveys or tests (self-reports):
    • Test-retest method, split-half analyses.
  • For observational data, use inter-rater reliability to ensure results are consistent across different observers.