Alternative Explanation

Alternative Explanations

  • Definition: A practice of critical evaluation.

  • Importance: When evaluating a claim or argument, it is essential to consider alternate reasons that could explain the results reported.

  • Assertion: There are nearly always alternative explanations.

  • Evaluation Process: Compare the alternatives with the explanations provided, determining their validity and reasonableness.

Key Considerations in Research Design

1. Experimental Studies

  • Critical Questions:

    • Is there a control group?: Essential for comparison.

    • Is there random assignment?: Key to eliminating bias.

2. Cross-Sectional Survey Studies

  • Critical Questions:

    • Are they cherry-picking data?: Look for selective reporting of data to support claims.

    • Is it a (Selective) small sample?: Assess if the sample adequately represents the population.

Experimental Design Foundational Characteristics

  • Longitudinal: Observations made over time.

  • Establishing Causality: Ability to infer cause-effect relationships.

  • Purpose: Explanation, prediction, and control of variables.

Basic Experimental Designs

  • One-Group Pretest-Posttest: Observations before and after treatment.

  • Two-Group Pretest-Posttest: Comparison between two groups with pre and post measures.

  • Two-Group Random Assignment Pretest-Posttest: Random assignment helps ensure groups are comparable.

  • Control Groups: Important for baseline comparisons of treatment effects.

  • Random Assignment: Necessary to mitigate pre-existing differences between subjects.

Control Groups

  • Function: Provide baseline data against which treatment effects can be compared.

  • Importance: Without a control group, it is impossible to determine if observed changes are genuinely due to the treatment.

Example of Missing a Control Group

  • Scenario: Improved quiz performance measured at Time 2 after an intervention.

  • Question: Can we confirm manipulation caused the observed improvement?

  • Alternatives: Consider other factors contributing to performance change.

Lecture Activity #1: Productivity Study Scenario

  • Context: Employees at Company A worked on a research study measuring productivity.

  • Intervention: Participants required to alter their sleep schedule to enhance productivity.

  • Findings: Significant increase in productivity noted post-intervention.

Discussion Questions

  1. Type of Study: Is it an experiment or a cross-sectional survey?

    • Answer: Experiment - measures over Time 1 and Time 2 with a manipulated variable.

  2. Design Name: One-Group Pretest-Posttest.

  3. Key Limitation: No control group.

    • Implication: Participants might alter behavior due to awareness of measurement.

  4. Alternative Explanations: E.g., productivity may improve with awareness of being watched.

Random Assignment

  • Definition: An experimental technique for assigning participants randomly to different groups.

  • Purpose: Ensures each subject has an equal chance of falling into any group, reducing systematic differences.

  • Importance: Without random assignment, discrepancies among groups may lead to alternative explanations for outcomes.

Lecture Activity #2: Vaccine Experiment Scenario

  • Context: A study compared outcomes of a vaccine against a placebo based on self-selection of participants.

Discussion Questions

  1. Type of Study: Experiment - measures over Time 1 and Time 2.

  2. Design Name: Two-Group Pretest-Posttest.

  3. Key Limitation: No random assignment leads to non-comparable participant health statuses.

    • Implication: Healthier participants may choose the treatment, skewing results.

Cross-Sectional Survey Studies Foundational Characteristics

  • Nature: Provides a snapshot of current status.

  • Goals: Primarily for exploration/description and explanation purposes.

Key Concerns

  • Cherry-Picking Data: Selective reporting that supports specific hypotheses while ignoring contradictory evidence.

  • Sample Size: Small samples can be non-representative of larger populations.

    • Statisticians apply the law of large numbers to ascertain sample sufficiency.

Cherry Picking Data

  • Definition: Selecting data that supports favored hypotheses while disregarding opposing evidence.

  • Importance: Valid scientific research corresponds with falsification rather than mere validation.

  • Researcher Responsibilities: Maintain an open mind and consider all evidence before drawing conclusions.

Activity Question #3: Evaluating Political Support Claims

  • Scenario: Report states five times as many Republicans support Candidate A than Candidate B.

  • Task: Critically assess possible cherry-picking and alternative explanations for reported data.

(Selective) Small Sample

  • Insight: Smaller samples are often unrepresentative, potentially invalidating findings.

  • Importance of Larger Samples: Increased accuracy in reflecting the population.

Activity Question #4: Evaluating Demographic Study Results

  • Scenario: Research results showing disparate gender ratios in newborns across two towns.

  • Task: Critically analyze conclusions focusing on sample sizes affecting reliability.

Sample Size Considerations

  • Determining Factors: Statistical and sensible assessments guide adequacy of sample sizes.

    • Base Rate Significance: Understand disease incidence rates when evaluating sufficiency.

  • Example: A sample size of 300 may suffice for rare diseases but may be inadequate for common conditions (e.g., 1 in 10 people).

Summary

  • Emphasis on critical evaluation practices in research.

  • Understanding different research designs and their implications regarding validity and reliability.

  • Consideration of alternative explanations in experiments and surveys, including control group utility and random assignment importance.

  • Awareness of cherry-picking data and the implications of small sample sizes in research findings.