In-Depth Notes on Experimental Design and Observational Studies

Experimental Design

  • Observational Studies

    • Researchers do not assign treatments or choices; instead, they observe them.

  • Retrospective Observational Study

    • Subjects selected based on past conditions or behaviors.

    • Focus on estimating differences between groups or associations between variables.

    • Drawbacks include lack of random sampling.

  • Prospective Observational Study

    • Subjects are followed to observe future outcomes without deliberate treatment.

    • Focus on estimating differences as groups are followed throughout the study.

  • Experiments

    • Purpose: Study the relationship between two or more variables by imposing treatments on subjects.

Key Elements in Experimental Design

  • Factors

    • Explanatory variables involved in the experiment.

  • Levels

    • Various treatments tested within each factor.

  • Treatment

    • Each combination of factors and levels applied to experimental units.

  • The Best Experiments

    • Characteristics include:

    • Randomized

    • Comparative

    • Double-blind

    • Placebo-controlled

Four Principles of Experimental Design

  1. Control

    • Helps reduce extraneous variation by making conditions similar across treatment groups.

  2. Randomization

    • Balances uncontrolled variability and reduces bias by equalizing effects spread.

  3. Replication

    • Involves repeating treatment applications to multiple subjects for reliability.

  4. Blocking

    • Groups subjects by a similar characteristic to reduce variability across treatment effects.

Statistical Significance and Confounding

  • Statistically Significant Results

    • Occur when outcome differences cannot be attributed to chance, thus linked to treatments.

  • Confounding Variables

    • When effects of two variables on a response cannot be distinguished.

Matched Pairs Design and Blinding

  • Matched Pairs Design

    • Compares subjects that are similar on untested attributes, ensuring each subject experiences all treatments sequentially.

  • Placebo

    • A control treatment that mimics the experimental treatment, enhancing the validity of the results through blinding.

Experimental Write-up Components

  1. Plan: Identify factors and levels.

  2. Response: Define the response variable being measured.

  3. Treatments: Outline the treatments used.

  4. Experimental Design: Explain selection processes for treatments (Control, Randomization, Blocking).

  5. Analysis: Measure and compare data collected to draw conclusions.

  6. Repeat and Replicate: Ensure validity through repeated trials.

Applying Experimental Design Concepts

  • Examples:

    • Tsunami Construction Materials

    • Considerations for building materials that withstand natural disasters while controlling for variables.

    • OptiGro Plant Fertilizer

    • Experiment to test a claim using different treatment groups with controlled application.

  • Responding to Questions:

    • Identify if a scenario is an observation or an experiment, determine subjects and response variables, check for hidden variables, and suggest improvements in methodologies.

Case Studies

  • Example of Exercise Testing Effectiveness

    • Test on insomnia treatment comparing diet restrictions and exercise among volunteers to assess effectiveness.

  • Reflecting on Results

    • Assessing how external variables may influence outcome measures in various studies (e.g., blood pressure, omega-3 fatty acids, etc.).