Study Notes on Experimental Design

1.3: Experimental Design

Introduction to Experimental Design

• Experimental design is the process of planning how to conduct an experiment or survey effectively, ensuring the study is statistically sound.

• The topic is complex and the notes will provide just a basic introduction to the principles of designing statistical studies.

Guidelines for Planning a Statistical Study

• Identify the Individuals:

  • Focus your study on a specific group.

  • Conclusions drawn can only apply to the identified individuals.

  • Example:

  • Using a fertilizer on a specific genus of plants does not allow for conclusions about other plant types.

• Specify the Variable:

  • The variable must be measurable and it is crucial to control for other factors.

  • Example:

  • To measure the impact of fertilizer on plant height, ensure control of other variables like sunlight, water, and temperature.

• Specify the Population:

  • Define the population from which you'll draw conclusions.

• Specify the Measurement Method:

  • Decide between a census or sample collection method. If sampling, choose the sampling method carefully.

• Collect the Data:

  • Use appropriate descriptive and inferential statistics to analyze the data.

  • Acknowledge any concerns about data collection methods and make recommendations for future studies.

Types of Studies

• Observational Study:

  • The researcher collects data by observing or asking questions without intervening.

• Experiment:

  • The researcher changes a variable or applies a treatment to evaluate its effect.

  • Examples:

  • Polling students about tuition increases (observational).

  • Giving students tutors to observe changes in grades (experiment).

Survey Considerations

• Surveys should be designed to avoid bias in the questions used to collect data.

Experimental Designs

• Randomized Two-Treatment Experiment:

  • Consists of two treatment groups where individuals are randomly assigned.

  • Control Group: Individuals receiving no treatment or a placebo.

  • Treatment Group: Individuals receiving the actual treatment.

  • Placebo: A sugar pill or fake treatment used to account for the placebo effect.

• Randomized Block Design:

  • Subjects are grouped into blocks based on similarities, and treatments are randomly assigned within those blocks.

  • Example: Separate full-time and part-time students before treatment assignment.

• Rigorously Controlled Design:

  • Subjects assigned to treatment groups carefully to ensure similarity across important factors.

  • May prove difficult to administer effectively due to differentiation complexities.

• Matched Pairs Design:

  • Special focus on pairing subjects by related characteristics.

  • Example: Testing a muscle relaxer cream on both arms of an individual and comparing results.

  • Before and After Studies: Comparing weight before and after a diet.

General Considerations for Experiments

• Replication:

  • Conducting the experiment multiple times to ensure a large enough sample size, which helps distinguish true effects from random ones.

• Blind Study:

  • The participant is unaware of which treatment is being given.

• Double-Blind Study:

  • Neither the participant nor the researcher knows which treatment is administered, minimizing bias.

Types of Time Periods in Studies

• Cross-Sectional Study:

  • Data gathered from a population at one point in time.

• Retrospective Study (Case-Control Study):

  • Data collected from the past through records, interviews, etc.

• Prospective Study (Longitudinal or Cohort Study):

  • Data collected going forward over time.

Homework Questions and Scenarios

1. Determine whether the following situations describe an observational study or an experiment:

   • Cinnamon and Insulin Sensitivity: Experiment — actively measuring glucose levels after administering cinnamon.

   • Fruit Consumption and Cancer: Observational — watching and recording dietary habits over time without manipulation.

   • Fertility Rates and Life Expectancy: Observational — collecting data from different countries without any experimental treatment.

   • Comparing Fertilizers: Experiment — applying different fertilizers to distinguish effects.

   • Blood Pressure Drug Study: Randomized experiment, as subjects are randomly assigned treatments.

   • Stent Study Decisions: Not a randomized experiment; decisions made ad hoc by the doctor.

   • Diet and Exercise Weight Loss Study: Not randomized — volunteers choose their groups.

   • Knee Flexibility Study: Randomized — volunteers are randomly assigned to exercise or not.

   • Tagged Fish Weights: Not matched pairs; comparisons are across different treatment groups.

   • Computer Homework System: Not matched pairs unless controls are applied, as different class formats are used.

   • Business Process Improvement: Potentially matched pairs, if pre- and post- measurements are comparable.

   • Generic vs. Named Brand Prices: Not matched pairs; different products rather than the same.

   • Double Blind Drug Experiment: Double blind because neither the patient nor the doctor knows who receives what.

   • Exercise Groups: Neither blind nor double-blind since participants know their group assignment.

   • Surgical Procedure Recovery Study: Blind since patients are unaware of the procedure but surgeons are informed.

   • Headache Pain Medication Study: Double blind due to the absence of bias from both parties.

   • Cancer Risk and Diet Tracking: Prospective — tracking future patterns and outcomes.

   • Medication and Pain Levels: Potentially cross-sectional or retrospective, depending on data collection methods.

Conclusion

• Proper experimental design is crucial for gathering reliable data in statistical studies. Understanding the distinctions between types of studies, experimental designs, and the importance of blinding and replication will significantly enhance the validity of the results.

Licensing Information

• Content shared under a CC BY-SA 4.0 license by Kathryn Kozak via LibreTexts.