4.2: Observational Studies and Experiments

Observational Studies

• Observational study: a study that observes subjects and measures variables but does not impose treatments on those subjects
  • Observe something passively and record information about individuals (experimental units) without disturbing or influencing the responses
• Valuable for discovering trends and possible relationships
• Not possible to determine cause/effect relationships through observational studies

Types of Observational Studies

• Retrospective study: looks back in time and examines data for a sample of individuals
  • Likely to have errors
• Prospective study: investigators follow a sample of individuals into the future collecting data in order to investigate a topic of interest about the population
• Longitudinal study: taking a cohort of subjects and watching them over a long period
  • Usually have fewer potential sources of bias and confounding then retrospective studies
• Sample survey: collects data through a survey in an attempt to learn about the population from which the sample was taken

Experiments

• Impose specific treatments on individuals being studied (experimental units) to measure the response variable to changes in the explanatory variable
  • Explanatory variables are also called factors and there can be different levels of factors
  • Eg. factor: color / levels: red, blue, yellow
  • Eg. factor: shape / levels: square, circle, triangle
• The only source of fully convincing data to understand cause and effect, if well-designed

Key Terms

• Experimental unit: the individuals being assigned to treatments
• Explanatory variable/factor: a variable whose levels are manipulated intentionally
  • Treatment: the level or combination of levels to which the explanatory variables are performed
• Response variable: an outcome being measured after the treatments have been administered

How to Experiment Well

• Use sophisticated random sampling from a population
• Use chance (random assignment) to assign treatments to subjects
• Usually do something (treatments) to experimental units
  • Humans are called “subjects” rather than “experimental units”
• Use a control to mitigate the effects of lurking variables
  • Lurking variable: a variable not among the explanatory or response variables, but that may influence the response variable
  • Make it hard to see the true relationship between explanatory and response variables
  • If they can be controlled, they should be, but often this is not possible
  • Confounding variable: occurs when two variables act in such a way that their effects on the response variable cannot be distinguished from each other
  • Well-designed experiments take steps to prevent confounding
• Compare the response variables of the treatments
  • Can give good evidence for causation

Principles of Experimental Design

1. Control

   1. Control the effects of lurking variables by comparing several treatments

   2. Ensure that the only difference between groups is the treatment

   3. Pay careful attention to details

   4. Use control groups

      1. Control group: a group which receives either no actual treatment or a standard and accepted treatment to compare to the experimental treatment(s)

   5. Well-designed experiments include comparison among treatment groups which allows interpreters to determine if the treatment being tested has an actual effect

2. Randomization

   1. Random assignment uses chance to assign treatments to subjects to create equivalent groups that will generally be the same in regard to all other (known or unknown) variables
   2. Helps balance out the effects of lurking variables that can’t be control or are unanticipated
   3. Accounts for differences for unknown/uncontrolled/confounding/other variables between treatment groups

3. Replication

   1. If many subjects are assigned to each group, the differences in the effects of treatments can be distinguished from chance

\