Comprehensive Study Notes on Observational Studies and Experimental Design
Foundations of Statistical Studies and Experimental Design
Two Primary Study Types in Statistics:
Experiments: Studies in which researchers intentionally apply a condition or treatment to subjects to observe and measure the outcome.
Observational Studies: Studies in which researchers observe and record data without directly interfering or imposing treatments.
Four Core Characteristics of Experimental Design:
Control: Managing outside variables and establishing baseline comparisons using control groups or placebos.
Randomization: Utilizing chance methods to assign subjects to experimental conditions to prevent bias.
Replication: Utilizing a sufficiently large sample size or repeating trials to ensure consistent, stable results.
Blocking: Grouping similar subjects together before assignment to reduce variability caused by known attributes.
Note: A study does not require all four elements to be fully active to be classified as an experiment, but control and intentional manipulation are primary indicators.
Core Statistical Concepts:
Population: The complete, exhaustive set of all individuals, objects, or subjects that a researcher wishes to draw conclusions about. Direct study of an entire population is usually impossible due to size and resource constraints.
Sample: A subset of individuals selected from the overarching population to be measured and analyzed.
Representative Sample: A sample that accurately reflects the key distribution of characteristics present in the target population. Generalization from a sample to a population is valid only if the sample is representative.
Formulating Research Questions:
Data collection must be guided by a clear research question that defines the study's primary objectives.
Example 1: Does the amount of daily time spent on social media influence the duration or quality of sleep in teenagers?
Example 2: Is an individual's weekly physical exercise routine significantly associated with their measured mental health outcomes?
Variables in Experimental Design: Explanatory vs. Response
Definition of Variables:
Variables are quantifiable or categorical characteristics and measurements collected from subjects in a study.
In an experiment comparing two or more groups, researchers classify variables into two core functional roles: Explanatory Variables and Response Variables.
Explanatory Variable:
The factor that is intentionally manipulated, varied, or categorized by researchers to observe its potential impact.
Thought to cause, influence, or produce the primary outcome of interest.
Corresponds to the independent variable in algebra ().
Response Variable:
The outcome or effect measured by researchers following the manipulation of the explanatory variable.
Responds directly to changes in the explanatory variable.
Corresponds to the dependent variable in algebra ().
Mathematical Analogy:
Consider the simple linear algebraic relation:
If , then .
If , then .
The value assigned to (explanatory variable) directly dictates and forces the resulting value of (response variable).
Applied Research Examples:
Cigarette Use & Health Outcomes: Research Question: "Does smoking cigarettes cause lung cancer?"
Explanatory Variable: Smoking behavior / cigarette consumption status.
Response Variable: Presence or absence of lung cancer diagnosis.
Social Media & Sleep Duration: Research Question: "Does time spent on social media alter sleep length?"
Explanatory Variable: Total time spent using social media platforms.
Response Variable: Total measured length of nightly sleep.
Experimental Design: Treatment and Control Groups
Deliberate Manipulation:
The fundamental feature distinguishing an experiment from an observational study is that researchers directly intervene by imposing a specific treatment on subjects rather than passively recording naturally occurring events.
Questions Investigated via Experiments:
Does daily intake of aspirin lower the probability of experiencing a heart attack?
Do college students prefer Pepsi over Coca-Cola during a blind taste test where brand identity is concealed?
Experimental Group Categorization:
Treatment Group: The group of experimental subjects that receives the active treatment or condition under investigation (e.g., given an aspirin dose, administered a new pesticide).
Control Group: The benchmark group of experimental subjects that does not receive the active treatment. They are either given a standard baseline treatment or an inactive substance.
Case Studies in Experimental Group Assignment:
Agricultural Crop Damage Study: Evaluating a new crop treatment against insect damage.
Explanatory Variable: Type of pesticide formulation utilized.
Response Variable: Measure of physical damage observed on corn crops / final crop yield.
Treatment Group: Corn plot sections sprayed with the new pesticide formulation.
Control Group: Corn plot sections sprayed with the standard traditional pesticide formulation (serving as the comparison baseline).
Auditory Distraction and Test Performance: Evaluating environmental noise effects on learning outcomes.
Treatment Group: Students taking an assessment in an environment with explicit background noise and audio distractions.
Control Group: Students taking the same assessment in a silent, quiet environment.
Control Mechanisms, Placebo Effects, and Blinding
Clinical Trial Case Study: Vaping Cessation and Antidepressants
Study Scope: youth subjects who currently vape and express a desire to quit.
Hypothesis: Administering an antidepressant daily over a -day intervention period reduces drug withdrawal symptoms and prevents relapse.
Explanatory Variable: Administration of antidepressant medication versus placebo.
Response Variable: Relapse status (whether subjects resume vaping or successfully abstain).
Group Structure: subjects assigned to the antidepressant treatment group; subjects assigned to the control group.
Imposed Experimental Controls: Restricting subjects' ability to purchase vaping products and prohibiting visits to bars or social venues where smoking/vaping commonly occurs.
Placebo and the Placebo Effect:
Placebo: A physical dummy treatment (such as a sugar pill, saline solution, or inert substance) formulated to visually and structurally resemble the active treatment without containing active ingredients.
Placebo Effect: A physical or psychological change brought about solely by a subject's expectation or belief that they are receiving an effective treatment.
Illuminating Metaphor (Little Jimmy and the Cookie):
A three-to-five-year-old child named Little Jimmy falls while playing in the living room and bumps his head on a chair.
Physical comforting and singing fail to quell his crying. His mother retrieves a sweet cookie from the kitchen cabinet and tells him that eating it will make his head injury stop hurting.
Upon eating the cookie, Little Jimmy smiles and stops crying.
A cookie possesses zero pharmacological painkilling properties. It functions as a placebo, leveraging psychological expectation to alter his subjective perception of physical pain.
Blinding Protocols:
Single-Blind Experiment: A study design in which the subjects do not know whether they have been assigned to the active treatment group or the control (placebo) group.
Double-Blind Experiment: A study design in which neither the experimental subjects nor the researchers/evaluators interacting with them (e.g., treating physicians, diagnostic dermatologists) know which subjects belong to the treatment or control groups.
Purpose of Double-Blinding: Eliminates investigator expectation bias and ensures uniform medical care, subjective assessment, and psychological environment across all experimental subjects.
Blinding Case Studies:
Clinical Topical Trial: A -week trial testing a novel medicated skin cream against a placebo cream. Patient skin condition is assessed at the conclusion by a dermatologist who is kept unaware of which cream each patient received. This represents a double-blind/blinded evaluation study design.
Acoustic Noise Study: Subjects complete testing in a room with a live musical band playing directly behind them. Because subjects are fully conscious of the live noise intervention, this design is non-blind.
Randomization, Replication, and Blocking
Randomization Protocols:
Randomization relies on pure chance processes to allocate subjects into treatment and control groups.
Purpose: Balances out unobserved, hidden background characteristics (confounding variables) across groups, preventing systematic baseline bias.
Mechanisms of Randomization:
Die Rolling: Rolling a fair six-sided die for each subject. Assigning even outcomes () to the treatment group and odd outcomes () to the control group yields an unbiased assignment process.
Coin Tossing: Tossing a fair coin to determine assignment. The exact individual outcome (Heads or Tails) cannot be predicted prior to the flip.
Replication and Sample Size:
Replication: The practice of assigning a treatment to a large number of experimental units or repeating the entire experimental procedure across multiple trials.
Mathematical Rationale (The Law of Large Numbers):
The theoretical probability of getting Heads on a fair coin flip is:
Flipping a coin times rarely yields exactly Heads and Tails.
Flipping a coin times rarely yields exactly Heads and Tails.
However, replicating the flip over trials will produce an empirical outcome converging extremely close to Heads, stabilizing at the theoretical metric.
Medical Replication Metaphor: Medical dosing guidelines (e.g., taking two Tylenol tablets every hours rather than every hour) are established based on replicated trials evaluated across broad sample populations.
Sample Size Thresholds:
A sample size of subjects provides insufficient replication, making the findings volatile and unreliable.
A sample size of subjects provides sufficient replication to stabilize mathematical estimates and minimize random chance variability.
Blocking Protocols:
Blocking: The intentional grouping of subjects into homogenous subsets (blocks) based on shared traits (such as age, gender, or behavioral risk factors) prior to random assignment.
Contrast with Randomization: While randomization acts as a indiscriminate filter for unknown confounding variables, blocking explicitly controls for known confounding characteristics.
Behavioral Blocking Metaphor (John and Peter):
Two close friends, John and Peter, both smoke and enter a smoking cessation study.
If assigned to the same experimental group, John's potential lack of commitment could directly demotivate Peter, introducing severe peer-influence confounding.
To block against this social interaction bias, researchers separate close acquaintances into distinct blocks prior to executing random assignment into treatment or control groups.
Interactive Scenarios and Evaluation Summary
Scenario 1: Agriculture Treatment
Intervention: Testing new vs. traditional pesticides on corn field damage.
Explanatory: Pesticide type (New vs. Old).
Response: Level of crop damage / overall yield.
Treatment: New pesticide formula.
Control: Traditional pesticide formula (placebo equivalent).
Scenario 2: Clinical Topical Skin Cream
Intervention: Medicated skin cream vs. placebo cream administered over weeks.
Evaluation Protocol: Evaluated by dermatologists withheld from treatment assignment data.
Design Classification: Double-blind clinical experiment.
Scenario 3: Auditory Testing Distraction
Intervention: Live musical band playing during assessment.
Design Classification: Non-blind experiment due to visible/audible subject awareness.
Scenario 4: Sample Size Replication Tests
Test A: Random assignment of subjects across treatment conditions. Result: Randomized and possesses sufficient replication.
Test B: Random assignment across only total subjects. Result: Randomized, but possesses insufficient replication.