Experimental Design, Bias, and Data Interpretation Notes

FDA Guidelines and Hypotheses

The FDA requires that experiments be well-designed and well-controlled so that claims like a medicine is good for treating a disease are reliable.
Concept of null and alternative hypotheses is part of the design; the discussion acknowledges prior coverage of these ideas.
In this context, the experiment should provide evidence strong enough for regulatory approval, not just personal belief.

This particular discussion centers on an experiment with three groups.
How many people to enroll? A student suggests a small number (three), but the speaker emphasizes the importance of replicates and larger sample sizes to obtain reliable data.
Measurements mentioned include heart rate, pulse, and blood pressure to ensure safety and monitor effects.
Replicates: the speaker defines replicates as repeated measurements across subjects; three individuals is a basic replicate; with thousands of subjects, there would be many replicates (e.g., 3,000 people could yield about 1,000 replicates if divided across groups).
Large sample sizes help average out variability and reduce the impact of outliers or unrelated health issues.

Bias is a key concern in large human experiments.
Strategy to reduce bias: use many individuals to obtain a broad data set so averages reflect typical outcomes rather than extreme cases.
Random assignment: for a large pool (e.g., a few thousand people), assign participants randomly to groups using numbers and a random-number generator; the experimenter does not decide who goes into which group.
Blinding and information about group assignment: if participants know which group they are in, bias can be introduced in their responses or in how outcomes are reported. The idea of keeping group assignment hidden to prevent bias is implicit in the discussion (the contrast is with giving everyone a sugar pill and the behavioral expectations that could arise).
Placebo control: a sugar pill serves as a placebo, enabling a comparison between the active treatment and the placebo group.
Important conceptual point: controlling for bias is central to achieving objective results and aligning with the anti-authoritarian ideal that data, not personal opinion, should drive conclusions.

Confounding variable: a variable that makes it impossible to interpret data because it mixes with the effect you’re trying to measure.
The speaker gives a concrete example from a future behavioral experiment with roly-poly-like organisms in a petri plate. If a student taps a pencil near the plate, vibrations, sight, or sound could influence behavior, confounding the data.
To be confounded means that the variable confuses the interpretation of the data; it is not the variable you intended to study.
Replicates (repeat measurements): three people per group is a basic replicate; with 3,000 people, you could have about 1,000 replicates.

Results are the data themselves; interpretation follows from the data.
Assessing whether the treatment is supportive involves comparing outcomes (e.g., healing ulcers, side effects).
Consideration of future experiments: what other studies should be done, and how adopting the treatment would affect many people’s lives.
Objectivity as an ideal: data-based conclusions, if carefully and systematically collected with little bias, should inform action rather than rely on opinion alone.
Anti-authoritarian aspect: when data are robust and objective, conclusions should guide practice rather than rely on authority.

Biology introduces a lot of noise because subjects (e.g., organisms or humans) are genetically diverse.
Controls for this variability include large sample sizes to average out genetic differences and other sources of variation.
In biology-like experiments, confounding variables are particularly challenging due to inherent biological diversity and external influences.
In general, the better the control for confounding variables, the more reliable the conclusions.
The facilitator notes that biology often has more complexity than simple controlled experiments.

When reading graphs:
- Look at the axes:
- The x-axis typically represents the experimental/independent variable (also called the independent or experimental variable).
- The y-axis typically represents the response/dependent variable (the outcome being measured).
- If time is the attribute studied, the x-axis may represent time points (e.g., Week 1, Week 2, Week 3, Week 4).
- The y-axis shows the response variable (e.g., degree of healing, ulcer status).
Standard error bars visually indicate the variation around the mean due to sampling.
If data points have small standard errors and the error bars do not overlap between groups or time points, there is stronger evidence of a real difference.
If error bars overlap, it suggests that repeating the experiment might yield similar means; the difference may be less biologically and statistically significant.
In scientific papers and reports, most numerical results include error bars because almost all data have some sampling error.
Concept: the standard error of the mean (SEM) quantifies sampling variability around the sample mean.
- A common expression: $$SE = rac{s}{\