Stat slides chapter 1 part 3
Introduction to Experimental Design
What you'll learn in this section:
What a census is.
About simulations, observational studies, and experiments.
How to identify control groups, placebo effects, and different types of experiments (completely randomized, randomized block).
Common problems that can make data unreliable.
Census
A census collects information from everyone in a population.
Sample
A sample collects information from only a part of a population.
Observational Studies vs. Experiments
Observational study: You watch and collect data without trying to change anything. You simply observe what's happening.
Experiment: You do something specific (a 'treatment') to individuals to see if it causes a change in what you're measuring.
Example 4: Hawaii Goats (Study on goats and silver sword plants, 1778-related)
Background: Captain James Cook brought goats to Hawaii. The number of goats grew, and the silver sword plants on Maui decreased.
Biologists thought goats were causing the plant decline and decided to study it.
(a) How the experiment was designed:
They set up stations in remote areas. At each station, they picked two plots of land that were similar in soil, climate, and plant count.
One plot was fenced to keep goats out; the other was left open.
They regularly counted plants in both plots.
This is an experiment because they changed something (added a fence) to one plot to see its effect.
(b) Control and comparison: The unfenced plot served as the 'control group.' It was similar to the fenced plot in every way except for the fence (the treatment), allowing for comparison.
Placebo Effect
Placebo effect definition: Happens when someone thinks they're getting treatment (even if it's fake) and feels better or responds favorably as a result.
Completely Randomized Experiment
Definition: In this type of experiment, each individual is put into a treatment group completely by chance (randomly).
Block and Randomized Block Experiment
Block: A group of individuals (or things) that are similar in some way that might affect the outcome of the experiment.
Randomized block experiment: First, individuals are sorted into 'blocks' (e.g., by age or gender). Then, within each block, a random process assigns individuals to different treatments.
Guided Exercise 5: Data-Collection Techniques (Solutions)
(a) Which technique is best?
To study stopping a nuclear reactor's cooling process: Simulation (using a computer model).
(b) For time spent exercising by full-time college students: Sampling and observational study (surveying a random group, as collecting data shouldn't change their exercise habits).
(c) For the effect of calcium on bone mass in young women: Experimentation (randomly assigning some to a calcium supplement and others to a fake pill, called a placebo).
(d) For the credit hour load of students after the drop/add period: Census (collecting records for every single student).
(More detailed solutions:)
1 of 3: (a) Simulation; (b) Sampling/observational study; (c) Experimentation; (d) Census.
2 of 3: (c) Calcium study: Groups were chosen randomly, one got calcium, the other a fake pill (placebo). (d) Credit-hour study: Look at registrar records for all students to get complete data.
3 of 3: Additional context for (c) bone-mass study: An example from a Tom Lloyd JAMA study with 94 participants. Half received a placebo, half received calcium. The calcium group gained about 1.3 ext{ extbf{%}} more bone mass per year.
More Guided Exercise 5: Continued Solutions
(d) Census-type solution: Getting all student records from the registrar to obtain complete data for all students.
Some Potential Problems with Surveys (Part 1)
Nonresponse: People can't be reached or refuse to answer. This can mean certain groups are not well represented.
Truthfulness of response: People might not tell the truth or accurately remember events.
Faulty recall: People might not remember events or timings exactly.
Hidden bias: How questions are phrased, the order they're asked, or the range of answer choices can push people towards certain answers.
Some Potential Problems with Surveys (Part 2)
Vague wording: Words like “often,” “seldom,” or “occasionally” mean different things to different people.
Interviewer influence: The interviewer's tone, body language, clothes, gender, perceived authority, or background can affect how people answer.
Voluntary response: Only people with strong opinions are more likely to respond, so the survey results might not represent the general population.
Lurking Variables and Confounding Variables
Lurking variable: Something not measured in the study but still affecting the results.
Confounding: When you can't tell if the observed effect is from one factor or another, or both, because their effects are mixed up.
Confounding variables can be part of the study or outside 'lurking' things.
Guided Exercise 6: How Useful is the Data-Collection Plan? (1 of 2)
(a) A uniformed police officer interviews 20 college freshmen about illegal drug use in the last month:
Potential issues: People might refuse to answer or lie, especially about sensitive topics, to avoid looking bad (social desirability bias).
(b) Frankie's data shows cities with more low-income housing seem to have more homelessness:
Caution: Be careful! Other things (like overall city size or economic conditions) might be causing this link, not just low-income housing itself. These are possible confounding or lurking variables.
Guided Exercise 6: How Useful is the Data-Collection Plan? (2 of 2)
(c) A cafeteria survey using forms at the cash register with a drop box:
This will likely only get responses from people with strong opinions (either very happy or very unhappy), so the results won't represent everyone (voluntary response bias or self-selection bias).
(d) Coronary studies only with male participants over 50:
The results might only apply to older men and not to women or younger people. It's difficult to assume the same findings for other groups.
(Solutions Recap):
(a) People might not answer truthfully, and some might refuse to participate.
(b) Other confusing factors (like city size) can affect how we understand the results.
(c) Because people choose to respond, the results won't represent everyone (self-selection bias).
(d) The findings are limited to the group studied; they might not apply to different sexes or age groups.
Important Features of a Data Collection Plan (Part 1)
A good data collection plan should clearly state:
Who or what you're studying (the population).
What you want to measure or observe (the variable(s) of interest).
If you're just watching (observational) or actively doing an experiment.
If you're using a control group, fake treatments (placebos), 'blinding' (where participants/researchers don't know who gets what), etc.
Important Features of a Data Collection Plan (Part 2)
More details for the plan:
How you'll choose your sample, and if you'll use 'blocks' (groups of similar individuals).
The specific way you'll gather the data (e.g., through surveys, measurements, counting, etc.).
Viewpoint Discussion Activity (Part 1 of 4): The Placebo Effect
The placebo effect is real but tricky. Early studies suggested people felt about 35 ext{ extbf{%}} better just by believing they got treatment.
Some researchers think improvements are sometimes just due to 'regression to the mean'—meaning patients having particularly bad days often get better naturally, regardless of treatment.
Surprisingly, expensive fake treatments (placebos) can sometimes work better than cheap ones (e.g., a branded placebo vs. a generic-looking one).
Your mind plays a huge role in how you respond to treatment, and this psychological part significantly influences outcomes, especially for subjective measures like pain.
Viewpoint Discussion Activity (Part 2 of 4): Blinded Placebo-Control Groups
Why 'blinded' studies with fake treatments are crucial in clinical trials:
They help us tell the difference between the actual treatment's effect and the placebo effect.
They reduce bias from both participants and researchers when judging results.
They make the trial results more trustworthy and easier to understand.
Viewpoint Discussion Activity (Part 3 of 4): Ethics of Placebos
Is it okay for doctors to give fake pills for long-term pain?
A survey of rheumatologists found over half had prescribed a placebo, and more than 80 ext{ extbf{%}} felt it was acceptable.
What about companies selling 'miracle cure' products (like the Q-Ray Ionized Bracelet, which was found to be fraud) that only work because of the placebo effect?
Viewpoint Discussion Activity (Part 4 of 4): Other Ethical Considerations
Kinesiology tape in sports: It claims to reduce pain and help recovery, but scientific consensus often shows its effects are mainly from the placebo.
Ethical questions arise: Does the cost of a fake treatment matter? Should misleading claims for these products be regulated like real drugs or medical devices?
Summary
A quick look back at what we covered in this chapter: Getting Started.
Key ideas: census, sample, observational study, experiment, placebo effect, completely randomized experiment, randomized block, lurking and confounding variables.
We also looked at how to plan data collection, common survey problems, and important ethical questions.
Real examples and exercises helped illustrate how to pick the right data method and spot weaknesses in studies.