the best method

Observations vs Experiments

  • A researcher applies a treatment to part of the population (treatment group) and does not apply the treatment to another part (control group).
  • The purpose is to observe responses after applying the treatment and compare outcomes between the groups.
  • In the transcript, examples come from medical realities and popular TV portrayals:
    • Cancer trials often use a cancer drug vs a placebo to form a treatment group and a control group.
    • The placebo is a harmless treatment that looks like the real one; subjects in the control group usually do not know it’s a placebo.
    • Responses from both groups are recorded and compared to assess effectiveness.
  • Ethics/real-world caveat mentioned in the talk: there are sometimes tensions around access to trials and rule-breaking in helping patients get in (contextualized in the pop-media example).
  • Basic vocabulary summary:
    • Treatment group: receives the active treatment.
    • Control group: receives no active treatment (often a placebo).
    • Placebo: a fake treatment that appears identical to the real treatment.
    • Experimental group vs control group: all subjects in experiments are divided into these two kinds of groups.
  • Observation vs experiment distinction:
    • If you change something in the environment, you are conducting an experiment.
    • If you simply observe without changing the environment, you are making an observation.
    • Classic example in talk: watching how many times you yawn in class is an observation if you don’t alter anything; changing conditions (e.g., providing different stimuli) makes it an experiment.
  • Practical framing:
    • In experiments, randomization and control are used to isolate effects of the treatment.
    • In observations, natural conditions are recorded with no intentional manipulation.
  • Quick real-world relevance:
    • Designing studies (treatment vs control) is foundational for assessing cause-and-effect.
    • Placebos and blinding (participants not knowing they’re in control) help reduce bias in responses.

Key examples discussed in the talk

  • Vitamin D trial (experimental):
    • Setup: seven patients received 4000 units of vitamin D3; another group of 70 patients received a placebo.
    • Treatment: the vitamin D3 (specifically, 4000 IU vitamin D3).
    • Conclusion (from the speaker): this is an experiment; treatment identified as vitamin D3.
  • Opinion survey on US economy (observational):
    • Researchers called many adults and asked them to rate current US economic conditions.
    • No manipulation of environment or treatment; therefore, observation.
  • Elk counting by Pennsylvania Game Commission (observational):
    • Observational study; data collection without deliberate experimental manipulation.
  • Whale tagging/presence in natural environment (observational with a note on capture/release):
    • Whales are captured or tagged briefly, then released back into the environment; the mechanism is observational in the main sense, with an interaction with the environment.
  • Medical information transparency survey (observational):
    • 1033 US adults; no treatment or manipulation; observed opinions.
  • Intensive program to reduce systolic blood pressure (experimental):
    • Treatment: intensive program to lower SBP to < or equal to 120 mm Hg.
    • Outcome: reduced risk of death.
  • Social media and teenagers’ brains (comparison of observation vs experiment):
    • Observation route: scan brains while teenagers view their own social media feeds (no manipulation).
    • Experimental route: show a set of photographs with a varying number of likes (altering the environment) to study brain responses.
    • Conclusion in talk: supplying photographs with varying likes constitutes an experiment; simply scanning brains while viewing their own feeds is an observation.

Sampling methods (six methods) — overview and quick definitions

  • There are two concepts that involve grouping of individuals:
    • Stratified sampling: split the population into groups (strata) and take a sample from each group (some from every group).
    • Cluster sampling: divide into subgroups (clusters) and sample entire clusters (100% from some clusters).
  • The other four methods deal with patterns or effort level in selecting individuals:
    • Systematic sampling: use a fixed pattern (e.g., every nth person) to select individuals; the “nth” pattern is explicit and repeatable.
    • Convenience sampling: select the easiest or most convenient respondents (lazy approach).
    • Voluntary sampling: individuals volunteer themselves to be part of the sample (self-selection); no one tells them to participate.
    • Simple random sampling: every member of the population has an equal chance of being selected; often described as a blind draw from a hat or a random selection process.
  • Summary mnemonic from the talk:
    • If it involves groups, think stratified or cluster.
    • If there is a clear repeating pattern, think systematic.
    • If it’s lazy or convenient, think convenience.
    • If participation is voluntary, think voluntary.
    • If it’s a true random draw, think simple random.
  • Quick examples from the talk that illustrate the six methods:
    • Stratified: Dividing students by majors and sampling from each major (some from every group).
    • Cluster: Selecting entire classes or groups (e.g., picking certain classes and surveying all students in those classes).
    • Systematic: Selecting every 10th person.
    • Convenience: Asking the 10 people closest to you in a class or outside a party.
    • Voluntary: People choosing to respond to a survey on their own, without being asked directly.
    • Simple random: Selecting participants via a random process such as random telephone numbers or a random draw.
  • Practice problem themes from the talk:
    • Classifying scenarios into observation vs experiment and identifying the treatment when it’s an experiment.
    • Examples included: limited manipulation vs no manipulation, and how the presence of manipulation changes the classification.

Bias in sampling and practical considerations

  • Potential sources of bias are discussed in the hurricane-grid sampling example:
    • If only certain grids with more damage are selected, the sample may not represent overall damage distribution (biased sampling).
    • If many grids are easy to reach and sampled preferentially, this convenience introduces bias.
  • Two biased scenarios discussed:
    • Sampling only grids with certain characteristics (e.g., higher damage) could over- or under-estimate true damage patterns.
    • Convenience sampling (selecting grids that are easy to reach) can bias results toward those areas rather than the population as a whole.
  • Non-bias/de-bias considerations to note from the talk:
    • Simple random sampling is presented as the