Lecture Notes: Experimental Design, Study Types, and Critical Evaluation

Experimental Design Basics

  • Two groups needed in an experiment: control and treatment.

  • Control group receives a placebo. The placebo is something that looks identical to the treatment (pill, shot, etc.) but has no active ingredient.

  • Placebo effect can be powerful because of psychological or other non-specific effects.

  • Population (number of subjects): more subjects provide more power to detect a true effect; larger sample size leads to more reliable conclusions.

  • Example mentioned: a large COVID vaccine study with thousands in both the treatment and control groups to provide enough power.

  • If you have only one subject, the study generally has no meaningful power.

  • Therefore, in studies, aim for a large population to increase the power of the study to validate the theory.

  • The treatment group is the group receiving the actual treatment.

  • The last step in the described sequence is the method (procedural aspect) of the study.

Placebo and Study Design Implications

  • Placebo can be used to control for psychological effects; the notion that people can lose weight in placebo groups is highlighted.

  • A hypothetical company could sell placebo (sugar pills) as a weight loss aid, illustrating the strength of placebo effects in weight loss studies.

  • In summary, placebo control is a key component of many experimental designs to isolate the effect of the actual treatment.

Data and Measurement

  • Data must be numeric and quantifiable so that statistical analysis can be performed.

  • Measurements need to be in numbers to enable numerical analysis and inference.

  • Population size affects the statistical power of the study.

  • Clarification question: "Which group receives the placebo?" Answer: the control group.

  • Placing the placebo in the control group ensures the comparison isolates the treatment effect.

  • Visualizing placebo: the placebo should look the same as the treatment to maintain blinding.

Study Designs: Blindness and Randomization

  • Single blind study: the human subjects do not know which treatment they receive, but the researchers do.

  • Double blind study: neither the subjects nor the investigators know which treatment is being given; a third party is responsible for randomization.

  • Double blind design is more unbiased, though more expensive.

  • In pharmaceutical research, double blind studies are typically required to ensure objectivity.

  • After data collection, results are published and subjected to expert peer review.

Publication and Source Evaluation

  • Large studies (e.g., COVID vaccination studies) are published after rigorous review.

  • There are different types of studies covered during the semester; the instructor will discuss published papers.

  • A good strategy for finding credible information is PubMed (funded by NIH) – free access to scientific papers.

  • When evaluating information found online (TikTok, Facebook, blogs): check if the source is legitimate and based on scientific studies; PubMed can help verify.

Types of Studies and Research Pathways

  • Epidemiology studies: study of populations, looking for patterns and associations; often rely on self-report (questionnaires, interviews) and data encoding.

    • Key characteristics: population-based, not necessarily in a controlled setting, reflects associations, not causation.

  • Human intervention studies (clinical trials): the gold standard for testing interventions in humans; demonstrate causation; essential for regulatory approvals (e.g., FDA).

    • Expensive due to costs of human subjects, housing, care, and controlled environments.

    • Outcomes can reflect causation if well designed.

  • Laboratory studies: conducted in university or research facilities; may involve animals or early-stage drug development; not directly generalizable to humans.

    • Limitations: results in animals or controlled lab settings may not translate to humans due to genetic and biological differences.

Drug Development and Why Drugs are Expensive

  • Drug development involves multiple stages:

    • Animal studies (preclinical) to assess safety and mechanism.

    • Phase I trials (small group) to assess safety and dosing in humans.

    • Phase II trials (larger group) to assess efficacy and side effects.

    • Phase III trials (large scale) to confirm efficacy and monitor adverse reactions.

    • Regulatory review and approval by agencies (e.g., FDA).

  • Timeline: development typically takes about 1010 years, often ranging from 88 to 1515 years.

  • Cost rationale: high upfront costs for preclinical and clinical testing, long development timelines, and high rate of failure.

    • A common figure cited: about 80extextpercent80 ext{ extpercent} of drugs fail at Phase III.

  • Example: Ozempic cost can be around ext{
    1000}permonth,illustratinghighout−of−pocketcostsfornewtherapies.</p></li><li><p>Whyearly−stagedrugsfail:resultsinanimalstudiesoftendonottranslatetohumansduetogeneticdifferences;translationtohumansisnotguaranteed.</p></li><li><p>Theprocessaimstoensurethatonlysafeandeffectivedrugsreachthemarket,butitisexpensiveandtime−consuming.</p></li></ul><h3id="27b5bdee−c982−4bdf−b067−e5d06ecbede8"data−toc−id="27b5bdee−c982−4bdf−b067−e5d06ecbede8"collapsed="false"seolevelmigrated="true">WhyAnimalStudiesAreLimited</h3><ul><li><p>Animalmodelscanshowpromisingresults,buttranslationtohumansisnotguaranteed.</p></li><li><p>Example:adrugmayreduceweightinratsbuthavenoefficacyinhumans;thisdiscrepancycanleadtolargefinanciallossesandmisdirectedresearch.</p></li><li><p>Thespeakermentionsthatanexpensivecompanyinvestedheavilybasedonanimaldataandfoundlittletonoclinicalbenefitinhumans.</p></li></ul><h3id="e60c4e3f−247b−479f−90d1−884547824011"data−toc−id="e60c4e3f−247b−479f−90d1−884547824011"collapsed="false"seolevelmigrated="true">CriticalThinking:EvaluatingClaimsinSocialMedia</h3><ul><li><p>WithwidespreadclaimsonTikTokandotherplatforms,verifyinformationbycheckingcrediblescientificsources.</p></li><li><p>PubMedisrecommendedforfindingpeer−reviewedstudiesandcredibleevidence.</p></li><li><p>Differentiatingbetweenlegitimatescientificfindingsandmarketingorsensationalclaimsisemphasized.</p></li></ul><h3id="04697cd1−50c6−4f25−9292−fdfbffda6d4a"data−toc−id="04697cd1−50c6−4f25−9292−fdfbffda6d4a"collapsed="false"seolevelmigrated="true">CreatineandNutritionTopicsMentioned</h3><ul><li><p>Creatineisapopulartopic;claimsaboutrapidmusclegainordramaticeffectsarecommononsocialmedia.</p></li><li><p>Thespeakernotesthathigherdosesofcreatinearenotmagicallymorebeneficial;thereisnuanceandaneedforevidence.</p></li><li><p>Afuturediscussiononcreatineisanticipated,butnotcoveredindepthinChapter1.</p></li></ul><h3id="7cd0e21c−efdc−4ca5−86db−b856efe84fec"data−toc−id="7cd0e21c−efdc−4ca5−86db−b856efe84fec"collapsed="false"seolevelmigrated="true">Chapter2andCourseStructure</h3><ul><li><p>OnMonday,Chapter2willbegin;thefollowingweekwillcontinuewithChapter2andcohort1.</p></li><li><p>OnTuesday,studentswillworkonquestionswithneighborsoralone;focusonin−classcompletionofthefirsthomework.</p></li><li><p>Theinstructorreassuresthatpriorchemistryknowledgemayhelp,butstudentswillnotneedtobalanceequationsforChapter2.</p></li><li><p>Thefirsttwoweeksaremeanttogiveafeelforthecourseandhelpstudentsdecidewhethertocontinueinthisclass.</p></li><li><p>Aquickrefresher:nutrientdensityisdefinedastheratioofnutrientstototalcalories;definedas<br>per month, illustrating high out-of-pocket costs for new therapies.</p></li><li><p>Why early-stage drugs fail: results in animal studies often do not translate to humans due to genetic differences; translation to humans is not guaranteed.</p></li><li><p>The process aims to ensure that only safe and effective drugs reach the market, but it is expensive and time-consuming.</p></li></ul><h3 id="27b5bdee-c982-4bdf-b067-e5d06ecbede8" data-toc-id="27b5bdee-c982-4bdf-b067-e5d06ecbede8" collapsed="false" seolevelmigrated="true">Why Animal Studies Are Limited</h3><ul><li><p>Animal models can show promising results, but translation to humans is not guaranteed.</p></li><li><p>Example: a drug may reduce weight in rats but have no efficacy in humans; this discrepancy can lead to large financial losses and misdirected research.</p></li><li><p>The speaker mentions that an expensive company invested heavily based on animal data and found little to no clinical benefit in humans.</p></li></ul><h3 id="e60c4e3f-247b-479f-90d1-884547824011" data-toc-id="e60c4e3f-247b-479f-90d1-884547824011" collapsed="false" seolevelmigrated="true">Critical Thinking: Evaluating Claims in Social Media</h3><ul><li><p>With widespread claims on TikTok and other platforms, verify information by checking credible scientific sources.</p></li><li><p>PubMed is recommended for finding peer-reviewed studies and credible evidence.</p></li><li><p>Differentiating between legitimate scientific findings and marketing or sensational claims is emphasized.</p></li></ul><h3 id="04697cd1-50c6-4f25-9292-fdfbffda6d4a" data-toc-id="04697cd1-50c6-4f25-9292-fdfbffda6d4a" collapsed="false" seolevelmigrated="true">Creatine and Nutrition Topics Mentioned</h3><ul><li><p>Creatine is a popular topic; claims about rapid muscle gain or dramatic effects are common on social media.</p></li><li><p>The speaker notes that higher doses of creatine are not magically more beneficial; there is nuance and a need for evidence.</p></li><li><p>A future discussion on creatine is anticipated, but not covered in depth in Chapter 1.</p></li></ul><h3 id="7cd0e21c-efdc-4ca5-86db-b856efe84fec" data-toc-id="7cd0e21c-efdc-4ca5-86db-b856efe84fec" collapsed="false" seolevelmigrated="true">Chapter 2 and Course Structure</h3><ul><li><p>On Monday, Chapter 2 will begin; the following week will continue with Chapter 2 and cohort 1.</p></li><li><p>On Tuesday, students will work on questions with neighbors or alone; focus on in-class completion of the first homework.</p></li><li><p>The instructor reassures that prior chemistry knowledge may help, but students will not need to balance equations for Chapter 2.</p></li><li><p>The first two weeks are meant to give a feel for the course and help students decide whether to continue in this class.</p></li><li><p>A quick refresher: nutrient density is defined as the ratio of nutrients to total calories; defined as<br> ext{Nutrient density} = rac{ ext{nutrients}}{ ext{total calories}}.</p></li><li><p>IftherearequestionsaboutChapter1,studentsareencouragedtoask.</p></li><li><p>Theinstructoremphasizesbuildingasolidfoundationandcomfortwithbasicconceptsbeforemovingforward.</p></li></ul><h3id="4d68ebed−839f−4906−861e−a2b3049510c0"data−toc−id="4d68ebed−839f−4906−861e−a2b3049510c0"collapsed="false"seolevelmigrated="true">NutrientDensity:DefinitionandFormula</h3><ul><li><p>Definition:Nutrientdensityistheratiobetweentheamountofnutrientsandthetotalcaloriesinafoodordiet.</p></li><li><p>Mathematicalrepresentation:<br></p></li><li><p>If there are questions about Chapter 1, students are encouraged to ask.</p></li><li><p>The instructor emphasizes building a solid foundation and comfort with basic concepts before moving forward.</p></li></ul><h3 id="4d68ebed-839f-4906-861e-a2b3049510c0" data-toc-id="4d68ebed-839f-4906-861e-a2b3049510c0" collapsed="false" seolevelmigrated="true">Nutrient Density: Definition and Formula</h3><ul><li><p>Definition: Nutrient density is the ratio between the amount of nutrients and the total calories in a food or diet.</p></li><li><p>Mathematical representation:<br>\text{Nutrient density} = \frac{\text{nutrients}}{\text{total calories}}.$$