Experimental Design, Scientific Method, and Graphical Analysis Study Guide

Course Administrative Requirements and Schedule

• Weekly Tasks and Assignments (Current Week):

  • Watch Slide Set 1 – Course Policies Recorded Lecture.

  • Homework 1 (Course Policies): Due Sunday at $11:55\,\text{pm}$. Must be completed on a plugged-in computer (not a mobile phone). Multiple attempts are allowed prior to submission; however, once submitted, the assignment cannot be reopened or revised.

  • Weekly Quiz 1 (Slide Set 2): Opens Friday at noon; due Sunday at $11:55\,\text{pm}$. Located in the Weekly Quiz module on Canvas. Must be taken on a plugged-in computer. Time limit is $10\,\text{minutes}$ for $5\,\text{questions}$ covering the lecture material.

• Weekly Tasks and Assignments (Upcoming Week):

  • Required Reading: Textbook sections 1.1, 1.2, 1.3, 1.4, and the Interpreting Graphs handout (pages 6–9).

  • Textbook Reading Assignment 1: Due Wednesday at $11:55\,\text{pm}$ via the Textbook Reading Assignments module on Canvas.

  • Homework 2 ("How College is Different from High School"): Due next Sunday at $11:55\,\text{pm}$.

  • Weekly Quiz 2 (Slide Sets 3 and 4): Opens next Friday at noon; due next Sunday at $11:55\,\text{pm}$.

• Recitation Requirements:

  • Recitation sessions begin next week.

  • Read the Recitation Syllabus (found in the Administration module on Canvas) prior to attending.

  • Read the Recitation 1: Interpreting Graphs handout (pages 6–9, found in the Recitation Handout module) prior to class.

  • Bring a physical or digital copy (downloaded to a phone or device) of the Recitation 1: Interpreting Graphs handout to class.

Core Learning Objectives

• Understand the fundamental concepts of experimental design.

• Identify and label components of real-world scientific experiments.

• Analyze basic experimental setups and identify flaws or errors in design.

• Design rigorous experiments to test specific hypotheses.

• Interpret data presented in graphical formats and describe trends verbally.

• Evaluate basic statistical metrics when analyzing experimental outcomes.

Principles of the Scientific Method

• Overview of Scientific Inquiry:

  • Biological research begins by making observations and posing questions about natural patterns.

  • Example Case Study: The case of the disappearing teaspoons: longitudinal cohort study of the displacement of teaspoons in an Australian research institute (Megan S. C. Lim, Margaret E. Hellard, Campbell K. Aitken; Centre for Epidemiology and Population Health Research, Macfarlane Burnet Institute for Medical Research and Public Health, Melbourne). The institute employs approximately $140\,\text{people}$ across four programs to investigate workplace spoon displacement rates.

• Sequential Steps of the Scientific Method:

  1. Observe a pattern in nature: Recognize a consistent phenomenon or anomaly.

  2. Ask a question: Formulate a question regarding what causes the observed pattern.

  3. Form a hypothesis: Propose a testable explanation that answers the question.

  4. Make a prediction and test it: Formulate a specific prediction based on the hypothesis and execute an experiment.

  5. Analyze the results: Process experimental data to determine if outcomes match predictions.

  6. Report the results: Document findings for scientific review and dissemination.

• Iterative Decision Pathways Based on Outcomes:

  • If experimental results DO NOT match predictions: Reject or revise the original hypothesis, construct an alternative hypothesis, and design a new experiment to test it.

  • If experimental results DO match predictions: Do not stop testing; instead, test the hypothesis in new systems, under different conditions, or using alternative methodologies to confirm generalizability.

Foundations of Experimental Design

• Key Experimental Components:

  • Independent Variable: The single factor that is intentionally manipulated by the researcher across test groups.

  • Experimental Group(s): The condition(s) receiving the manipulated independent variable treatment.

  • Control Group: The unmanipulated baseline condition used for comparison.

  • Standard Conditions: Environmental and operational variables held strictly identical across all conditions to prevent confounding.

  • Dependent Variable: The variable measured to assess the effect of manipulating the independent variable.

• Five Major Errors in Experimental Design:

  • Insufficient Sample Size: Sample size is too small, leading to inadequate statistical power.

  • Lack of Standard Conditions: Test subjects or environmental conditions are inconsistent across treatments, introducing confounding variables.

  • Lack of Replication: The experiment is not repeated or reproduced to verify consistency.

  • Lack of Appropriate Controls: Unmanipulated baseline treatments are omitted or improperly designed.

  • Investigator / Participant Bias: Individuals involved in conducting or evaluating the study have preconceptions that skew outcomes.

• Analysis of Experimental Flaws (Commute Case Study):

  • Goal: Determine the fastest commuting method from Jersey City to Rutgers-Newark.

  • Hypothesis: Public transit is faster than driving.

  • Scenario 1: Driving daily during the week between Christmas and New Year's Day vs. taking public transit daily during the week of January 23rd. Conclusion: Driving is faster.

    • Design Flaw: Lack of standard conditions. Traffic density varies drastically between holiday weeks and normal workday weeks.

  • Scenario 2: Driving on Monday of an average week vs. taking public transit on Monday of a similar week. Conclusion: Transit is faster.

    • Design Flaw: Sample size is too small / Lack of replication. A single day ($n = 1$) per modality does not account for daily variance.

  • Scenario 3: Asking a local car mechanic of $20\,\text{years}$ for advice instead of testing directly. Conclusion: Driving is faster.

    • Design Flaw: Bias toward the outcome. The source possesses an inherent preference/bias toward automotive travel.

• Best Practices for Experimental Design:

  • Maintain a sufficiently large sample size.

  • Keep standard conditions uniform across all treatments.

  • Replicate the entire experimental procedure across multiple trials.

  • Ensure researchers and subjects remain unbiased (e.g., double-blind protocols).

Applied Biological Inquiry: Poison Ivy and Jewelweed

• Poison Ivy Etiology:

  • Contact with poison ivy (Toxicodendron radicans) causes severe contact dermatitis.

  • The active allergenic components are urushiols (catechol derivatives with long hydrophobic alkyl chains).

• Testing Jewelweed Efficacy:

  • Jewelweed (Impatiens capensis) is traditionally cited as a topical remedy for poison ivy rash.

  • Hypothesis formulation: Impatiens capensis contains compounds that neutralize urushiol-induced contact dermatitis.

  • Experimental Parameters to Define:

    • Subject selection ($n$, demographic standardization).

    • Method and concentration of poison ivy/urushiol application.

    • Plant tissue selection (flower, leaf, stem, root, or whole plant extract/mash/soap).

    • Application method and dosage.

    • Control conditions (negative controls such as distilled water; positive controls such as commercial soaps).

    • Standardized conditions (environment, duration of exposure before treatment, washing protocol).

    • Dependent variable measurement (dermatitis rash severity score on a defined numerical scale).

• Experimental Findings (Motz et al., 2012):

  • Study reference: V. Abrams Motz et al., Journal of Ethnopharmacology 143 (2012) 314–318.

  • Tested treatments: Distilled water, I. capensis extract, Lawsone solution, I. balsamina extract, I. capensis mash, I. balsamina mash, I. capensis soap, I. balsamina soap, Dawn dish soap.

  • Evaluation metrics: Rash severity score ($0\text{--}14\,\text{scale} \pm \text{SE}$) and active chemical lawsone content ($\mu\text{g}/\text{g}$ of plant material).

  • Significant reductions in rash severity were primarily associated with soap formulations (which remove urushiol mechanically) rather than inactive plant extracts alone.

Data Visualization and Graphical Analysis

• Selecting Visualizations by Data Type:

  • Bar graphs are optimal for displaying categorical independent variables paired with continuous dependent means (e.g., comparing mean systolic blood pressure between high-stress and low-stress groups).

  • Line graphs are optimal for continuous independent variables plotted against continuous dependent variables over time or gradient scales.

• Case Study: Lizard Body Temperature Over Time:

  • Raw Data Points: $(t = 0\,\text{min}, T = 25\,^\circ\text{C})$, $(t = 30\,\text{min}, T = 27\,^\circ\text{C})$, $(t = 60\,\text{min}, T = 29\,^\circ\text{C})$, $(t = 90\,\text{min}, T = 31\,^\circ\text{C})$, $(t = 120\,\text{min}, T = 32\,^\circ\text{C})$, $(t = 150\,\text{min}, T = 32\,^\circ\text{C})$.

  • Data Table Formatting:

    • Standard convention dictates placing the Independent Variable (Time in $\text{min}$) in the first column and the Dependent Variable (Temperature in $^\circ\text{C}$) in the second column.

  • Graphing Conventions:

    • X-axis: Independent Variable (Time in $\text{minutes}$).

    • Y-axis: Dependent Variable (Temperature in $^\circ\text{C}$).

    • Axis Truncation Effect: Truncating the Y-axis range (e.g., starting at $22\,^\circ\text{C}$ vs. $0\,^\circ\text{C}$) visually exaggerates the slope and rate of temperature change, though the underlying data are identical.

• Framework for Describing Graphs (EASUT):

  • E - Experimental units: Identify the subjects or items being studied.

  • A - Axes: Identify what variable is assigned to the X-axis and Y-axis.

  • S - Scale: Determine the numerical range and increment of each axis.

  • U - Units: Note the specific units of measurement used ($\text{min}$, $^\circ\text{C}$, $\mu\text{g}/\text{g}$, etc.).

  • T - Treatments: Identify the experimental groups and control conditions shown.

• Framework for Interpreting Graphs (PVC):

  • P - Pattern: Describe the primary trends, slopes, or differences between groups.

  • V - Variation: Analyze error bars, dispersion, overlap, and statistical indicators.

  • C - Conclusion: Formulate scientific conclusions supported directly by the data.

Statistical Principles in Scientific Research

• Quantifying Error and Uncertainty:

  • Error Bars: Visual representations on graphs denoting variability or statistical uncertainty. Larger error bars indicate greater sample variation.

  • Standard Deviation (SD): A statistic measuring data dispersion relative to the mean. In a normal distribution, approximately $68\%$ of individual measurements fall within $\pm 1\,\text{SD}$ of the mean.

• Standard Error of the Mean (SEM): An estimate of how far the sample mean is likely to be from the true population mean.

• Statistical Significance and $p$-Values:

  • Statistically Significant: A result where it is highly unlikely to have occurred solely due to random chance or sampling variability.

  • $p$-value: The probability of observing experimental results at least as extreme as those measured, assuming the null hypothesis (no real effect) is true.

  • Significance Cut-off: Generally, a result is deemed statistically significant when $p \le 0.05$.

  • Interpretation: Very small $p$-values (e.g., $p < 0.001$ denoted by asterisks like ***) demonstrate that chance is an unlikely explanation, supporting a biologically meaningful difference.