Lecture 30 - Intro to Experimental Design (Mod 3)

Introduction to Experimental Design

Importance of Experimental Design

• Organized and Efficient Planning: Enables a structured and efficient plan for a project and its experiments.
• Understanding Rationale and Purpose: Allows for clear comprehension of the experiments' rationale and purpose.
• High-Quality Research: Promotes reliable and replicable research.

Fundamentals Before Experiments

• Discipline Selection: Choose a discipline, area, and topic, and define its scope.
• Literature Review: Conduct a thorough literature search and review related to the topic and scope.
• Research Problem Identification: Identify the specific research issue or problem to be addressed. Avoid focusing solely on knowledge gaps, which may seem like academic curiosity.
• Research Question Formulation: Develop a research question that directly relates to the identified issue or problem.
• Hypothesis Construction:
  • Construct a hypothesis to address the research issue and questions.
  • Formulate predictions based on the hypothesis.
  • Ensure these predictions are testable.
  • Determine how to test these predictions.
  • The collected data will either accept or reject the hypothesis.
  • Consider the direction of the investigation: Hypothesis → data or Hypothesis ← data.
• Significance Elaboration: Explain the significance of addressing the research question. Who will benefit from this research? How does it impact:
  • The field of study?
  • The discipline?
  • Knowledge advancement (basic, foundational, applied)?
  • Stakeholders?
  • Industry?
• Aim Formulation: Formulate and narrow the aims of the study, ensuring they are linked to the central research question.

Rationale and Scope

• Rationale Definition: Define the logic and overall details of the research approach.
• Scope Definition: Define a specific and narrowed topic.
• Feasibility Assessment: Assess the feasibility and resolution of the approach to address the study aims.
• Proposal Drafting: Draft a research proposal and seek feedback from colleagues and the research team.

Designing and Planning Experiments

• Parameter Definition: Clearly and concisely define parameters for the experiment to focus on experimental methods and avoid ambiguity.
• Measurement Type:
  • Mensurative Experiment: Involves making measurements at different times or in different areas.
  • Manipulative Experiment: Involves physically altering a treatment group and always has two or more treatments.
• Sample Size:
  • Choose an appropriate sample size to obtain accurate and generalizable results.
  • Smaller sample sizes can lead to inaccurate generalizations and smaller effect sizes.
• Control Groups:
  • Introduce a control group to isolate changes due to the experimental treatment.
  • Control groups help account for temporal changes or influencing third variables in biological systems.
• Randomization:
  • Randomize sample units to different treatment groups to avoid experimenter bias.
  • Randomization is a critical aspect of experimental design as it intersperses the samples being tested.
• Replicates:
  • The number of replicates needed varies with the design but ensures precision in experiments.
• Sample Distribution:
  • Ensure samples are dispersed in space or time to avoid pseudoreplication.
  • This ensures that replicates are statistically independent.
  • Avoid making inferences based on data collected from the same unit as independent samples, as this is not genuine replication.
• Statistics:
  • Use linear regression-based analysis before introducing analysis of variance (ANOVA).
  • Linear regression is more powerful for analyzing data and indicates how dependent variables change with the independent variable.
  • Refrain from deducing results based on P-value alone.
  • P-value indicates the confidence interval in statistics but does not show how a system changes.
  • Effect size measures are more meaningful in ecology and should be given more weight in findings.

Experimental Design in Conservation Genetics/Genomics

• Topic and Scope: Clearly define the topic and scope of your research questions.

Experimental Design Session Summary

1. Define the problem.
2. Define the hypothesis.
3. Set objectives.
4. Outline the overall approach.
5. Obtain feedback.
6. Design experiments.
7. Conduct experiments.
8. Analyze data.
9. Interpret results.

Mapping and Direction of Experimental Design

• All dimensions of experimental design are interconnected.
• Multiple directions and sections need to communicate with each other:
  • The Introduction should align with the Experimental Design and Findings.
  • The Methodology should align with the Results section.
  • The Discussion section should address the issues and questions raised in the Introduction.
  • The Discussion section should align with the Results section, and vice versa.
• Key elements:
  • Issue/Problem
  • Context/Background
  • Objectives/Aims
  • Approach
  • Justification/Significance
  • Experiments
  • Data Analyses/Pipelines
  • Results/Findings
  • Interpretation/Discussion of Results
  • Recommendations

Group Project Assessment (AVBS3004)

• 20% of the final grade.
• Maximum 3000 words (excluding tables, figures, and reference list).
• Structure: Executive Summary/Abstract, Introduction, Methods, Results, Discussion, and References.
• Use real-world data to answer a research question.
• Work with a supervisor to develop ideas and analysis.

Proposal Structure and Alignment with Experimental Design:

• Summary/Abstract:
  • A clear and concise summary of the research project.
  • Align with the main body of the report.
  • Accurately reflect the findings within a broader conservation context.
• Introduction:
  • Research question situated within the context of the conservation challenge, issue/problems, and broader literature.
• Hypothesis (Hypotheses):
  • Appropriately worded hypothesis.
  • Relevant to the broader research question.
  • Testable by the data analysis approach.
• Methods:
  • A clear summary of the data collection/generation approach, data analysis approach recounted in sufficient detail to allow a third party to repeat the analysis.
• Results & Data Presentation:
  • The appropriate analysis taken given the data and research question.
  • Results presented in the appropriate format, including well-labelled diagrams/graphs, with any necessary measures of data variability (as appropriate).
  • Results clearly and concisely summarized in the text.
• Discussion and Management Recommendations:
  • Findings situated within the context of the broader literature.
  • Appropriate management/future research recommendations made, which align with the results.
• References:
  • Provide an appropriate evidence base to support all sections of the text.
  • Well-formatted citations (in-text and reference list).
• Group Work:
  • Teamwork, setting clear milestones, and response to advisor feedback.

Recommended Readings

• Please read the following papers. These can be found on the web or through the website of the University Library. These papers present information on interesting aspects of randomization and balancing of sampling and molecular ecology experiments.
• Sections 1.1 Experimental Design Background and 1.2 Sampling Design of An Introduction to Statistical Analysis in Research:With Applications in the Biological and Life Sciences, First Edition. Kathleen F.Weaver, Vanessa C. Morales, Sarah L. Dunn, Kanya Godde and Pablo F.Weaver. © 2018 JohnWiley & Sons, Inc. Published 2018 by JohnWiley & Sons, Inc.
• Bálint, M, Márton, O, Schatz, M, Düring, R‐A, Grossart, H‐P. Proper experimental design requires randomization/balancing of molecular ecology experiments. Ecol Evol. 2018; 8: 1786– 1793. https://doi.org/10.1002/ece3.3687