Power Analysis in Grant Proposals

Power Analysis Calculations in Grant Proposals

Introduction

  • Researchers must justify sample size in grant applications for human subject studies.
  • Formal power analysis may not be needed for small feasibility studies, but sample size justification is still required.

Underpowered Studies

Meaning

  • Concerns that the proposed sample size is insufficient to address the research question.
  • Reviewers may think the researcher is overreaching with the proposed sample size.

Addressing Underpowering

  • Increase Sample Size: Suitable for large grants (e.g., NIH R series grants like R01, R21).
  • Scale Back Research Question: Suitable for smaller grants (e.g., internal pilot grants, NIH K series awards).

Example: Small Pilot Clinical Trial

  • Scenario: Pilot intervention with 40 participants (20 treatment, 20 control).
  • Goal: Demonstrate feasibility and preliminary efficacy on a primary outcome.
  • Issue: Maximized sample size due to budget constraints.
  • Researcher solves for minimum detectable effect size, finding N = 40 allows detecting Cohen's d = 0.91.
  • This large effect size is unlikely, raising concerns of underpowering.

Solutions for Underpowered Studies

  • Increase Sample Size: May not be feasible for smaller grants with short time windows.
  • Tone Down Goals: Focus on feasibility rather than efficacy.
    • Feasibility aspects: patient recruitment, participant retention, outcome measurement.

Overpowered Studies

The Flip Side

  • Reviewers may critique that a study is overpowered.
  • More power is generally better, but issues can arise with secondary data analysis.

Secondary Analysis of Large Datasets

  • Scenario: Analyzing electronic health records or administrative claims data with thousands or millions of observations.
  • Even small effects can be statistically significant due to the large sample size.
  • Risk of over-interpreting small differences.

Clinical Significance

  • Emphasis on clinical significance or practical importance is crucial with big data.
  • A statistically significant but tiny effect may not be meaningful.
  • Power section should focus on clinical significance in addition to statistical significance.

Conclusion

  • Statistical power should be "just right" in relation to aims and conclusions.
  • Seek feedback from mentors and peers to ensure appropriate power.