Uncertainty in Economic Evaluation

Uncertainty in Economic Evaluation

Introduction

  • Economic evaluations rely on input variables that are often estimates.
  • It's important to measure and quantify the uncertainty associated with these inputs.
  • Sensitivity analysis helps understand how susceptible the evaluation is to input uncertainty.

Key Terms

  • Inputs: Variables used in the model (e.g., quantities, prices, probabilities).
  • Examples include those used in cost-effectiveness analysis (CEA), return on capital employed (ROCE), and cost-benefit analysis (CBA).
  • Outputs: Outcomes derived from the model.
  • Examples include expected treatment costs, probability of an outcome, or measures of multiple outcomes.
    • Expected costs
    • Expected years of life gained
    • Incremental cost-effectiveness ratio (ICER)
    • Net monetary benefit (in CBA)

Sensitivity Analysis

  • Purpose: To identify the most important inputs and how they impact the model's output.
  • Aims to quantify the overall degree of uncertainty in the model.
  • Traditional statistical methods (p-values, confidence intervals) are not applicable due to the lack of individual data points.
  • Instead, the model is modified to observe changes in the outcome.

Types of Sensitivity Analysis

Deterministic Approach
  • Choose specific values (minimum, maximum, or +/- percentage change) for one or more parameters.
  • Keep other parameters constant.
  • Observe the impact on the model's output.
Probabilistic Approach
  • Assign probability distributions to input parameters (e.g., normal, binomial, Poisson).
  • Use simulations (e.g., Monte Carlo) to compute the ICER.
  • This approach was demonstrated with a Markov model, running it repeatedly to observe the range of outcomes.

Terminology

  • Different fields may use different terms for the same concept (e.g., income/margin in finance).
  • Variability: Variation due to chance (random).
    • Also known as stochastic uncertainty or first-order uncertainty.
  • Heterogeneity: Differences between patients that can be explained by patient characteristics (e.g., demographics).
  • Uncertainty: Lack of knowledge about the true value of an input parameter.
    • The need to estimate values based on research or sampling.

Types of Uncertainty

Parameter Uncertainty
  • Uncertainty around the value used for a specific input parameter.
  • Arises because the true value is unknown and must be estimated.
  • Variability in estimates (e.g., mean cost in a sample) due to sampling.
  • Measured by standard error (SE).
  • SE≠SDSE \neq SD (Standard Error does not equal Standard Deviation)
  • Standard error is calculated as mean divided by standard deviation.
  • Larger sample sizes can reduce parameter uncertainty, leading to smaller standard errors and more precise estimates.
Model Uncertainty
  • Uncertainty about whether the model accurately represents reality.
  • All models are simplifications and cannot capture every relevant element.
  • Internal and external validity are important considerations.
  • Balancing realism with feasibility is key.
  • A more realistic model is not always better if it doesn't significantly influence decisions.
  • Model uncertainty arises from uncertainty about the assumptions and elements included in the model.

Additional Points

  • Consistency in sampling methods (random sampling) is crucial for accurate estimates.
  • Limitations of models are typically discussed in the limitations section of research.