Uncertainty in Economic Evaluation
Uncertainty in Economic Evaluation
Introduction
- Economic evaluations rely on input variables that are often estimates.
- It's important to measure and quantify the uncertainty associated with these inputs.
- Sensitivity analysis helps understand how susceptible the evaluation is to input uncertainty.
Key Terms
- Inputs: Variables used in the model (e.g., quantities, prices, probabilities).
- Examples include those used in cost-effectiveness analysis (CEA), return on capital employed (ROCE), and cost-benefit analysis (CBA).
- Outputs: Outcomes derived from the model.
- Examples include expected treatment costs, probability of an outcome, or measures of multiple outcomes.
- Expected costs
- Expected years of life gained
- Incremental cost-effectiveness ratio (ICER)
- Net monetary benefit (in CBA)
Sensitivity Analysis
- Purpose: To identify the most important inputs and how they impact the model's output.
- Aims to quantify the overall degree of uncertainty in the model.
- Traditional statistical methods (p-values, confidence intervals) are not applicable due to the lack of individual data points.
- Instead, the model is modified to observe changes in the outcome.
Types of Sensitivity Analysis
Deterministic Approach
- Choose specific values (minimum, maximum, or +/- percentage change) for one or more parameters.
- Keep other parameters constant.
- Observe the impact on the model's output.
Probabilistic Approach
- Assign probability distributions to input parameters (e.g., normal, binomial, Poisson).
- Use simulations (e.g., Monte Carlo) to compute the ICER.
- This approach was demonstrated with a Markov model, running it repeatedly to observe the range of outcomes.
Terminology
- Different fields may use different terms for the same concept (e.g., income/margin in finance).
- Variability: Variation due to chance (random).
- Also known as stochastic uncertainty or first-order uncertainty.
- Heterogeneity: Differences between patients that can be explained by patient characteristics (e.g., demographics).
- Uncertainty: Lack of knowledge about the true value of an input parameter.
- The need to estimate values based on research or sampling.
Types of Uncertainty
Parameter Uncertainty
- Uncertainty around the value used for a specific input parameter.
- Arises because the true value is unknown and must be estimated.
- Variability in estimates (e.g., mean cost in a sample) due to sampling.
- Measured by standard error (SE).
- SE=SD (Standard Error does not equal Standard Deviation)
- Standard error is calculated as mean divided by standard deviation.
- Larger sample sizes can reduce parameter uncertainty, leading to smaller standard errors and more precise estimates.
Model Uncertainty
- Uncertainty about whether the model accurately represents reality.
- All models are simplifications and cannot capture every relevant element.
- Internal and external validity are important considerations.
- Balancing realism with feasibility is key.
- A more realistic model is not always better if it doesn't significantly influence decisions.
- Model uncertainty arises from uncertainty about the assumptions and elements included in the model.
Additional Points
- Consistency in sampling methods (random sampling) is crucial for accurate estimates.
- Limitations of models are typically discussed in the limitations section of research.