Module 3: Clinical Prediction Rules (CPR)

what is a CPR?

clinical (decision making) tool which contains the best combination of individual components from history, physical examination, and/or lab tests that can help the clinician make a diagnosis, establish a prognosis, or select the appropriate treatment

what can CPRs do?

classify pt into diagnosis

predict future outcomes

identify who will best respond to interventions

purpose of a CPR

increase the efficiency and accuracy of clinical assessment

promote tx/diagnosis based on best available evidence

reduce variation in practice

promote efficient use of resources and reduce unnecessary costs

simplify for a complex diagnosis, or inc accuracy when clinical stakes are high

what are the types of CPRs?

diagnostic

treatment classification (more common in PT)

purpose of diagnostic CPR

identify patient with a disorder

example of diagnostic CPR

Ottawa Ankle Rules - rule in/out an ankle or foot fx and decide if X-ray needed

purpose of tx classification CPR

identify pts that will respond favorable to a tx

example of tx classification CPR

lumbar spine manipulation rule

what are the steps to developing a CPR?

1. derivation - identify predictors and create a rule

2. validation - apply the rule in a separate, independent samples

3. impact analysis - assess how the rule impacts clinical behavior and outcome

what are appropriate study designs for CPRs?

prospective cohort study

randomized control trial

what types of CPRs are prospective cohort studies appropriate for?

diagnostic/prognostic CPRs

tx based CPRs in derivation stage

cross-sectional acceptable for diagnostic CPR

what types of CPRs are randomized control trials appropriate for?

tx based CPRs in validation stage

Level 1 Evidence for CPR Studies

prospective validation in a different population and 1 impact analysis with change in clinician behavior and improved outcomes

rules can be applied widely w/ confidence to change clinician behavior and improve patient outcomes

Level 2 Evidence for CPR Studies

1 large prospective study w/ broad spectrum of patients and clinicians or several smaller settings that differ from each other

rules can be used in various settings w/ confidence in their accuracy

Level 3 Evidence for CPR Studies

1 narrow prospective sample

rules clinicians may consider using w/ caution and only if patients in the study are like those in the clinician's clinical setting

Level 4 Evidence for CPR Studies

derived but not validated or validated only in split samples, large retrospective databases, or by statistical techniques

rules need further evaluation before they can be applied clinically

Questions to consider when asking: Is the CPR in derivation stage valid?

1. was sample operationally defined and representative of the target population?

2. were all relevant predictor factors included in development process?

3. was sample large enough to accommodate all predictor factors in the derivation process?

4. were the outcomes of interest operationally defined?

5. were the outcome assessors blinded to the pt's status on the predictive factors?

6. was data collected on all subjects enrolled in the study? (attrition/loss to follow up)

how to determine whether or not the sample was operationally defined?

clearly articulated inclusion/exclusion criteria

potential to develop outcome of interest

how to determine whether or not the sample was representative of the target population?

represent population CPR will be applied on

study conducted in population CPR is intended

how to determine if all relevant predictor factors were included in the development process?

what were the potential predictors included in CPR derivation?

how were they selected?

do they make sense clinically?

reliable and valid measures for CPR?

operationally defined?

how are predictor variables analyzed?

1. identify various that significantly predict outcome

2. then put them together using multivariable regression analysis and see which variables still predict outcome when used together

what should you look for to determine if the sample size is large enough to accommodate all predictor factors in derivation process?

power or sample size analysis in methods

note: regression analyses w/ multiple predictor variables require large sample sizes to accurately provide predictions

what type of error can an inadequate sample size lead to?

type 2 error

how to determine if the outcomes of interest were operationally defined?

look for objective criteria on who is considered "positive" on the outcome of interest and who is considered "negative" on the outcome of interest

why do those identifying patient's status on the outcome of interest must be blinded to the patient's status on the predictive factors?

to minimize assessor (tester) bias

what is important to look for if there is attrition?

were the participants who were lost to follow-up different from those who remained in an important way

rule of thumb for attrition

>20% questions study validity

Questions to consider when asking: is the CPR in validation stage valid?

1. did the study compare results from the CPR to an established criterion or outcome of interest?

2. were all subjects evaluated w/ the criterion test (gold standard) or the outcome of interest?

3. were assessors of the CPR blinded to the results from the criterion test and vice versa?

4. were findings confirmed in second independent group of subjects?

5. was the CPR validated in population other than what it was intended for?

what is important to compare the outcome based on the CPR to?

outcome on the established criteria/gold standard

who should be evaluated on the criterion test?

all subjects enrolled in study must be evaluated on outcome of interest or the criterion test

if not --> bias

why is it important for findings in a validation study to be confirmed in a second independent group of subjects?

to increase generalizability of the CPR

note: second group may not be included in study so would need to find it through a literature search

benefit of validating the CPR in population other than what it was intended for

increase generalizability

what factors could lead to a CPR failing at the validation stage?

rule inadequately developed

drastic differences in development population and validation population

inadequate sample size or statistical power to detect effects

what factors determine use of CPR in clinical practice?

familiarity w/ tests/procedures on the CPR

confidence in rule's usefulness

user-friendliness

if impractical or not easy to implement in clinical settings --> clinicians will not use it

impact of CPR results in the derivation stage

indicate most important predictors associated w/ condition of interest or tx classification

indicate accuracy of CPR in classifying or diagnosing a condition

results may not directly apply to clinical decision making

several go through derivation but many fail validation

impact of CPR results if it survives validation

indicate accuracy of CPR in predicting outcome of interest

accuracy of CPR can impact clinical decision making

what is a CPR impact analysis?

randomize clinics to use the rule or not w/ cluster of RCTs and collect data on all applicable pts in both settings and assess outcomes

important info to obtain in impact analysis

do clinicians use the rule - if so, how often

is it acceptable and feasible

is it effective in improving clinical or healthcare utilization outcomes

example CPR that has underwent impact analysis

Ottawa Ankle Rules

what to ask when considering applying results of a CPR to your pt?

is pt similar to participants

is CPR feasible

will the CPR inform better tx choices and are outcomes in line w/ pt preference

what stage are most CPRs pertinent to PT practice in?

derivation stage