EBP Exam Review Part 2

Sensitivity

How well a test finds people with (+ test result) the condition.

High Sensitivity means?

High sensitivity → negative result rules OUT the condition (SnNOut)=High sensitivity + Negative test result + Rule Out condition

A/A+C

Sensitivity Example

The Ottawa Ankle Rules — if negative, you likely don't need an X-ray (rules out fracture).

Specificity

How well a test finds people without (- Test result) the condition

High Specificity means?

High specificity → positive result rules IN the condition (SpPIn) = High specificity + Positive Test result + Rule In Disorder

D/B+D

Specificity Example

Lachman Test — if positive, strongly points to ACL tear.

What happens if a test has low sensitivity?

Not good at detecting people with the condition, false negatives are frequent

What happens if a test has low specificity?

Not good detecting people without the condition, false positives are frequent

Positive Predictive Value

% of people with a positive test who actually have the condition.

A/A+B

Negative Predictive Value

% of people with a negative test who actually don’t have it.

D/C+D

Why do PPV and NPV have limits?

Dependent on the prevalence of the condition, samples studied, and differences among them

PPV/NPV example

Adhesive capsulitis has low prevalence → Even a good test will have low PPV because few people actually have the condition.

Lower PPV, and higher NPV, more people will test negative

Positive Likelihood Ratio

How much more likely a positive result is in someone with the condition vs without.

Sensitivity/1-specificity

Negative Likelihood Ratio

How much more likely a negative result is in someone with the condition vs without.

(1-sensitivity)/specificity

Positive Likelihood Ratio Values

Large: >10

Moderate: 5-10

Small but important: 2-5

Negligible: 1-2

Negative Likelihood Ratio Values

Large: <0.10

Moderate: 0.10-0.20

Small, but important: 0.20-0.50

Negligible Change: 0.50-1

What value represents 50/50 chance of a coin flip increasing or decreasing a diagnosis?

LR=1

Pre-Test Probability

Your estimated guess before testing (often from prevalence).

Post-Test probability

Updated probability after using the test results.

How does likelihood ratio interact with the pre and post test probabilities?

LRs help convert pretest odds → posttest odds using math

Use the following 2 x 2 table to calculate the sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios for a hypothetical diagnostic test:

Odds Ratio

Used in retrospective studies, the "odds" that an individual with the prognostic risk factor will develop the outcome of interest

(a/b) / (c/d)

Odds Ratio Values if outcome is Negative

Odds ratio >1 = Odds problem will develop

Odds ratio <1= Odds are against the adverse outcome

Odds Ratio Values if outcome is positive

Odds Ratio >1 = Odds are in favor of positive outcome

Odds ratio <1 = Odds against positive outcome

Relative Risk

Used in prospective studies, risk for developing adverse outcome in patients with the risk factor vs not

(a/a+b) / (c/c+d)

Relative Risk Values

RR >1 = increased risk of an adverse outcome

RR <1 = decreased risk of an adverse outcome

Odds Ratio Example

people able to walk independently after stroke have nearly 4x the odds of returning to work compared to people who don't walk independently

Relative Risk Example

people receiving pre-operative physical therapy prior to elective cardiac surgery have half the risk of developing pulmonary complications as people who don't

What does it mean when the 95% Confidence interval includes 1? How about a number different than 1?

The odds ratio is statistically meaningful and useful because the confidence interval does not include 1. If it included 1, there is one possibility for true value of the odds ratio being 50/50

Intention to treat analysis

Used to handle dropouts in a study, allows researchers to analyze outcomes if all the subjects remained in the study in their assigned groups (and did not leave / drop out). It preserves the original distribution of the subject characteristics and maintains the sample size

Why Magnitude of Treatment Effect is way more important than a P-Value?

The results of a study may be statistically significant however, if the actual change produced in the subjects is not meaningful to them or the therapist, it doesn't really matter

Including the magnitude of the treatment effect in a study's results allows readers to determine the clinical relevance of the findings

Example: 50 ft increase in 6MWT for heart failure = statistically significant, but might not matter if the patient needs to walk 200 ft to the bathroom.

Effect Size

Effect Size is the magnitude of the difference between average scores between groups or between a post-intervention and pre-intervention score

Absolute Effect Size

actual difference between the final measure of the outcome in the intervention

Group and the final measure of the outcome in the control group Final Distance walked Intervention Group (100 ft) - Control Group (50 ft) = 50

Standardized Effect Size

Uses the raw scores divided by the variability in the data set by the standard deviation, Based on starting scores

strength measured with dynamometer vs weights

Want >.80

Standardized Response Mean

Based on changes over time

Mean of Change Scores / Standard Deviation of Change Scores

"good" SRM > 1.0

Effect Size Values

0-0.49 = Small effect

.50-.79 = Moderate effect

>.80 = Large effect

Minimal Clinically Important Difference

the minimal level of change required in response to an intervention before the outcome would be considered worthwhile, should exceed SEM for outcome of interest

Benefit Increase

How much does the intervention increase the risk of a desirable event (resolution of symptoms)

Relative Benefit Increase

%therapy group w/ outcome - %control with outcome / % control with outcome x100

Absolute Benefit Increase (ABI)

% therapy group w/ outcome - % control group w/ outcome

Risk Reduction

How much does the intervention reduce the risk of an unwanted event (injury)

Relative Risk Reduction (RRR)

(% risk in control group - % risk in treatment group) / % risk in control group

Indicates how much the risk is reduced in the treatment group compared to the control group

Absolute Risk Reduction (ARR)

% risk in control group - % risk in treatment group

What are the 3 reliability tests?

ICC, Kappa and Cronbach's Alpha

Chronbach's Alpha

Measures internal consistency and the relationship between items in a questionnaire, want value between .70-.90

Minimal Detectable Change Vs Minimal Clinically Important Difference

MDC: The smallest change that's not due to error

MDIC: The smallest meaningful change that matters to the patient

Qualitative Paradigm

Involves interpretivism and constructivism

Stories, meaning, experience, Understand and explore

Quantitive Paradigm

Involves Positivist/Post Positivist

Numbers, statistics, cause-effect, Test and measure

Phenomenology, Ethnography, Case Study, Grounded Theory are all?

Qualitative Research Designs

Phenomenology

Describe a phenomena by exploring it from participants perspective

How do PT students experience their first clinical rotation?

Ethnography

Develop an in-depth understanding of culture from a participants point of view

How do PT students learn differently in inpatient vs outpatient settings?

Case Study

Through study of a single organization, situation or patient

What can we learn from a PT student's journey through school?

Grounded Theory

Develop theories based on real-world data

What factors affect learning under different mentorship styles?

What is the most common qualitative method?

Interviews: Get deep and detailed information about participants' thoughts, feelings, and experiences

Observational Method for Qualitative Research Method

Researcher watches participants in their natural environment

Hawthorne Effect

People act differently when they know they're being watched or part of a research study

Artifact Analysis

Examining existing materials:

Written (journals, policies)

Visual (videos, photos)

Audio (interviews, recordings)

Learn background or context about a person, group, or system

Quantitative vs. Qualitative Data Analysis

Qualitative

Data Type Words, visuals

Goal Build or refine a theory

Timing Analyze during collection

Output Themes, narratives, models

Quantitative

Data type Numbers

Goal Confirm a theory

Timing Analyze after collection

Output Statistical results

Floor Effect

Unable to differentiate scores at lower end and scores become "stacked up" at low end

Example: Inability to register a further decline in health status in an individual with amyotrophic lateral sclerosis

Ceiling effect

Unable to differentiate scores at higher end and scores become "stacked up" at high end

Inability to register a further gain in health status in a person recovering from Guillain-Barre Syndrome

Why can floor and ceiling effects be limited?

Floor and ceiling effects may limit the utility of a self-report instrument because of their inability to detect additional changes in status

Clinical Prediction Rules

Systematically derived and tested to make predictions on diagnostic categories , prognostic estimates and treatment responses

CPR Creation Process

1. Derivation (Identification of factors with predictive power) - 2. Validation (Evidence of reproducible accuracy) - 3. Impact analysis (Evidence that rules changes physician behavior and improves patient outcomes and reduces costs)

Narrow Validation

application of rule in a similar clinical setting and population as in step 1

Broad Validation

application of rule in multiple clinical settings with varying prevalence and outcomes of disease

Clinical Prediction Guidelines

Statements to assist practitioner and patient decisions in specific circumstances while being systematically developed

CPGs reflect?

Current best evidence, expert clinical judgement and patient opinion and perspective

How are CPGs developed?

Government agencies and professional societies