appraising and applying evidence

random error cause

chance variations in measurements, participant differences that aren’t controlled, environmental factors that aren’t constant

how to reduce random error

repetition, larger sample size

random error measurement

p value

systematic error

bias, consistent, repeatable error associated with flawed study design either intentional, inadvertent, or unavoidable

random error accuracy/precision

accurate but not precise

systematic error accuracy and precision

precise but not accurate

internal validity

correct study design, were variables measured in a reasonable way?

external validity/generalizability

how well results of this study can be applied to larger population

bias can occur…

at beginning of study, during study, or at completion

non probability sampling

probability that a subject is selected is unknown and may reflect selection biases of the person doing the study

does not fulfill requirements of randomness needed to estimate sampling errors

probability sampling

each member of the population has a known chance of being selected

types of probability sampling used in medicine

simple random, systematic, stratified, cluster

non probability sampling usage

exploratory and qualitative research

representative sample

contains all attributes of the population in the same proportion they exist in the population

allows one to generalize and make inferences

simple random sampling

each unit of population has equal probability of being included in the sample and probability is independent of previous drawings

simple random sampling method

table of random numbers or computer generated list of random numbers

systematic sampling

population is arranged according to some order and units are selected at regular intervals

cluster sampling

selecting intact groups, not individuals, within defined population sharing similar characteristics

selection bias

selecting or retaining individuals, groups, or data for a study leads to a sample not truly representative of larger population study aims to understand

admission rate bias

hospital patients tend to be sicker

healthy worker effect

people who can work are generally healthier than the overall population

late look bias

excludes patients who have already died or recovered and makes disease look more or less severe

typically in cross sectional studies

non response bias

people that actually respond differ from those who don’t

volunteer bias

people who volunteer for a study differ in significant ways from those who do not

confounding bias

outside factor affects dependent and independent variables

publication bias

typically only successful studies are published

reverse causality bias

condition and cause being investigated may actually be reversed

placebo effect

thinking you’re getting treatment makes symptoms improve

allocation bias

people who actually receive the treatment are impacted by researchers

cointervention

caretakers give additional treatment

3 questions for every study

are results valid, what are results, how can i apply results to my patients

are results valid further questions

were patients randomized and in study groups with respect to prognostic variables?

was group allocation sealed?

what extent was study blinded?

was follow up complete?

was trial stopped early?

were pts analyzed in groups they were first assigned?

groups that should be blind to treatment assignment

patient, clinician, data collectors, adjudicators of outcome, data analysts

absolute risk

risk of an event

absolute risk calculation

# patients experiencing adverse events / # participants

absolute risk reduction

absolute difference in rate of events between control and treatment groups

absolute risk reduction calculation

control group risk - treatment group risk

relative risk

ratio of risk of an event among treatment group to risk among the control group

intention to treat

participants assigned to group stay in group no matter what they actually do

what are the results further questions

how large was the treatment effect?

how precise was the estimate to the treatment effect?

relative risk reduction

reduction rate of the adverse event in the treatment compared to the rate in the control group

MC reported measure of treatment effects

relative risk reduction calculation

absolute risk reduction / risk of control group OR 1 - relative risk

p value

probability value, measure of probability that a result is purely due to chance

p value cutoff

0.05 or greater than 5% means statistically insignificant/due to chance

confidence intervals

range of values within which one can be confident the true effect lies

narrow confidence interval

point estimate is more precise to true effect

factors affecting confidence interval width

number of patients, frequency of study outcome

how can i apply results to my patients further questions

were the study patients similar to mine?

were all patient important outcomes considered?

are likely tx benefits worth potential harm and cost?

number needed to treat

how many patients must be treated with an intervention to produce one positive outcome or prevent one negative outcome

number needed to treat calculation

100 / ARR

number needed to harm

how many patients must be treated for one to experience a particular adverse event

grandmother test

outcomes that would be valued by the average person (like someone’s grandmother) are clinically important

attributable risk

risk of harm in control - risk in treatment group

risk of harm calculation

find # individuals experiencing adverse effect in each group

number needed to harm calculation

find attributable risk and take reciprocal (1 / AR)

best diagnostic study

prospective cohort study

spectrum bias

if a dx test is evaluated only in a narrow or unrepresentative group its performance may look better than it would in a broader, more typical clinical setting

verification bias

not all patients recieve the reference (gold) standard test to confirm the presence/absence of disease

likelihood ratio

how many times more/less likely a test result is to be found in diseased vs non diseased people

likelihood ratio calculation

p(person w/condition having test result) / p(person w/out condition having same test result)

benefit of likelihood ratio

incorporates sensitivity and specificity of test into single measure

sensitivity

p(test is positive) given pt has the disease

specificity

p(test is negative) given pt does NOT have the disease

meaning of highly sensitive test

low false negative rate

meaning of highly specific test

low false positive rate

positive likelihood ratio

p(positive test in person w/disease) / p(positive test in person w/out disease)

positive likelihood ratio calculation

sensitivity / 1- specificity

high positive likelihood ratio meaning

positive test is much more likely in someone w/disease than someone w/out

negative likelihood ratio

p(negative test in person w/disease) / p(negative test in person w/out disease)

meaning of low negative likelihood ratio

negative test result is less likely in someone with disease compared to someone w/out