HYMS Year 2 Critical Appraisal

What are background questions?

> general knowledge inquiries about patients condition that ask who, what, when , how, why

> eg what causes? how does? why does?

What are foreground questions?

> specific knowledge you require to manage patient condition

> question created through PICO framework

What is a cross-sectional study?

> descriptive study that provides a snapshot of the frequency of a disease at a given time

> used to generate a hypothesis as multiple exposures can be identified

> surveys are used to ID prevalence

What biases are seen in cross-sectional studies?

recall, selection and survivor

Strengths of cross sectional studies

> Cheap and quick to conduct

> Data on all variables collected at once

> Multiple outcomes and exposures can be studied

> Good for descriptive analysis and generating hypothesis

weaknesses of cross-sectional design

> Can't determine cause effect relationships

> Not suitable to study rare diseases or diseases with a short duration

> Can't determine incidence

> Susceptible to bias- low response and misclassification due to recall bias and selection bias

Best study design for diagnosis

cross sectional

What is a case-control study?

> groups with disease and without disease (2 different outcomes) are observed

> look back on any exposures (risk factors)- retrospective study- and whether they have any associations

Strengths of case-control study

> Efficient use of resources as you don't need a long time period, you just ask someone with a disease some questions

> Useful in studying rare diseases and those with a long latency period

> Can study multiple risk factors

Weaknesses of case- control studies

> Susceptible to selection and recall bias

> can be difficult to measure the exposure

> reverse causality- effects of the disease on the risk factor > bad for IDing rare exposure

Best study design for aetiology

> case-control study

> cohort study

What is a cohort study?

> observational study that compares incidence of disease over time where half have the exposure and half don't- prospective study

> historical- half have exposure, half don't and look back in time

Strengths of cohort studies

> can ask participants about multiple outcomes eg does diabetes lead to CVD, cancer,

> useful for studying rare exposures eg asbestos

> Historical cohort studies are effective in the investigation of diseases that have a long development time

> Reduces risk of survivor bias

Best study for prognosis

prospective cohort study

What are RCTs?

experimental studies where participants are randomly allocated into intervention and control groups, and compare the outcomes of each

Strengths of RCTs

Able to establish causation

Minimizes biases

One treatment can be directly compared to another

Publishable

Weaknesses of RCTs

> expensive

> selection and performance bias (eg those going to surgical intervention know they're having surgery so behave different)

> ethical limitations

> generalisability

best study design for treatment

RCTs

systematic review of RCTs

best study design for evaluation

survey

qualitative research

What is a meta-analysis?

a statistical technique where results from multiple individual studies for a specific question are combined and re-analysed and a summary estimate is given (often a forest plot)

What is a systematic review?

a rigorous summary of all the research evidence that relates to a specific question

What is the advantage of meta-analysis?

> overall sample size is increased, which can yield more precise estimates of benefits and harms of treatments

> top of evidence hierarchy

Strengths of systematic review

> include all available evidence to answer a question -unpublished and non-English

> increases total sample size therefore certainty and precision better

> indicates heterogeneity among studies

> can indicate need for further research

> permits sub-group analyses- how do different patient groups respond to intervention

> allows sensitivity analyses

What does critical appraisal assess?

Bias• Applicability (what type of data is collected from what type of ppl?) • Limits (is it unrepresentative sample size) • Overall value (useful for pt care?

steps of critical appraisal

> validity of results

> what are the results

> can they be applied to patient care

Why is the randomisation of treatments important (RCT)

confounidng fxs generally equally distributed between treatment arms, ethical as treatment not decided by a person, eliminates systematic bias in allocation of intervnetiins

How to randomise (RCT)

> centralised computer randomisation for multi-centre trials

> smaller trials may use an independent person eg hospital pharmacy to manage the randomisation

Why is it important that both groups are treated the same apart from the intervention? (RCT)

having more consultations, more tests etc could cause a difference in outcome rather than the intervention itself

Withdrawal in RCT

> stopped the intervention

> or withdrew from the trial

Loss to follow up

not providing data at a particular point

should be less than 20%

Why is it important that those who enter the trial are accounted for at its conclusion?

> losses may bias the results if loss more from one group than other

> reduces sample size

> small losses reduce power of study if the outcome of interest in a trial is infrequent

> patients should be analysed in the group they were randomised to even if they stop treatment although they are effectively a control- INTENTION TO TREAT

In what outcome in blinding important?

subjective outcomes- eg measurements of symptoms or functions, or patient self report

blinding is unimportant in objective measures eg death

What is risk?

a quantified uncertainty

What does relative risk tell you in RCT?

> how many times more likely an event will occur in a treatment group compared to control

> less than 1 means treatment decreases risk of outcome

> greater than one means it increases risk of outcome

What does relative risk reduction tell you in RCT?

tells us the reduction of the rate of the outcome in the treatment group relative to that in the control group

What does absolute risk reduction tell you in RCT?

tells us the absolute difference in the rates of events between the two groups- indication of baseline risk and treatment effect

What does numbers needed to treat tell us in RCT?

the number of patients you need to treat with experimental therapy to prevent 1 additional bad outcome

1/ARR

What is the 95% confidence interval?

range of values of the study sample within which we can be 95% sure the true value lies

What are some disadvantages to providing an intervention in RCTS?

> adverse effects of meds, risk of cancer in imaging

> additional treatment costs- new equipment, staff etc

> opportunity cost eg extra clinic appointment means loss of work for patient

Assessing external validity of RCTs

> checking whether the patient you have is similar to that in the study

> whether the treatment is feasible in my setting

Internal validity

the degree of confidence that the causal relationship you are testing is not influenced by other factors or variables.

What is one of the main problems with observational studies?

Recall bias- patients may both over and under estimate, which is a form of error, making relationships harder to identify

How can how an outcome is measured affect validity of cohort studies?

how the outcome is measured must be kept the same in exposed and non-exposed groups as this may lead to systematic bias ie identifying the outcome better in the exposure group due to better surveillance

What study designs is confounding seen the most?

observational studies- cohort, case-control, cross-sectional

Appropriate techniques to account for confounding in observational studies?

> restriction

> stratification

> matching

> multiple regression techniques

look out for whether something has been 'adjusted for' (cohort) or 'matched' (case-control)

What is statistical power?

to conclusively reject a null hypothesis when it is in fact false

increases with sample size

How can loss to follow up (attrition) affect observational studies?

> Attrition bias- If the participants lost to follow-up differ systematically from those who remain in the study, the results may be biased.

> reduced statistical power due to sample size, leads to type 2 error (null accepted)

Why is having too long a follow up period bad?

> wastes money

> unethical as it involves data collection of no scientific value

What is a weak value for relative measure of effect (odds ratio/relative risk)

less than 1.5 considered weak

Relative risk

risk in exposed/risk in unexposed

What studies usually report relative risk? Odd ratio?

> cohort

> case control

Why is it good to have 2 authors looking at the papers and assessing quality in systematic reviews?

> independent assessment can help counteract unconscious bias

> less likely errors are made

> increases transparency of process therefore unbiased by using pre-agreed criteria

> less chance of missing a relevant trial

When are studies acceptable to combine in systematic reviews?

when studies are similar (i.e. they used similar outcomes). If studies are very dissimilar, then results should not be combined.

- Because combined results provide stronger evidence, but only when appropriate to combine.

What are features of a good search strategy in systematic reviews?

hand-searching

conference proceedings

contacting authors

non-English papers

Bradford Hill criteria of causality

> temporal- risk precedes outcome

> specific- risk factor is specific

> consistent- seen in different profiles in different parts of world

> strong

> dose-response: greater exposure leads to greater effect

> experimental evidence

> plausible- knowledge about mechanism of disease

> coherence- between epidemiological and lab findings

> analogy

reverse causality

> biology of the disease/ patient behaviour modifies risk factor. Before disease is diagnosed, risk factors are altered. > eg decreased weight= lung cancer (lung cancer decreased weight before it was diagnosed)

> eg decreased alcohol= angina (pain began, behaviours altered before diagnosed)

Weaknesses of cohort studies

> expensive and timely

> difficult in measuring risk

> attrition, recall and selection bias

pyramid of evidence (top to bottom)

Weakness of meta-analysis

Publication bias. Researchers may not select all relevant studies, leaving out negative or non-significant results. Data may be biased because it only represents some of the data and incorrect conclusions are drawn.

Why is having just English language reports not favourable in systematic reviews and meta-analyses?

> reports that are originally conducted in non-English but then translated and published in English reports larger effect sizes of interventions compared to those not translated- systematic review then overestimates the true effect of intervention

What is intention to treat analysis?

participants in a trial are analysed within the group they were allocated in randomisation, even if they have not completed the trial or changed treatment

Importance of random allocation

t to ensure that the observed effects are uninfluenced by selection bias and that confounding factors are generally equally distributed between the treatment arms

Importance of placebo arm

ensures that any effect found is attributable to the active intervention and not to non-specific intervention of healthcare or healthcare professional

Features of case-control that distinguishes them from other studies

> retrospective

> at the start people with the condition are matched with those without from the same population to try and understand the causes of the condition

Define confounding

an apparent relationship between an exposure and and outcome that is false; that is, it implies a causal relationship that actually does not exist due to a shared relationship with something else

Purpose of controls in a case-control study

> offers a comparison group with people from the same population but without outcome of interest

> Allows a baseline value of the exposure in a normal population

Problems with restriction technique for confounding

> less data

> difficult when you have other possible confounders