Exam 2 - Systematic Review/Meta-analysis Appraisal

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/48

encourage image

There's no tags or description

Looks like no tags are added yet.

49 Terms

1
New cards

Meta-analysis (Definition + usage of it)

Def: Systematic Review with additional
statistical techniques that combine the
results of multiple quantitative studies to
provide a more meaningful measure of
the effects



  • Can use it, b/c someone already appraised it 

  • The only thing is that you’ll have to appraise it again 


2
New cards

Scoping Review

Def: Preliminary assessment of the evidence
to establish the size and nature of
evidence base; no formal quality
assessment


  • No statistical analysis 

  • It gives scoping sense of what is there


3
New cards

Narrative Review (definition + usage)

Def: Articles are discussed around an organizing theme in a user-friendly format; methodology may not be formal



  • Opinion and is biased 

  • No statistical analysis in it


4
New cards

Umbrella Review (definition)

Def: Review of Reviews; summarizes the
results from multiple systematic reviews
on a topic; useful when research topic is
broad and substantial amount of
evidence exists.

5
New cards

Cochrane database of SRs

They are gold standard b/c they adhere to the standard

6
New cards

Meta analysis have ____ plots

forest plots

7
New cards

Good vs Poor Systematic review makes

  • Good Systematic Review makes a more powerful and informative result than any single study can give 


  • Poor Systematic Review makes multiplied bias and multiplied flaws of analysis and conclusion


8
New cards

What does a Good Systematic review have

  • A focused research question

  • A comprehensive search of the literature

  • A careful appraisal of applicability and validity of each study

  • An appropriate analysis of the cumulative results

    • Quantitative and Qualitative depend on the heterogeneity
      of the studies


9
New cards

Contents of a Meta-analysis

  1. Introduction (background and objective)

  2. Methods

  • Study selection criteria and search strategy

  • Study review and analysis

  1. Results (description of studies, methodological quality, and
    results of meta-analysis if possible)

  2. Discussion and conclusions

  3. Standard tables and figures: showing characteristics of the included studies

  4. Results of the included studies

  5. Log of the studies that were excluded

  6. References


10
New cards

How to know if the SR is external valid/applicable?

  • if the study’s purpose is relevant to my clinical question

  • if the study population (sample) sufficiently similar to your patient to
    justify expectation that your patient would respond similarly to the
    population

  • inclusion and exclusion criteria (related to which studies
    were included) clearly defined and are studies that would answer
    your clinical question likely to be included

  • if the types of interventions investigated relevant to your clinical
    question (feasible and explained in enough detail to be replicable)

  • if the outcome measures relevant to the clinical question and are
    they conducted in a clinically realistic manner

    • the outcome measures spanning the ICF model



11
New cards

How to know if the study is internally valid (overall good study)?

  • if literature search is comprehensive (consider publication bias from protocol)

  • If there is an objective, reproducible, and reliable method used to judge the quality of the studies in the systematic review

  • if there is a standardized method used to extract data from studies
    included in the systematic review

  • if there was clinical heterogeneity assessed to determine whether a meta-
    analysis was justified

  • if meta-analysis conducted and statistical heterogeneity was assessed

  • if methods used to report the results of the SR is good

  • if systematic review inform clinical practice related to my clinical question in a good manner


12
New cards

What are search terms

single word, phrase, question, or voice input that a user enters into a search engine, database, or library catalog to find information


example: critical care, physical therapy, mobilization, exercise, physiotherapy or mobility, mobility

13
New cards

Pedroscore interpretation

  • High Quality = PEDro Score 6-10

  • Fair Quality = PEDro Score 4-5

  • Poor Quality = PEDro Score < = 3


14
New cards

Clinical Test of Heterogeneity

  • Variability caused by differences in clinical variables
    such as the patient population, interventions, outcome
    measures or setting of the studies included.


  • Common Sense Test

  • Does the pooling/combination of studies make sense?

    • Patient populations

    • Interventions

    • Outcomes



  • similar → more similar and precision

  • diff → harder for interpretation and more variable


15
New cards
16
New cards
17
New cards

Vote counting qualitative analysis (and is it GOOD or NOT?)

  • a simple method that tallies the direction or outcome of multiple independent studies or evaluations rather than pooling their raw statistical effect sizes


  • NOT good

    • do not do it b/c studies have different level of precision, variability—meaning that you can NOT say the studies are equal


18
New cards

More about Meta-Analysis

  • Statistical synthesis of the data from multiple studies that address the same question

  • Results in a "pooled" result from the combined studies

  • Generally compares one intervention to another and identifies which was more beneficial

    • Standardized mean difference/effect size (to know magnitude of significance btw groups in the studies and overall effect sizes)

    • Relative risk (dichotomous, example of RRisk: mortality)

  • Data are commonly presented as a Forest Plot


<ul><li><p><span>Statistical synthesis of the data from multiple studies that address the same question</span></p></li><li><p><span>Results in a "pooled" result from the combined studies</span></p></li><li><p><span>Generally compares one intervention to another and identifies which was more beneficial</span></p><ul><li><p><span><strong>Standardized mean difference/effect size </strong><em>(to know magnitude of significance btw groups in the studies and overall effect sizes) </em></span></p></li><li><p><span><strong>Relative risk </strong><em>(dichotomous,  example of RRisk: mortality) </em></span></p></li></ul></li></ul><ul><li><p><span>Data are commonly presented as a Forest Plot</span></p></li></ul><p></p>
19
New cards

Effect size

  • number of standard deviations of difference between the means of the control and intervention groups

  • it is also standardized mean difference (continuous data)

  • It is Dimensionless

  • The General classification is

    • ≥ 0.8 = large

    • > 0.5 = moderate effect

    • > 0.2 = small effect

    • < 0.2 = trivial (follows for distance from 0 for negative numbers)


20
New cards

How to calculate Effect size

(mean avg experimental group - mean avg control group) ÷ standard deviation


(mean avg experimental group minus mean avg control group) and all divided by standard deviation

<p>(mean avg experimental group <strong><mark data-color="#4c540d" style="background-color: rgb(76, 84, 13); color: inherit;">- </mark></strong>mean avg control group) <strong><mark data-color="#41440e" style="background-color: rgb(65, 68, 14); color: inherit;">÷ </mark>standard deviation</strong></p><p></p><p>(mean avg experimental group minus mean avg control group)  and all divided by standard deviation</p>
21
New cards

Relative Risk (for dichotomous data)

  • Compares rate of an event in one group to another

    • Event example, “disease/no disease” or “dead/alive”

    • Same general classification as Effect size
      ≥ 0.8 = large
      > 0.5 = moderate effect
      > 0.2 = small effect
      < 0.2 = trivial (follows for distance from 0 for negative
      numbers)


22
New cards

How to calculate Relative Risk (for dichotomous data)

Intervention Event Rule ÷ Control Rate

23
New cards
24
New cards
<p>Define the parts of this Meta-analysis of continuous outcome measures</p>

Define the parts of this Meta-analysis of continuous outcome measures

  • Opaque Squares = groups (effect sizes)

  • opaque diamond = overal effect

  • vertical line = line of no effect

  • horizontal line attaches to both ends of opaque squares = Confidence Interval

  • Favours control, favours intervention w/ the numbers = scale of treatment effect


What does this mean?

  • if it crosses line of no effect → NO effect or NO superiority (NOT significant)

  • Bigger squares, mean some of the studies are weighted differently → larger sample size and narrower confidence interval





<ul><li><p>Opaque Squares = groups (effect sizes) </p></li><li><p>opaque diamond = overal effect</p></li><li><p>vertical line = line of no effect </p></li><li><p>horizontal line attaches to both ends of opaque squares = Confidence Interval</p></li><li><p>Favours control, favours intervention w/ the numbers = scale of treatment effect</p></li></ul><p></p><p><strong>What does this mean?</strong></p><ul><li><p>if it crosses line of no effect → NO effect or NO superiority (NOT significant) </p></li><li><p>Bigger squares, mean some of the studies are weighted differently → larger sample size and narrower confidence interval </p></li></ul><p></p><p></p><p></p><p></p><p> </p>
25
New cards
<p>Forest plot A (Interpret it)</p>

Forest plot A (Interpret it)

  • hedges = effect sizes

  • lower limit and upper limit = confidence interval


what does it mean?

  • b/c studies (diamonds) cross over each other it means they are homogenenous

  • touching line also represents crossing the the line of NO effect

  • if higher scores are better → interventions are on the right

    • in this case you want more physical function, quality of life, peripheral muscle strength, respiratory muscle strength, ventilatory-free days



26
New cards
<p>Forest plot  B(Interpret it)</p>

Forest plot B(Interpret it)

  • lower limit and upper limit = confidence interval


What does it mean?

  • if lower scores are better → interventions are on the left

    • in this case, want less length of hospital stay and less length of ICU stay

  • Muehling et al has biggest effect size b/c farthest from the line of NO effect and big value

  • Schweickert et al has more weight square and shortest line meaning more higher quality and good effect size and more variable and narrower CI

  • Pooled effect size for Length of ICU studies → significant and effect size is small


27
New cards
<p>Forest Plot C (Interpret it)</p>

Forest Plot C (Interpret it)

for dichotomous data


  • lower limit and upper limit = confidence interval


what does it mean?

  • 1 is the line of No effect for dichotomous data due to the ratio of 1 = NO difference

  • In this data there is NO difference in mortality due to the crosing of the line

  • lower limit and upper limit = confidence interval


28
New cards

Statistical Test of Heterogeneity

  • Measures the variability between studies – gives an indication how comparable studies in meta-analysis are.

  • Q statistic: If p < 0.05 indicates that there may be diversity between
    studies that is not due to chance

  • I² ranges from 0-100%

  • If I² = < 25%

    • it means that studies are regarded as homogenous and fixed effect
      model of meta-analysis can be use


  • If I² = > 75%

    • it means that heterogeneity is very high and a random effects
      model for meta-analysis should be used.



29
New cards

Quick visual check to assess homogeneity is to check overlap of the CI’s (horizontal lines or whiskers in the meta-analysis graph). Studies are homogenous if CI’s of all studies __________

Studies are homogenous if CI’s of all studies overlap.


30
New cards

Sensitivity Testing

  • If heterogeneity is high, a sensitivity analysis (sub-analysis) should be performed

  • Secondary analysis with "suspect" studies removed

    • Ex: "suspect" = poor quality

  • Looking to see if pooled result changes

  • If pooled result changes, results are less valid


31
New cards

A bigger square means

larger sample size and more precision (b/c the lines that represent Confidence intervals are smaller)

32
New cards

Hohn and Yahr (H & Y) stages

1-5


1 means low disease severity


5 means high disease severity

33
New cards

motor severity range and meaning

  • 20-30, means moderate motor severity

  • <20-30, means less motor severity



34
New cards

Is 6-12 subjects per group is too small, what is bad about it

small studies,

  • aren’t the same at baseline

  • have high standard deviation

  • do Not have precise confidence intervals

  • have decreased generalizability


35
New cards

Sureness /power of what percent is good

80% and should include a point difference (clinical)

36
New cards

More than 20% dropout is good or bad for experiment

BAD

37
New cards

3 types of Data

Nominal data:

  • categories where the order does NOT matter (e.g. red, blue, green)

  • if only 2 categories = dichotomous (heads or tails)


Ordinal data:

  • categories where the order matters (example MMT, GROC, NPRS)

  • NOT consecutive and can NOT be added (ex: pain = 6 is not 3 pain + 3 pain)


Continuous data:

  • data measured by consecutive numbers that represent equal intervals such as 0 to 100 (ex: time)

  • can be added: (ex: you can add seconds )



38
New cards

How are the data types commonly reported?

Nominal: proportion


Ordinal (MMT): proportion + range


Continous normally distributed (ROM): mean, SD, range


Continous NOT normally distributed (ROM): mean, interquartile range

39
New cards

95% confidence interval means what?

  • 95% chance that the actual population value falls w/in 2

  • if we were to do the same experiment 100 times, we will see the same result


40
New cards

Power

Def: probability that study will detect a statistically significant difference btw variables or groups if it exists



The threshold power in most studies: 80%, meaning

  • 80% chance of detecting a diff btw groups if it is really there

  • 20% chance of committing a Type 2 error



This is determined before experience and also determines sample size

  • it is part of estimation


41
New cards

Type 1 error vs Type 2 error

Type 1 error:

  • study might find diff btw groups when a diff does NOT exist



Type 2 error:

  • study might NOT find diff btw groups when a diff does exist


42
New cards

Power is a function of

1) Alpha level

2) Sample size (most dependent on this)

3) Effect size

4) Type of stats test being conducted

5) Type of study design used

43
New cards

When to make alpha small or large

small (like 0.02):

  • analyzing airplane crashes due to engine failure

  • want lower probability of making a wrong decision


larger (like 0.10):

  • analyzing airplane crashes using paper airplanes

  • want to accept higher risk of making wrong decision


44
New cards

MCID (minimal clinically important difference) aka MID (minimal important difference)

  • It measures responsiveness, which reflects an outcome measure’s ability to detect change over time

  • it is the smallest difference in score that is important

  • represents the minimum # of change that pts perceive as good and that would justify a change in care


45
New cards

If MCID is smaller than MDC

the natural variability of measure hides the ability to detect clinically meaningful change

46
New cards

F-Ratio for variance

  • Between group variance divided by within group variance


47
New cards

larger the F-ratio, means what?

more likely you will have a p-value that will tell you that you’ll have statistical difference

48
New cards

Between group vs Within group

Btw group variance:

  • compares separate groups of participants where each group experiences 1 specific condition or treatment

  • treatment effect


With-in groups/ Error variance:

  • compares the same group of participants across multiple time points, conditions, or treatments

  • chance (random error + individual differences)


49
New cards

MDC (Minimal Detectable Change)

  • smallest amount of change required on an outcome measure to exceed measurement error and variability

  • it must be observed before being confident that real changed happened and the score change should NOT be linked to measurement error and variability