IRAT #4

validity

the extent to which the instrument used in the experiment measures exactly what it aims to measure

reliability

the ability of an instrument to measure the attributes of a concept consistently

reliability reports

• numbers between 0 and 1

• closer to 1 the more reliable and lower error variance

reliabilty tests

• cronbachs alpha

• kuder-richardons

• split-half pearson’s

critical appraisal

the process of carefully and systematically examining a research report to judge its trustworthiness, make sense of the results and assess the relevance of the findings in a particular context

critical appraisal process

1. Read article. Determine the appropriate critical appraisal form to use

2. Identify strengths and weaknesses while reading

3. Use a critical appraisal form to answer questions and provide rationale

4. Review your responses. Be objective

5. Total the score of the appraisal. Consider whether your overall impression of the article aligns with the score.

synthesizing evidence

incorporating different sources of evidence

steps of synthesis

• review sources together

• meaningfully synthesize each column in the evidence table

• present the evidence in a logical flow

• present evidence clearly and with sufficient detail

• do not include everything about the evidence source

• concise and clear

Level 1 evidence

• experimental study, randomized controlled trial

• explanatory mixed-methods design that includes only 1 quantitative study

• systemic review of RCTs with or without meta-analysis

level 2 evidence

• quasi-experimental study designs

• explanatory mixed-method design that includes only a level II quantitative study

• systemic review of a combination of RCTs and quasi-experimental studies or quasi-experimental studies only with or without meta-analysis

level 3 evidence

• nonexperimental study

• systematic review of a combination of RCTs, quasi and nonexperimental studies

• exploratory, mixed methods design that includes only a level III quantitative study

• quantitative study

• systemic review qualitative studies without meta-analysis

level 4

• opinion of respected authorities and or nationally recognized expert committees or consensus panels based on scientific evidence

  • clinical practice guidelines

  • consensus panels/positions

level 5

experimental and non-research evidence

• reviews