Unit 3: Measurement Validity, Research Validity, Reliability, and Diagnostic Accuracy

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/48

flashcard set

Earn XP

Description and Tags

Comprehensive vocabulary flashcards covering measurement validity, research validity, subject/investigator/logistics threats, reliability coefficients, diagnostic accuracy metrics, and likelihood ratios.

49 Terms

1
New cards

Validity

The degree to which a concept, conclusion, or measurement is well-founded and accurately reflects the real world.

2
New cards

Measurement Validity

The extent to which a test and/or instrument measures what it claims to measure (e.g., a goniometer measuring range of motion).

3
New cards

Research Validity

The extent to which a quantitative study of any type produces truthful, believable, and generalizable results.

4
New cards

Diagnostic Validity

The extent to which a diagnostic tool accurately identifies or predicts a specific health condition (e.g., EMG and nerve conduction diagnosing muscle function and nerve damage).

5
New cards

Face Validity

The extent to which an instrument or test appears to measure what it should on the surface; it is an intuitive and quick yes/no judgment that remains subjective and prone to bias.

6
New cards

Content Validity

The degree to which an assessment tool covers all relevant aspects and the entire domain of the construct being measured, typically judged by a panel of experts.

7
New cards

Construct Validity

The degree to which a test measures the theoretical construct it is intended to assess, consisting of convergent validity and discriminant validity.

8
New cards

Convergent Validity

A subtype of construct validity demonstrated when a test correlates highly with other instruments that measure the same or similar constructs.

9
New cards

Discriminant Validity

A subtype of construct validity demonstrated when a test does not correlate strongly with measures of unrelated constructs.

10
New cards

Criterion Validity

The degree to which the scores of an instrument are related to scores obtained with a current reference standard (gold standard).

11
New cards

Concurrent Validity

A subtype of criterion validity that reflects how well a test correlates with a current reference standard measured at the same time or within a reasonable time interval.

12
New cards

Predictive Validity

A subtype of criterion validity reflecting how well an assessment or test score predicts future performance, outcomes, or events (e.g., predicting falls over the next 66 months).

13
New cards

Internal Validity

The believability of a study, representing the extent to which observed results reflect the truth in the studied population without being caused by methodological errors or bias.

14
New cards

External Validity

The extent to which the findings and conclusions of a study can be generalized to other populations, settings, and conditions.

15
New cards

Selection Threat

A threat related to subjects where a nonrepresentative sample is chosen relative to the population of interest, producing inaccurate study results.

16
New cards

Assignment Threat

A threat related to subjects where the allocation process yields unequal baseline characteristics between study groups that could influence outcomes.

17
New cards

Attrition Threat

A threat related to subjects caused by the loss of participants over time (due to death, drop-out, or relocation), leading to reduced sample size or group baseline inequality.

18
New cards

Maturation Threat

A threat related to subjects involving natural internal biological or functional changes over time (e.g., spontaneous recovery or healing) that influence the outcome independently of intervention.

19
New cards

Compensatory Rivalry

A threat related to subjects where participants who discover they are assigned to a control group alter their behavior and work harder to prove they can improve anyway.

20
New cards

Resentful Demoralization

A threat related to subjects where participants who find out they are in a control group become resentful or discouraged and decrease effort or engagement.

21
New cards

Diffusion or Imitation of Treatments

A threat related to subjects occurring when control group participants communicate with experimental participants and learn or adopt parts of the intervention.

22
New cards

Statistical Regression to the Mean

The mathematical tendency for participants who score at the extreme high or low ends on baseline outcome measures to score closer to the average upon repeat testing.

23
New cards

Compensatory Equalization of Treatments

An investigator-related threat occurring when clinicians or researchers inadvertently or purposefully provide extra attention or alternative treatments to the control group.

24
New cards

History Threat

A study logistics threat involving external concurrent events occurring outside the experiment that are beyond the researcher's control and affect the outcome.

25
New cards

Instrumentation Threat

A study logistics threat arising from the inappropriate selection, malfunction, limitation, lack of calibration, or inaccurate application of measurement devices or techniques.

26
New cards

Testing Threat

A study logistics threat in which outcome scores change due to a participant's increased familiarity, practice with the procedure, or cues given by the tester.

27
New cards

Reliability

The ability of a measurement, test, or instrument to produce consistent and repeatable results when administered under identical conditions.

28
New cards

Intra-rater Reliability

The consistency of repeated measures taken by the very same evaluator on the same individuals under identical conditions.

29
New cards

Inter-rater Reliability

The consistency or agreement among measurements performed by different evaluators assessing the same individuals under identical conditions.

30
New cards

Test-Retest Reliability

The consistency of scores obtained when the exact same test is administered to the same individuals under identical conditions across two separate occasions.

31
New cards

Percent Agreement

A categorical reliability metric indicating the percentage of matching ratings; limited because it cannot distinguish chance agreement from true systematic agreement.

32
New cards

Kappa Coefficient (κ\kappa)

A statistical measure of inter-rater agreement for categorical data that accounts for and removes agreement expected by chance alone: κ=observed agreement−chance agreement1−chance agreement\kappa = \frac{\text{observed agreement} - \text{chance agreement}}{1 - \text{chance agreement}}.

33
New cards

Kappa Interpretation Thresholds

Standard guidelines for Cohen's kappa where values >0.75> 0.75 indicate excellent/perfect agreement, values between 0.40–0.750.40\text{--}0.75 indicate fair to good agreement, and values <0.40< 0.40 indicate poor agreement.

34
New cards

Index Test

The new or experimental diagnostic test undergoing appraisal and comparison against an established gold standard.

35
New cards

Reference Standard (Gold Standard)

The most accurate, established, and reliable diagnostic method used to establish the true presence or absence of a specific pathology or condition.

36
New cards

Spectrum Bias

A diagnostic appraisal pitfall where the index test is studied in a nonrepresentative spectrum of subjects, typically overrepresenting extreme severe cases and healthy controls.

37
New cards

Verification Bias (Work-Up Bias)

A diagnostic appraisal pitfall occurring when the decision to perform the reference standard depends on the index test outcome or prior clinical suspicion rather than being applied to all subjects.

38
New cards

Incorporation Bias

A diagnostic appraisal pitfall occurring when the index test being evaluated is already included as an active part or component of the reference standard.

39
New cards

Observer Bias

A bias that arises when assessors evaluating the reference standard or index test are not masked/blinded to previous results or patient status, influencing subjective interpretation.

40
New cards

Sensitivity

The ability of a diagnostic test to correctly identify individuals who truly have the condition, calculated as AA+C\frac{A}{A + C} (True Positives divided by all individuals with the disease).

41
New cards

Specificity

The ability of a diagnostic test to correctly identify individuals who do not have the condition, calculated as DB+D\frac{D}{B + D} (True Negatives divided by all individuals without the disease).

42
New cards

SnNOUT

A clinical rule stating that with high Sensitivity, a Negative test result effectively rules OUT the diagnosis because false negative rates are very low.

43
New cards

SpPIN

A clinical rule stating that with high Specificity, a Positive test result effectively rules IN the diagnosis because false positive rates are very low.

44
New cards

Positive Likelihood Ratio (LR+LR+)

A diagnostic ratio assessing the odds of a positive test in diseased vs non-diseased individuals, calculated as Sensitivity1−Specificity\frac{\text{Sensitivity}}{1 - \text{Specificity}}.

45
New cards

Negative Likelihood Ratio (LR−LR-)

A diagnostic ratio assessing the odds of a negative test in diseased vs non-diseased individuals, calculated as 1−SensitivitySpecificity\frac{1 - \text{Sensitivity}}{\text{Specificity}}.

46
New cards

Large and Conclusive Likelihood Ratio

A diagnostic likelihood ratio of LR+>10LR+ > 10 or LR−<0.10LR- < 0.10, representing a large and decisive shift from pre-test to post-test probability.

47
New cards

Moderate Likelihood Ratio

A diagnostic likelihood ratio of LR+=5–10LR+ = 5\text{--}10 or LR−=0.10–0.20LR- = 0.10\text{--}0.20, generating a moderate shift from pre-test to post-test probability.

48
New cards

Small Likelihood Ratio

A diagnostic likelihood ratio of LR+=2–5LR+ = 2\text{--}5 or LR−=0.20–0.50LR- = 0.20\text{--}0.50, indicating a small but sometimes clinically important change in disease probability.

49
New cards

Negligible Likelihood Ratio

A diagnostic likelihood ratio of LR+=1–2LR+ = 1\text{--}2 or LR−=0.50–1.0LR- = 0.50\text{--}1.0, indicating a negligible or clinically insignificant change in pre-test probability.