SI - Understanding Reliability and Validity in Measurements

Reliability

The degree to which the same event produces consistent results.

Types of Measurement Error

Includes systematic errors (predictable, consistent) and random errors (unpredictable, inconsistent).

Systematic Errors

Errors that are predictable and consistent, such as improper use of landmarks or improperly calibrated tools.

Random Errors

Errors that are unpredictable and inconsistent, such as examiner fatigue or environmental disruptions.

Test-retest Reliability

The consistency of a test over time.

Internal Consistency

The correlation among items within a test, commonly measured using Cronbach's alpha.

Intra-rater Reliability

The stability of one rater's measurements over time.

Inter-rater Reliability

The agreement between two or more raters.

Intraclass Correlation Coefficient (ICC)

Ideal for continuous data, reflecting both relationship and agreement; ranges from excellent (>0.90) to poor (<0.75) reliability.

Cronbach's alpha

A measure of internal consistency.

Kappa statistic

Used for assessing agreement in categorical data.

Face Validity

Determines whether the test appears effective at first glance.

Content Validity

The extent to which a test covers all aspects of the concept being measured.

Construct Validity

Measures an abstract concept, such as intelligence or personality.

Criterion-related Validity

Compares a test to an established gold standard.

Concurrent Validity

Correlates with current performance.

Predictive Validity

Predicts future outcomes.

Effect Size (ES)

Measures change between groups or over time, categorized into large (0.8), moderate (0.5), and small (0.2) effects.

Key Differences Between Reliability and Validity

Reliability is necessary for validity, but reliability alone does not guarantee validity.

Minimizing Errors in Measurement

Include using operational definitions, proper training of examiners, regular equipment inspection, and blinded assessments.

Knee Outcome Survey

Demonstrated high ICC (0.97) indicating excellent reliability and strong associations between baseline and follow-up scores.

Responsiveness

The ability to detect meaningful changes over time in clinical outcomes.