Chapter 15: Reliability and Validity

Reliability

The extent to which an instrument measures the concept of interest consistently.

Validity

The extent to which an instrument measures the concept of interest accurately.

Reliability vs. Validity relationship

An instrument can be reliable without being valid, but a valid instrument is always reliable.

Content Validity

Assesses whether the instrument is representative of the concept of interest, often using a panel of experts.

Criterion-Related Validity

Degree to which a subject's performance on an instrument and their behavior are related (includes concurrent and predictive).

Construct Validity

The extent to which an instrument measures a theoretical construct and validates its underlying theory.

Stability (Reliability)

The ability of an instrument to produce the same results with repeated testing, often measured by test-retest reliability.

Internal Consistency (Homogeneity)

The extent to which all items in an instrument measure the same concept, commonly measured using Cronbach's Alpha.

Cronbach's Alpha Threshold

Alpha values above 0.70 provide sufficient evidence supporting the internal consistency of an instrument using a Likert scale.

Equivalence

Consistency or agreement among observers using an instrument designed to capture observational data.

Cronbach's Alpha

checks if a questionnaire is consistent/reliable

Interrater Reliability

Tested using Kappa (K), where a score of 0.80 or better indicates good reliability among observers.