Ch 5: Validity

Validity

Degree to which evidence supports interpretations of test scores for intended uses.

Validity evidence

Evidence supporting the interpretation and use of test scores.

Construct underrepresentation

Test fails to adequately represent the construct.

Construct-irrelevant variance

Score variation caused by factors unrelated to the intended construct.

Item relevance

Degree to which an item relates to the intended construct.

Content coverage

Extent to which test adequately covers the construct.

Face validity

Extent to which a test appears to measure its intended construct.

Factor analysis

Statistical method for identifying underlying dimensions/factors.

Factor

Underlying dimension accounting for relationships among variables.

Exploratory Factor Analysis (EFA)

Explores possible underlying factor structures.

Confirmatory Factor Analysis (CFA)

Tests a hypothesized factor structure.

Factor loading

Strength of association between an item and a factor.

Model fit

Degree to which a proposed model matches observed data.

Test-criterion evidence

Evidence from relationship between test and external criterion.

Predictive study

Test measured first; criterion measured later.

Predictive validity evidence

Evidence that test scores predict a future criterion.

Concurrent study

Test and criterion measured at approximately the same time.

Validity coefficient

Correlation between test scores and a relevant criterion/variable.

Convergent evidence

Test relates strongly to measures of similar constructs.

Discriminant evidence

Test remains distinct from measures of different constructs.

Multitrait-multimethod (MTMM)

Examines multiple traits using multiple methods.

Monotrait-monomethod

Same trait measured with same method.

Monotrait-heteromethod

Same trait measured with different methods.

Heterotrait-monomethod

Different traits measured with same method.

Heterotrait-heteromethod

Different traits measured with different methods.

Validity generalization

Extent validity evidence generalizes across settings/populations.

Local validation

Validation study conducted in a specific setting/population.

Incremental validity

Additional predictive information beyond existing measures.

Test bias

Systematic score/interpretation error affecting a group.

Differential validity

Test predicts/measures differently across groups.