Lecture 11 Validity

Validity Definition

The extent to which a test measures what it was designed to measure.

Content Analysis

examining test content

Incremental Validity

How much a test adds to the prediction of criteria that are already being predicted.

Reliability vs Validity

A test can be reliable without being valid, but cannot be valid without being reliable.

Content Validity

The degree to which a test represents the entire domain of skills and behaviors it is designed to measure.

Face Validity

The physical appearance of a test in relation to its intended purposes.

Achievement Tests

tests specifically measure how well students have learned the material based on the instructional objectives set for them

Concurrent Validity

Used when test is done across different categories of population to see distinction in scores

Predictive Validity

How well test scores predict future performance based on correlation with future outcomes.

Factors Affecting Criterion-Related Validity

Includes incremental validity, group differences, test length, and criterion contamination.

Test Length Impact

Validity varies directly with test length; longer tests have higher predictive validity.

Criterion Contamination

Validity is impacted by the measurement methods of the criterion itself. Can introduce biases/errors

Construct Validity Definition

The extent to which a test measures a psychological construct, such as anxiety or extroversion.

Establishing Construct Validity

Includes expert judgments, internal consistency analysis, and correlation with relevant variables. Questions examinees in thought process during Reponses.

Construct Characteristics

examining psychological contructs measured by the test

Convergent Validity

High correlation with measures of the same construct.

Discriminant Validity

Low correlation with measures of different constructs.

Test Administration

The conditions under which the test is administered can impact validity.

Target Population

The specific group for whom the test was designed affects its validity.

Purpose of Test

The specific goals for which the test is created influence its various validities.

Psychological Constructs

Characteristics, such as motivation and personality traits, that a test may aim to measure.

Validity Procedures

Methods include analyzing test content, correlating scores, and investigating measured constructs.

Expert Agreement

Content validity is confirmed when subject matter experts agree on test's design and measurement intentions.

Utility of Test

Measured increase in the quality of decisions made using the test as opposed to not using it.

Group Differences

Variables such as sex, age, and personality traits that may affect test validity.

Cross Validation

Administering a test to a second sample to determine its validity across different groups.

Predictor-Criterion Relationship

Correlational relationship between test scores (predictor) and future performance measures (criterion).

Testing Cognitive Processes

Evaluating the mental processes involved in test responses helps determine construct validity.

Table of Specifications

An outline comparing test content to subject matter expectations to evaluate content validity.

Analyzing Test Composition

Evaluating a test's content ensuring validity

Correlation Criteria

Comparing Test scores with performance criteria