PSYASS 2 - 6 Validity

Validity

A judgement or estimate of how well a test measures what it purports to measure in a particular context.

Universally valid

No test or measurement technique is “_______________.”

Validation

Process of gathering evidence about validity.

Local Validation Studies

Absolutely necessary when altering a test.

Content Validity

Measure of validity based on an evaluation of the subjects, topics, or content covered by the items in the test.

Criterion-related Validity

Measure of validity obtained by evaluating relationship of scores obtained on the test to scores on other tests or meassures.

Construct Validity

Measure of validity that is arrived at by executing a comprehensive analysis of:

• How scores on the test relate to other test scores and measures

• How scores on the test can be understood within some theoretical framework for understanding the construct that the test was designed to measure

Face validity

Relates more to what a test appears to measure to the person being tested that to what the test actually measures.

Content Validity

A judgment of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample.

Test Blueprint

A plan regarding the types of information to be covered by the items, the number of items tapping each area of coverage, the organization of the items in the test, etc.

Criterion-Related Validity

A judgment of how adequately a test score can be used to infer an individual's most probable standing on some measure of interest (i.e., the criterion).

Concurrent validity

An index of the degree to which a test score is related to some criterion measure obtained at the same time.

Predictive validity

An index of the degree to which a test score predicts some criterion, measure.

Criterion

The standard against which a test or a test score is evaluated.

Criterion contamination

A term applied to a criterion measure that has been based, at least in part, on predictor measures.

Validity Coefficient

A judgment of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample.

Incremental Validity

The degree to which an additional predictor explains something about the criterion measure that is not explained by predictors already in use or existing.

Incremental Validity

This type of validity asks, "Does this new test give me any extra information that will help me make better predictions, beyond what I already know?"

Construct Validity

Judgment about the appropriateness of inferences drawn from test scores regarding individual standings on a construct.

Evidence of Homogeneity

How uniform a test is in measuring a single concept.

Evidence of Changes with Age

Some constructs are expected to change over time.

Evidence of Pretest–Posttest Changes

Test scores change as a result of some experience between a pretest and a posttest.

Evidence from Distinct Groups

Scores on a test vary in a predictable way as a function of membership in some group.

Convergent Evidence

The degree to which a test strongly correlates with others measuring the same or a related construct, confirming its validity.

Discriminant Evidence

Ensures a test does not correlate highly with unrelated constructs, proving it measures a distinct concept.

Multitrait-Multimethod Matrix

A framework (Campbell & Fiske, 1959) that assesses convergent and discriminant validity by comparing multiple traits and methods for consistency.

Factor Analysis

A statistical method used to identify underlying variables that explain individual differences in observed data. It helps group related variables and uncover hidden patterns in psychological and behavioral assessments.

Exploratory Factor Analysis

Used when the factor structure is unknown, grouping related variables based on data patterns without prior assumptions.

Confirmatory Factor Analysis

Tests whether a predefined factor structure fits the data, validating hypothesized relationships between variables and factors.

Bias

A factor inherent in a test that systematically prevents accurate, impartial measurement.

Rating

The process of assigning a score or judgment to an individual’s behavior, traits, or performance based on predefined criteria, often using a structured scale.

Rating error

Inaccuracies in scoring due to subjective biases, misinterpretations, or inconsistencies by the rater, affecting the reliability and validity of assessments.

Leniency Error

A type of rating bias where a rater consistently assigns overly high scores, making individuals appear more favorable than they actually are.

Severity Error

The opposite of leniency error, where a rater systematically gives excessively low scores, making individuals appear worse than they actually are.

Halo Effect

A cognitive bias where a rater’s overall impression of an individual influences specific ratings, leading to inflated or deflated scores across different traits.

Fairness

The principle that psychological assessments should be free from bias, ensuring equal opportunities for all individuals regardless of background, culture, or personal characteristics.