1/45
This flashcard set covers the vocabulary and foundational concepts of psychological test validity, including types of validity evidence, rating errors, and psychometric bias/fairness as presented in Chapter 6.
Validity
A judgment or estimate of how well a test measures what it purports to measure in a particular context.
Inference
A logical result or deduction based on evidence about the appropriateness of test scores.
Validation
The process of gathering and evaluating evidence about validity.
Local validation study
A study conducted by test users with their own group of testtakers, which is necessary when altering the format, instructions, language, or content of a test.
Trinitarian view
A classic conception of validity that categorizes it into three types: content validity, criterion-related validity, and construct validity.
Content validity
A measure of validity based on an evaluation of the subjects, topics, or content covered by the items in the test.
Criterion-related validity
A measure obtained by evaluating the relationship of scores obtained on the test to scores on other tests or measures.
Construct validity
An analysis of how scores on a test relate to other measures and how they can be understood within a theoretical framework; often viewed as the "umbrella validity" covering all other varieties.
Ecological validity
A judgment regarding how well a test measures what it purports to measure at the time and place that the variable being measured is actually emitted.
Ecological momentary assessment (EMA)
The in-the-moment and in-the-place evaluation of targeted variables in a natural, naturalistic, or real-life context.
Face validity
What a test appears to measure to the person being tested rather than what the test actually measures.
Test blueprint
A detailed plan regarding the types of information to be covered by the items, the number of items tapping each area, and the organization of the items in the test.
Criterion
The standard against which a test or a test score is evaluated, which ideally is relevant, valid, and uncontaminated.
Criterion contamination
A condition where a criterion measure is based, at least in part, on predictor measures (e.g., using the same source for both the prediction and the evaluation).
Concurrent validity
An index of the degree to which a test score is related to some criterion measure obtained at the same time.
Predictive validity
An index of the degree to which a test score predicts some criterion measure obtained at a future time.
Base rate
The extent to which a particular trait, behavior, characteristic, or attribute exists in the population, typically expressed as a proportion.
Hit rate
The proportion of people a test accurately identifies as possessing or exhibiting a particular trait, behavior, characteristic, or attribute.
Miss rate
The proportion of people the test fails to identify as either having or not having a particular characteristic (an inaccurate prediction).
False positive
A type of miss where the test predicted the testtaker possessed a characteristic when they actually did not.
False negative
A type of miss where the test predicted the testtaker did not possess a characteristic when they actually did.
Validity coefficient
A correlation coefficient that provides a measure of the relationship between test scores and scores on the criterion measure.
Incremental validity
The degree to which an additional predictor explains something about the criterion measure that is not explained by predictors already in use.
Construct
An informed, scientific idea developed or hypothesized to describe or explain behavior, such as intelligence or anxiety.
Homogeneity
As applied to a test and its items, the degree to which a test is uniform in measuring a single concept.
Method of contrasted groups
A procedure for providing validity evidence by demonstrating that scores on a test vary in a predictable way as a function of membership in some group.
Convergent evidence
Evidence for construct validity showing that test scores correlate highly in the predicted direction with scores on older, established, and already validated tests designed to measure the same or a similar construct.
Discriminant evidence
Also known as discriminant validity; a coefficient showing an insignificant relationship between test scores and variables with which the test should not theoretically be correlated.
Multitrait-multimethod matrix
An experimental technique for examining convergent and discriminant validity by correlating variables within and between methods.
Method variance
The similarity in scores that is due to the use of the same method rather than to the natural correlations among the constructs.
Factor analysis
A class of mathematical procedures designed to identify factors or specific variables (attributes, dimensions) on which people may differ.
Exploratory factor analysis
A type of factor analysis involving estimating factors, deciding how many to retain, and rotating factors to an interpretable orientation.
Confirmatory factor analysis
A type of factor analysis where researchers test the degree to which a hypothetical model (including factors) fits the actual data.
Factor loading
A metaphor for the extent to which a factor determines a test score, conveying how much of a specific ability or trait is "carried" by the test.
Bias
A factor inherent in a test that systematically prevents accurate, impartial measurement.
Intercept bias
Occurs when the use of a predictor results in consistent underprediction or overprediction of a specific group's performance or outcomes.
Slope bias
Occurs when a predictor has a weaker correlation with an outcome for specific groups.
Fairness
In a psychometric context, the extent to which a test is used in an impartial, just, and equitable way.
Rating
A numerical or verbal judgment that places a person or attribute along a continuum identified by a scale of descriptors.
Rating error
A judgment resulting from the intentional or unintentional misuse of a rating scale.
Leniency error
Also known as a generosity error; an error arising from a rater's tendency to be lenient in scoring, marking, or grading.
Severity error
An error in rating where the rater tends to be overly harsh or pan the performance/attribute being reviewed.
Central tendency error
A systematic reluctance on the part of a rater to give ratings at either the positive or negative extremes, causing scores to cluster in the middle.
Halo effect
The tendency to give a ratee a higher rating than objectively deserved due to a failure to discriminate among independent aspects of behavior.
Banding
A technique that makes equivalent all test scores that fall within a particular range or band.
Within-group norming
The conversion of raw scores into percentile or standard scores based on the test performance of one's own group; known as race-norming when race is the criterion.