Psy Ass - Chapter 6

Validity

is a judgment or estimate of how well a test measures what it purports to measure in a particular contexta judgment based on evidence about the appropriateness of inferences drawn from test scores

Validity

a judgment based on evidence about the appropriateness of inferences drawn from test scores

inference

logical result or deduction.

Validation

the process of gathering and evaluating evidence about validity.

Local validation studies

necessary when the test user plans to alter in some way the format, instructions, language, or content of the test

Content validity

This is a measure of validity based on an evaluation of the subjects, topics, or content covered by the items in the test.

Criterion-related validity.

This is a measure of validity obtained by evaluating the relationship of scores obtained on the test to scores on other tests or measures

Construct validity

This is a measure of validity that is arrived at by executing a c o m p r e h e n s i v e a n a l y s i s o f

a. how scores on the test relate to other test scores and measures, and

b. how scores on the test can be understood within some theoretical framework for understanding the construct that the test was designed to measure.

ecological validity

a judgment regarding how well a test measures what it purports to measure at the time and place that the variable being measured is actually emmitted

face validity

a judgment concerning how relevant the test items appear to be.

content validity

describes a judgment of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample

test blueprint

a plan regarding the types of information to be covered by the items, the number of items tapping each area of coverage, the organization of the items in the test, and so forth

Criterion-related validity

a judgment of how adequately a test score can be used to infer an individual’s most probable standing on some measure of interest—the measure of interest being the criterion.

Concurrent validity

an index of the degree to which a test score is related to some criterion measure obtained at the same time (concurrently)

Predictive validity

an index of the degree to which a test score predicts some criterion measure

criterion

the standard against which a test or a test score is evaluated

relevant, valid, uncontaminated

characteristics of criterion

Criterion contamination

the term applied to a criterion measure that has been based, at least in part, on predictor measures

concurrent validity

If test scores are obtained at about the same time as the criterion measures are obtained, measures of the relationship between the test scores and the criterion provide evidence of ________________

predictive validity

how accurately scores on the test predict some criterion measure

base rate

the extent to which a particular trait, behavior, characteristic, or attribute exists in the population (

hit rate

the proportion of people a test accurately identifies as possessing or exhibiting a particular trait, behavior, characteristic, or attribute.

miss rate

the proportion of people the test fails to identify as having, or not having, a particular characteristic or attribute

false positive

a miss wherein the test predicted that the testtaker did possess the particular characteristic or attribute being measured when in fact the testtaker did not

false negative

a miss wherein the test predicted that the testtaker did not possess the particular characteristic or attribute being measured when the testtaker actually did

validity coefficient

a correlation coefficient that provides a measure of the relationship between test scores and scores on the criterion measure

incremental validity

the degree to which an additional predictor explains something about the criterion measure that is not explained by predictors already in use

construct validity

a judgment about the appropriateness of inferences drawn from test scores regarding individual standings on a variable called a construct

construct

an informed, scientific idea developed or hypothesized to describe or explain behavior

homogeneity

refers to how uniform a test is in measuring a single concept

method of contrasted groups

one way of providing evidence for the validity of a test is to demonstrate that scores on the test vary in a predictable way as a function of membership in some group

convergent evidence

Evidence for the construct validity of a particular test may converge from a number of sources, such as other tests or measures designed to assess the same (or a similar) construct. Thus, if scores on the test undergoing construct validation tend to correlate highly in the predicted direction with scores on older, more established, and already validated tests designed to measure the same (or a similar) construct, this would be an example of _____________________________________

discriminant evidence

A validity coefficient showing little (a statistically insignificant) relationship between test scores and/or other variables with which scores on the test being construct-validated should not theoretically be correlated provides ______________ of construct validity

factor analysis

a shorthand term for a class of mathematical procedures designed to identify factors or specific variables that are typically attributes, characteristics, or dimensions on which people may differ.

Exploratory factor analysis

typically entails “estimating, or extracting factors; deciding how many factors to retain; and rotating factors to an interpretable orientation”

confirmatory factor analysis

researchers test the degree to which a hypothetical model (which includes factors) fits the actual data

factor loading

“a sort of metaphor. Each test is thought of as a vehicle carrying a certain amount of one or more abilities”

bias

a factor inherent in a test that systematically prevents accurate, impartial measurement

rating

a numerical or verbal judgment (or both) that places a person or an attribute along a continuum identified by a scale of numerical or word descriptors known as a rating scale.

rating error

a judgment resulting from the intentional or unintentional misuse of a rating scale

leniency error or generosity error

an error in rating that arises from the tendency on the part of the rater to be lenient in scoring, marking, and/or grading

rankings

a procedure that requires the rater to measure individuals against one another instead of against an absolute scale

halo effect

describes the fact that, for some raters, some ratees can do no wrong

fairness

in a psychometric context as the extent to which a test is used in an impartial, just, and equitable way