Midterm Review: Measurement, Validity, Intelligence, and Classical True Score Theory

Comprehensive question-and-answer flashcards for midterm review covering introductory measurement concepts, validity types, Messick's validity matrix, screening errors, intelligence as a latent construct, and Classical True Score Theory.

How is measurement defined?

Assigning numbers to individuals systematically as a means of representing properties of the individual.

What is a test?

A device for obtaining a sample of an individual's behavior.

What is testing, and what type of skills does it require?

The process of measuring variables by means of a device; it requires technician-like skills, meaning one technician can be substituted for another.

How is assessment defined in educational evaluation?

Gathering and integrating data to make a psycho-educational evaluation, requiring educated selection of tools and organization of data.

What is evaluation?

Integrating data to make data-based decisions.

What is a norm-referenced test, and what is its typical purpose?

A test that uses the test performance of others on the same test as a comparison; it is intended for classification/selection (e.g., SAT/ACT).

What is a criterion-referenced test, and what is its primary purpose?

A test that measures performance with respect to content standards; it is intended for instructional planning (e.g., state tests, driving tests).

What is curriculum-based measurement (CBM)?

An assessment of academic skill fluency characterized as a direct (low-inference) assessment, often used for progress monitoring (e.g., Acadience benchmark tests).

What is the general definition of validity?

The degree to which a test measures what it purports to measure.

What is content validity?

The degree to which a test's contents contain an appropriate and representative sample of the content contained in the instructional objectives.

What is criterion-related validity?

The degree to which an individual's score or performance on a test predicts their performance in another area.

What distinguishes predictive validity from concurrent validity?

Predictive validity predicts performance at a future point in time, whereas concurrent validity predicts performance within the same window of time.

What is construct validity?

The degree to which a test measures the theoretical concept or trait that it was designed to measure.

What is convergent validity, and what is an example of it?

How well performance on different measures of the same construct correlate positively (e.g., two intelligence tests producing similar scores for the same individual).

What is divergent validity, and what is an example of it?

The degree to which tests designed to measure unrelated constructs should not correlate (e.g., the WISC should not correlate with Beck Depression Inventory scores for an individual).

What are the core premises of Messick's validity matrix regarding valid judgments and decision validity?

Valid judgments are value judgments (involving score meaning and consequences), and decision validity refers to validity in the meaning of test scores.

In Messick's validity matrix, what constitutes the evidential basis for test interpretation?

Construct validity (CV), which ensures that the test measures what it purports to measure.

In Messick's validity matrix, what constitutes the evidential basis for test use?

Construct validity plus relevance/utility ($\text{CV} + \text{R/U}$), meaning that when used in a given situation, the test will still measure what it purports to measure.

In Messick's validity matrix, what constitutes the consequential basis for test interpretation?

Construct validity plus value implications ($\text{CV} + \text{VI}$), addressing concern for the value-laden labels used to describe the test taker.

In Messick's validity matrix, what constitutes the consequential basis for test use?

$\text{CV} + \text{R/U} + \text{VI} + \text{social consequences}$, which involves considering the social consequences of test use or the application of inferences.

What is a false positive error in the context of screening?

Predicting that a student would be at risk when they are not (e.g., predicting a student to be below benchmark, but they perform at or above benchmark).

What is a false negative error in the context of screening?

Predicting that a student is not at risk when they actually are (e.g., predicting a student to be at or above benchmark, but they perform below benchmark).

What is the base rate, and what constitutes a good hit rate?

The base rate is the proportion of individuals with the most frequently occurring outcome; a good hit rate depends on the base rate and should improve on it.

Why is intelligence considered a common factor or latent variable?

It cannot be measured directly; it can only be measured indirectly through effect indicators such as vocabulary, short-term memory, problem solving, pattern recognition, and nonverbal reasoning.

What is the Flynn effect?

The substantial and long-sustained increase in both crystallized and fluid intelligence test scores over time.

What is the equation for Classical True Score Theory, and what does each variable denote?

$X = T + E$, where $X$ is the observed score, $T$ is the true score, and $E$ is the error of measurement (random).

Why is $T$ (true score) defined by the test rather than the trait being measured?

$T$ is a theoretical construct, and we cannot take infinite samples.

Why is it impossible to actually observe a true score ($T$)?

$T$ is fixed but unknown because it is impossible to take infinite samples.

How are confidence intervals phrased and interpreted in Classical True Score Theory?

"We can say with [$\%$] confidence that the true score falls between [lower bound] and [upper bound]."