Psychometric Reliability Concepts and Models

Fill-in-the-blank practice flashcards covering classical test theory, reliability models, measurement error, internal consistency, inter-scorer agreement, and psychometric evaluation principles.

In 1904, Charles Spearman published a foundational paper titled "__________".

The Proof and Measurement of Association between Two Things

Reliability broadly refers to the __________ and consistency of test scores over time or across different items.

dependability

For most basic research purposes, reliability estimates in the range of __________ and __________ are considered good enough.

$.70$; $.80$

In Classical Test Theory, the fundamental formula expressing an observed score is __________.

$$X = T + E$$

In Classical Test Theory, $X$ represents the observed score, $T$ represents the true score, and $E$ represents __________.

Error

Classical Test Theory assumes that measurement errors are random and that their distribution is __________.

bell-shaped

The standard deviation of measurement errors is used as the basic measure of error and is known as the __________.

Standard Error of Measurement

If an examinee achieves an observed score of $70$ on a test with an SEM of $4$, their possible true score falls between __________ and __________.

$66$; $74$

Variation among items within or between tests due to item sampling or content sampling is a major source of error variance in __________.

TEST CONSTRUCTION

Test-retest reliability is also referred to as __________ reliability.

time-sampling

When the time interval between test-retest administrations exceeds $6$ months, the resulting coefficient is known as the coefficient of __________.

stability

In parallel-form reliability, two forms are constructed such that the __________ and variances of observed test scores are equal.

means

To correct for half-length when correlating split halves of a single test, researchers use the __________ formula.

Spearman-Brown

The Kuder-Richardson 20 (KR-20) formula is designed to calculate reliability for tests whose items are __________, meaning scored $0$ or $1$.

dichotomous

According to the common interpretation table for KR-20, a value between $0.90$ and $1.00$ indicates __________.

Very high consistency

Cronbach developed coefficient alpha as a general estimate of internal consistency for tests whose items are not scored as __________.

$0$ or $1$

As illustrated in the diagram, a High Cronbach's Alpha ($\alpha = .90$) means items are highly correlated, move together, and __________.

Measure the same thing

As shown in the diagram, a Low Cronbach's Alpha ($\alpha = .40$) means items are weakly correlated, don't move together, and __________.

May be measuring different things

When assessing inter-scorer agreement, __________ Kappa is used for $2$ raters, whereas Fleiss' Kappa is used for $3$ or more raters.

Cohen's

According to the inter-scorer agreement table, values greater than $0.75$ indicate __________ Agreement.

Excellent

Tests that contain items measuring a single trait or factor are described as __________.

homogenous

Traits or states that are presumed to be ever-changing as a function of situational experiences are called __________ characteristics.

dynamic

A test that contains relatively easy items but strict, short time limits is known as a __________ test.

speed

Item Response Theory (IRT) uses computer adaptivity to tailor questions based on item __________.

difficulty

In clinical settings where important decisions are made about an individual's future, evaluators should seek tests with a reliability greater than __________.

$$.95$$