Psychological Testing & Assessment: Theories and Techniques of Reliability Estimates

Vocabulary flashcards covering core definitions, theories, and sources of measurement error from the lecture notes on psychological reliability estimates.

Psychometric Reliability

The consistency of test scores, indicating that they are sufficiently consistent across repeated measurements and free from measurement error to be useful.

Reliability Coefficient

A statistic that quantifies reliability, ranging from $0$ to $1$.

Measurement Error

Any fluctuation in scores resulting from factors related to the measurement process that are irrelevant to what is being measured.

Classical Test Theory (CTT)

A theoretical framework in psychometrics holding that an observed score ($X$) consists of a true score ($T$) plus measurement error ($E$), represented as $X = T + E$.

True Score

Hypothetical entities that would result from error-free measurement ($T = X - E$) if there were no errors in measurement at all.

Observed Score

The actual score obtained on a test ($X$), which is the sum of the true score ($T$) and measurement error ($E$).

Random Error

Unpredictable fluctuations and inconsistencies of variables in the measurement process that unpredictably increase or decrease test scores and affect score consistency.

Systematic Error

Errors that consistently inflate or consistently deflate test scores in all testing situations; because their influence is consistent, they do not affect the consistency of scores.

Standard Error of Measurement (SEM)

The standard deviation of the distribution of errors ($\frac{}{}$) denoted as $\text{SEM}$ or $\text{O}_{meas}$, which tells on average how much a score varies from the true score.

Domain Sampling Model

A model that evaluates how much error is introduced by using a sample of items instead of the full domain, positing that longer tests with more items reflect the full domain more accurately and are more reliable.

Generalizability Theory

A psychometric framework that replaces the concept of 'true score' with 'universe score' and acknowledges multiple sources of score variability ('facets') across different testing contexts.

Universe Score

A concept in Generalizability Theory reflecting an individual's performance across all possible similar testing conditions.

Time Sampling Error

The variability inherent in test scores as a function of being obtained at one point in time rather than another, assessed via test-retest reliability.

Construct Maturation

The process where the psychological construct being measured changes, develops, or matures over the time elapsed between test administrations.

Inter-Rater Differences

Errors that enter into test scores whenever subjectivity plays a part in scoring, resulting in different raters assigning slightly different scores for the same test performance.

Content Sampling Error

Trait-irrelevant variability in test scores resulting from fortuitous factors related to the specific items included in a test.

Parallel- and Alternate-Forms Reliability

A procedure that estimates content sampling error by administering two or more different forms of a test (identical in purpose but differing in specific content) to the same group and correlating the results.

Internal Consistency Estimates

An estimate of test reliability reflecting inter-item consistency that can be obtained from a single test administration without developing alternate forms or testing twice.

Split-Half Reliability

An internal consistency measure obtained by correlating two pairs of scores from equivalent halves of a single test administered once.

Spearman-Brown Formula

A mathematical formula used to estimate internal consistency reliability from split halves or adjust reliability when a test is shortened or lengthened, calculated as $r_{corrected} = \frac{2r_{half}}{1 + r_{half}}$.

Cronbach's Alpha

An internal consistency measure equal to the mean of all possible split-half correlations corrected by the Spearman-Brown formula, primarily used for items with polytomous response options.