Psychological Testing and Assessment: Chapter 5 - Reliability

Flashcards covering the fundamental concepts of reliability, error variance, and different psychometric models based on lecture materials.

Reliability

In the field of psychometrics, this term refers to consistency in measurement, regardless of whether the results are positive or negative.

Reliability Coefficient

A statistic used to quantify reliability, typically ranging from $0$, indicating no reliability, to $1$, indicating perfect reliability.

Measurement Error

The inherent uncertainty associated with any measurement, consisting of both avoidable mistakes and inevitable imprecision.

True Score ($T$)

A theoretical measurement of a quantity that would be obtained if there were no measurement error at all.

Construct Score

A person's standing on a theoretical variable, such as reading ability or depression, independent of any specific measurement tool.

Observed Score ($X$)

The actual result obtained from a measurement, relating to the true score and error through the equation $X = T + E$.

True Variance

Variance in test scores that is attributed to genuine differences between individuals.

Error Variance

Variance in test scores attributed to irrelevant or random sources, often identified as noise.

Random Error

Unpredictable fluctuations and inconsistencies in the measurement process that tend to cancel each other out in the long run.

Systematic Error

Error that influences test scores in a constant direction, either consistently inflating or consistently deflating results.

Bias

A statistical term referring to the degree to which systematic error predictably influences a measurement.

Item Sampling

A source of error variance that refers to differences in the specific items selected for inclusion within or between tests.

Carryover Effects

A situation where the act of measurement itself alters the quantity being estimated, such as practice effects or fatigue effects.

Test-Retest Reliability

An estimate obtained by correlating results from the same group of people on two separate administrations of the identical test.

Coefficient of Stability

An estimate of test-retest reliability specifically when the time interval between the two test administrations is greater than six months.

Parallel Forms

Different versions of a test where the means and variances of the observed scores are equal for each form.

Alternate Forms

Different versions of a test designed to be equivalent in content and difficulty but not meeting the strict requirements of parallel forms.

Internal Consistency

A measure of reliability based on the degree to which different items on a single scale relate to one another.

Split-Half Reliability

A method of estimating reliability by correlating two equivalent halves of a test administered a single time.

Spearman-Brown Formula

A mathematical tool used to estimate the internal consistency of a test that has been shortened or lengthened.

Coefficient Alpha (Cronbach's Alpha)

A measure of internal consistency representing the mean of all possible split-half correlations, corrected by the Spearman-Brown formula.

McDonald's Omega

A measure of reliability that estimates internal consistency without assuming all test item loadings ($\lambda$) are equal.

Inter-Scorer Reliability

The degree of agreement and consistency between two or more independent raters or judges.

Classical Test Theory ($CTT$)

The most widely used psychometric model, also known as the true score model, which assumes an observed score is the sum of a true score and error.

Generalizability Theory

A theory suggesting that a person's test scores vary due to specific variables in the testing situation, referred to as facets.

Universe Score

The long-term average of a person's performance in a given testing situation, analogous to a true score in classical theory.

Item Response Theory ($IRT$)

A family of psychometric methods that models the probability of a specific performance level based on a person's latent trait or ability.

Standard Error of Measurement ($SEM$)

The standard deviation of a theoretically normal distribution of test scores that provides a measure of precision for an observed score.

Confidence Interval

A range or band of scores that is likely to contain the true score, calculated using the standard error of measurement.

Standard Error of the Difference ($\\sigma_{diff}$)

A statistical measure used to determine if the difference between two scores is large enough to be considered statistically significant.