Psychometrics and Psychological Testing: Scores, Distributions, and Reliability

A set of vocabulary flashcards defining key psychometric concepts, score types, reliability methods, correlation types, levels of measurement, and item response theory principles.

Central tendency

A descriptive statistic that describes a typical score, consisting of the mean (average), median (middle), and mode (most common).

Variability

A descriptive statistic that describes score spread, including range, variance, and standard deviation (SD).

Normal distribution

A symmetric, bell-shaped distribution where approximately $68\%$ of scores fall within $\pm 1\text{ SD}$, $95\%$ within $\pm 2\text{ SD}$, and $99.7\%$ within $\pm 3\text{ SD}$.

Skewed distribution

An asymmetric distribution; positive skew has a tail toward high scores, while negative skew has a tail toward low scores.

Kurtosis

A characteristic that describes how peaked or flat a distribution is.

Modality

The number of peaks in a distribution (e.g., unimodal or bimodal).

Norm-referenced test

A test that compares an individual's performance with that of a norm group.

Criterion-referenced test

A test that compares performance with a defined standard or skill level.

Norms

Comparative data that show how scores compare with a relevant population, requiring large representative samples to provide meaningful subgroup comparisons.

Expectancy table

A table that estimates the likelihood of an outcome based on a test score.

Simple random sampling

A sampling procedure that gives each person in the population an equal chance of selection.

Stratified random sampling

A sampling procedure that samples individuals from important subgroups.

Cluster sampling

A sampling procedure that samples natural groups, such as schools or classrooms.

z score

A transformed standard score that has a typical mean of $0$ and a typical SD of $1$, indicating how many standard deviations a score is from the mean.

T score

A transformed score with a typical mean of $50$ and a typical SD of $10$.

IQ scale

A standardized score scale characterized by a typical mean of $100$ and a typical SD of $15$.

CEEB/SAT scale

A transformed score scale characterized by a typical mean of $500$ and a typical SD of $100$.

Stanine

A transformed score scale characterized by a typical mean of $5$ and a typical SD of $2$.

Percentile rank

The percentage of the norm group scoring below a person; it does not represent percent correct.

Standard scores

Scores that allow comparisons of performance on a common scale.

Classical test theory (CTT)

A theoretical framework that treats an observed score as the person's true score plus measurement error: $X = T + E$.

Systematic error

Error that creates consistent bias and can threaten test validity.

Unsystematic error

Random fluctuation that generally lowers test reliability.

Reliability

The consistency of scores; in classical test theory, the degree to which observed scores reflect true scores rather than varying due to measurement error.

Test-retest reliability

A type of reliability measuring stability over time, which can be affected by the duration between test administrations and real change in the construct.

Parallel forms reliability

A type of reliability evaluating consistency across equivalent test versions, which is affected by differences in item samples.

Split-half reliability

An internal consistency measure calculated by correlating scores obtained from two halves of one test.

Spearman-Brown formula

A statistical formula that adjusts a split-half correlation to estimate reliability for the full-length test.

Coefficient alpha

An internal consistency reliability estimate designed for items with multiple possible scores.

KR-20

An internal consistency reliability coefficient designed for dichotomous items (e.g., right/wrong).

KR-21

A simplified KR-20 estimate of internal consistency that assumes all items possess similar difficulty.

Interscorer reliability

The degree of agreement between different people scoring the same performance.

Standard error of measurement (SEM)

An estimate of the typical error around an observed score, calculated using $\text{SEM} = \text{SD} \times \sqrt{1 - \text{reliability}}$.

Correlation

A statistical measure describing the direction and strength of a relationship between variables, ranging from $-1$ to $+1$.

Pearson correlation

A correlation index used to evaluate the linear relationship between two continuous variables.

Spearman correlation

A correlation coefficient used for ranked or ordinal data.

Phi coefficient

A correlation coefficient used to measure the relationship between two dichotomous variables.

Point-biserial correlation

A correlation coefficient used between one continuous variable and one genuinely dichotomous variable.

Biserial correlation

A correlation coefficient used between one continuous variable and one variable artificially split into two categories.

Nominal scale

A level of measurement consisting purely of distinct qualitative categories.

Ordinal scale

A level of measurement consisting of ranked categories.

Interval scale

A level of measurement with equal intervals between scale units but no true zero.

Ratio scale

A level of measurement characterized by equal intervals and a meaningful, absolute zero.

Speed test

A test consisting of easy items administered under strict time limits, where performance depends heavily on the number of items completed.

Power test

A test that provides sufficient time and presents items of increasing difficulty to measure the highest level of performance someone can solve.

Criterion-referenced reliability

A form of reliability that focuses on whether an examinee is classified consistently as meeting or not meeting a set standard, particularly near the cutoff score.

Standard error of difference (SED)

A statistic estimating uncertainty in the difference between two scores, calculated as $\text{SED} = \sqrt{\text{SEM}_1^2 + \text{SEM}_2^2}$ when errors are independent.

Item response theory (IRT)

A measurement framework that models the probability of answering an item correctly across different examinee ability levels.

Item characteristic curve (ICC)

An IRT curve that illustrates the probability of answering an item correctly as a function of ability level.

Item information function

A function in IRT showing the ability range where an individual test item measures most precisely.

Computerized adaptive testing (CAT)

A testing method supported by IRT that dynamically selects subsequent items based on the examinee's performance on earlier answers.