Test Construction & Psychometrics

The process of developing a test including defining the purpose, item writing, pilot testing, and standardization.

Test Construction

A statistic that measures inter-rater agreement for categorical items, correcting for chance agreement.

Kappa Coefficient

A measure of internal consistency reliability for dichotomous items (e.g., KR-20, KR-21).

Kuder-Richardson Formula

In item response theory, shows the probability of a correct response as a function of ability.

Item Characteristic Curve (ICC)

The proportion of test-takers who answer an item correctly; lower values indicate harder items.

Item Difficulty

A theory of test scoring that considers the difficulty and discriminatory power of items.

Item Response Theory (IRT)

Assessments where scores are interpreted relative to a normative sample.

Norm-Referenced Tests

Assessments where performance is measured against a fixed set of criteria.

Criterion-Referenced Tests

The predictor is the variable used to forecast another (criterion) variable.

Predictor/Criterion

A standard score with a mean of 50 and a standard deviation of 10.

T-Score

A statistical adjustment to account for the probability of guessing answers correctly on multiple-choice tests.

Correction for Guessing

A technique to assess construct validity using multiple traits and methods.

Multitrait-Multimethod Matrix (MTMM)