Psychological Testing: Norms, Reliability, and Validity

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/38

flashcard set

Earn XP

Description and Tags

Vocabulary practice flashcards covering psychometric norms, standard score transformations, classical test theory, reliability estimation, validity types, item analysis, and test development principles.

39 Terms

1
New cards

Standard Scores (zz-scores)

Linear raw-score transformations that express an individual's score in terms of distance from the mean in standard deviation units, preserving equal intervals across the distribution and enabling direct comparison across different tests.

2
New cards

Percentile Rank

A raw-score transformation indicating the exact percentage of scores in a normative distribution that fall below a specific raw score on an ordinal, non-interval scale.

3
New cards

Percentile Distortion

The scaling phenomenon where small raw score differences near the center of a normal distribution result in exaggerated percentile rank jumps, whereas large raw score differences in the extreme tails produce minimal percentile rank changes.

4
New cards

Empirical Rule of Normal Distributions

The mathematical properties of a normal curve stating that approximately 68.26%68.26\% of scores fall within ±1 SD\pm 1\,SD, 95.44%95.44\% within ±2 SD\pm 2\,SD, and 99.74%99.74\% within ±3 SD\pm 3\,SD from the mean.

5
New cards

Deviation IQ Score

A standardized score distribution with a mean of 100100 and a standard deviation of 1515, calculated using X′=z(15)+100X' = z(15) + 100 (for example, a score of −1 SD-1\,SD equals 8585).

6
New cards

T-Score

A standardized score distribution with a mean of 5050 and a standard deviation of 1010, calculated using T=z(10)+50T = z(10) + 50 (for example, a score of −1 SD-1\,SD equals 4040).

7
New cards

CEEB Score

A standardized score distribution commonly used in college entrance examinations with a mean of 500500 and a standard deviation of 100100, calculated using CEEB=z(100)+500CEEB = z(100) + 500 (for example, a score of −1 SD-1\,SD equals 400400).

8
New cards

Standardized Test

An assessment constructed by professionals featuring uniform administration and objective scoring guidelines, direct score comparability across examinees, and norm-referenced interpretation via representative standardization samples.

9
New cards

Test Reliability (r11r_{11} or rxxr_{xx})

The consistency or repeatability of test scores across administrations, alternate forms, or internal items, mathematically defined as the ratio of true score variance to total observed score variance (σT2σX2\frac{\sigma_T^2}{\sigma_X^2}).

10
New cards

Classical Test Theory (CTT)

A psychometric measurement framework holding that an observed test score (XX) is the sum of an examinee's true score (TT) and unsystematic measurement error (ee), expressed as X=T+eX = T + e.

11
New cards

CTT Measurement Error Assumptions

Theoretical assumptions asserting that measurement errors are random unsystematic fluctuations, have a theoretical mean of zero (Me=0M_e = 0), and are uncorrelated with true trait scores (rTe=0r_{Te} = 0).

12
New cards

Standard Error of Measurement (SEMSEM)

The standard deviation of the theoretical normal distribution of test scores an individual would obtain if tested repeatedly, computed as SEM=sx1−r11SEM = s_x\sqrt{1 - r_{11}}, which quantifies the average amount of unsystematic measurement error.

13
New cards

Confidence Interval (Psychometrics)

A band of error placed around an observed score (XX) where an examinee's true score is likely to fall, calculated as X±1(SEM)X \pm 1(SEM) for 68%68\%, X±2(SEM)X \pm 2(SEM) or 1.96 SEM1.96\,SEM for 95%95\%, and X±3(SEM)X \pm 3(SEM) for 99%99\%.

14
New cards

Test-Retest Reliability

A reliability estimation method using a single test form administered across two separate sessions to evaluate score stability over time, with administration time and temporal fluctuations serving as the primary source of error.

15
New cards

Alternate-Forms Reliability

A reliability estimation method correlating scores between two equivalent test forms; administered immediately to evaluate item sampling error, or delayed across two sessions to evaluate both item sampling and temporal changes.

16
New cards

Split-Half Reliability

An internal consistency method that correlates scores from two halves of a single test administered on one occasion and applies the Spearman-Brown correction (rSBr_{SB}) to compensate for shortened test length.

17
New cards

Coefficient Alpha (Cronbach's Alpha)

An internal consistency reliability coefficient yielding the mean reliability across all possible split-half combinations, specifically designed for tests with multipoint or Likert-type items.

18
New cards

Kuder-Richardson Formula 20 (KR-20)

A specialized internal consistency formula for dichotomously scored items (such as Right/Wrong or True/False) when items vary in item difficulty.

19
New cards

Kuder-Richardson Formula 21 (KR-21)

A simplified internal consistency formula for dichotomously scored items that assumes all items possess equal difficulty.

20
New cards

Interscorer Reliability

A consistency estimate obtained by correlating ratings across independent examiners, commonly applied to subjectively scored tests or behavioral observations to evaluate scorer consistency.

21
New cards

Item Sampling Error

Measurement error arising from selecting a particular subset of items from the population of all possible questions, evaluated in immediate alternate-forms, split-half, and internal consistency estimates.

22
New cards

Administration Time Error

Measurement error caused by temporal fluctuations in testing environments (such as noise or temperature) or internal states of examinees (such as fatigue, illness, or practice effects) across testing occasions.

23
New cards

Item Characteristic Curve (ICC)

A curve in Item Response Theory plotting latent trait level (θ\theta, on a scale from −3-3 to +3+3) against the probability of answering correctly (0.00.0 to 1.01.0), illustrating item difficulty at p=0.5p = 0.5 and item discrimination by its slope.

24
New cards

Item Information Function (IIF)

A psychometric function plotting trait level against the precise amount of measurement precision an item contributes, showing that easy items provide precision at low trait levels and difficult items provide precision at high trait levels.

25
New cards

Validity Coefficient (rxyr_{xy})

A correlation coefficient computed between test scores (XX) and an external criterion variable (YY), whose square (rxy2r_{xy}^2) represents the coefficient of determination or criterion variance accounted for by the test.

26
New cards

Content Validity

The primary form of validity for achievement tests, ensuring that test items adequately sample and represent the universe of knowledge, skills, and material students were expected to master.

27
New cards

Criterion-Related Validity

The degree to which test scores correlate with an external criterion variable, evaluated through either predictive validity (forecasting future criteria) or concurrent validity (criteria collected at the same time).

28
New cards

Predictive Validity

A subtype of criterion-related validity determined by correlating test scores with a criterion measure obtained at a future date (such as SAT or GRE scores predicting college GPA).

29
New cards

Concurrent Validity

A subtype of criterion-related validity determined by correlating test scores with a criterion measure collected simultaneously.

30
New cards

Construct Validity

The primary form of validity for personality tests, evaluating whether test scores align with theoretical expectations regarding abstract psychological traits.

31
New cards

Convergent Validity

A component of construct validity demonstrated when a new test correlates significantly (r≥.50r \ge .50) with established tests that measure the same or theoretically related constructs.

32
New cards

Discriminant Validity

A component of construct validity demonstrated when a test exhibits low or non-significant correlations with measures of distinctly different, unrelated constructs.

33
New cards

Reliability Ceiling on Validity

The mathematical principle stating that test unreliability acts as an upper bound on validity, defined by the formula rxy≤rxx⋅ryyr_{xy} \le \sqrt{r_{xx} \cdot r_{yy}}; a test cannot be valid if it lacks reliability.

34
New cards

Item Reliability

The correlation between an individual test item score and the total test score (point-biserial correlation rpbr_{pb}), which assesses an item's contribution to internal consistency.

35
New cards

Item Validity

The correlation between an individual item score and an external criterion variable (point-biserial correlation rpbr_{pb}), which is prioritized over item reliability when predicting complex, multi-faceted criteria.

36
New cards

Item Difficulty Index (pp)

The proportion of examinees answering an item correctly, ranging from 0.00.0 to 1.001.00, where values below the lower bound are difficult, between the lower bound and 0.700.70 are moderately difficult, and above 0.700.70 are easy.

37
New cards

Item Discrimination Index (dd)

A metric ranging from −1.00-1.00 to +1.00+1.00 indicating how effectively an item differentiates high from low performers; items with d≥.30d \ge .30 are acceptable, d=.20−.29d = .20 - .29 are marginal, d=.10−.19d = .10 - .19 are very marginal, and d<.09d < .09 are poor.

38
New cards

Item Difficulty Lower Bound (L.B.L.B.)

The cutoff defining acceptable difficulty for multiple-choice items accounting for guessing chance (g=1kg = \frac{1}{k}) and sample size (NN), computed as L.B.=g+1.645g(1−g)NL.B. = g + \frac{1.645\sqrt{g(1-g)}}{\sqrt{N}}.

39
New cards
<p>Chance Sampling Margin for Item Difficulty Lower Bound</p>

Chance Sampling Margin for Item Difficulty Lower Bound

The statistical sampling error term added to chance performance (gg) in the formula L.B.=g+1.645g(1−g)NL.B. = g + \frac{1.645\sqrt{g(1-g)}}{\sqrt{N}}, which accounts for sample size (NN) and guessing probability (g=1kg = \frac{1}{k}).