Psychometrics - Semantic 3

Definitions, facts, general knowledge

Validity

the extent to which a test measures what it claims to measure.

Ipsative

refers to a type of measurement where an individual's scores are compared to their own previous scores rather than to others, often used in personality assessments.

Nomothetic

refers to a type of measurement that involves comparing individuals' scores to a larger norm group, focusing on general patterns rather than individual differences.

Item information

The ability of a test item to discriminate between participants' abilities or characteristics.

Construct bias

When the standard error of the measure is systematically different between groups despite them having equivalent means.

CFA model hypotheses

H₀: there is no difference (two-tailed) in goodness of fit between the proposed model and the nested model.

H₁: there is a difference (two-tailed) in goodness of fit between the proposed model and the nested model.

Factor loading

The correlation between a test item and the underlying latent factor it measures, indicating how much variance in the item is explained by the factor.

r_ab

Parallel forms reliability: for a test administered to a person multiple times with different ordering of (comparable) items upon each administration, the proportion of variance in obtained scores attributable to the true score variance.

Dichotomous decision table

A table used to estimate the likelihood (frequency) of obtaining a true positive, true negative, false positive, or false negative.

Predictive bias

A systematic error that occurs when a predictive model performs differently (i.e. different r² values) for different groups, measured by linear multiple regression.

Parallel

Referring to comparable tests, they have the same number of questions, the same question format, the same difficulty, and the same content of items.

Bipolar adjectives

Descriptive terms used in semantic differential scales where two opposing extremes are presented on either side of a numeric (interval or ordinal) scale.

α in an item characteristic curve

Represents the slope (discrimination parameter) of the curve, indicating how well an item differentiates between individuals at different ability levels. A steeper slope indicates better discrimination.

β in an item characteristic curve

Represents the difficulty parameter of the item, indicating the level of ability required for an individual to have a 50% chance of answering the item correctly.

Overall factor solution

Percent of variance among variable measurements explained by the factors and factor loading pattern generated in an exploratory factor analysis.

Ψ

Uniqueness: The proportion of variance in a variable that is not explained by the common factors in an exploratory factor analysis, reflecting the item's specific or unique contribution.

It is computed as 1 - communality.

Statistically significant change

A ratio of the estimated difference between mean pre- and post-test scores to the quantified accuracy of that estimate.

X²/df

Alternative fit index: the ratio of the chi-square statistic to the degrees of freedom, used in goodness-of-fit tests to assess model fit in confirmatory factor analysis. Compared to chi-square alone, it is less sensitive to changes in sample size and more sensitive to measures of invariance.

Interaction of intercept and slope bias

Systematic discrepancies in item quality (slope) across ability levels (intercepts) for the same item.

Information criterion

A statistical measure used for comparing models based on their complexity and goodness of fit, helping to select the best model by penalizing complexity.

It ranges from 0.00 - 1.00, and smaller is better.

Item difficulty

Ratio of the number of people who have passed/endorsed a test item to the number of people total who have taken the test item. A smaller number means fewer people passed/endorsed the item, thus indicating that the item is harder.

Ranges from 0.00 - 1.00.

Predictive bias

A situation where the predictive accuracy of a test varies across different groups, often leading to unfair advantages or disadvantages based on demographic characteristics.

Coefficient kappa (k)

Evaluates the agreement between two or more raters or methods beyond chance, providing a value between -1.00 (perfect disagreement) and 1.00 (perfect agreement).

Programmed test

A type of assessment where items are presented in a structured sequence based on the test taker's ability level.

Coefficient α

Measure of internal consistency, measures the extent to which items on a test are conceptually homogenous.

The proportion of variance, across items representing the same construct, attributable to the true score variance.

Rotation

A statistical method used in factor analysis to simplify the interpretation of factors by changing the orientation of the factor axes.

Slope bias

A type of systematic error in which the relationship between test scores and true ability differs across groups, potentially leading to unfair advantage or disadvantage.

Rosenthal effect

The phenomenon where research participants' performance is influenced by the expectations of the experimenter, often leading to biased outcomes.

Structural validity

The extent to which a factor analysis explains (or predicts) variance in scores.

Thurstone scale

A Likert scale in which all numerical values are assumed to be equidistant and therefore interval as opposed to ordinal.