Psychometrics final

What is measurement bias?

Systematic error that causes test scores to differ for reasons unrelated to the construct being measured.

Impact of bias

Bias threatens the validity, fairness, and interpretability of test scores.

Construct-irrelevant variance

Variance in scores caused by factors other than the intended construct (e.g., reading difficulty, anxiety, stereotypes).

Construct underrepresentation

When a measure fails to capture important components of the construct, limiting validity.

Sources of bias

Translation issues, cultural differences, test anxiety, item wording, stereotype threat, and differential familiarity with content.

Item bias definition

Occurs when individuals from different groups (e.g., gender, ethnicity) with equal ability do not have equal probability of answering correctly.

Differential Item Functioning (DIF)

Statistical evidence that an item functions differently for different groups after controlling for ability.

Uniform DIF

Item consistently favors one group across all levels of the underlying trait.

Non-uniform DIF

Item favors different groups at different trait levels; relates to item-trait interaction.

Methods for detecting bias

DIF analysis (IRT-based), Mantel-Haenszel procedure, logistic regression approaches.

Cultural bias

Occurs when test items assume knowledge or experiences more typical of one cultural group than another.

Linguistic bias

Arises from complex vocabulary, idioms, or translation differences that disadvantage certain examinees.

Response bias

Systematic tendencies in responding unrelated to actual construct levels (e.g., social desirability, acquiescence).

Social desirability bias

Participants answer in a way they believe is socially approved rather than truthful.

Acquiescence bias

Tendency to agree with statements regardless of content, impacting Likert-type scale validity.

Ways to reduce bias

Use clear wording, avoid cultural references, pilot test items, conduct DIF analyses, and include diverse item-writing teams.

Test fairness

Fairness means scores reflect true differences in the construct, not group membership or irrelevant factors.

Bias vs. Fairness

Bias refers to specific score distortions; fairness is a broader concept involving equitable test use and interpretation.

Importance of examining bias

Bias detection improves validity, fairness, and the ethical use of measurement tools in social science research.