psychometrics final use this one

Measurement Bias

Systematic error in measurement that causes scores to differ for reasons unrelated to the construct. Example: A math test requiring heavy reading penalizes students with low reading skills, even if they are good at math.

Construct-Irrelevant Variance

Variance in scores caused by factors unrelated to the intended construct. Application: If anxiety affects test performance, the test partly measures anxiety instead of ability.

Construct Underrepresentation

When a measure fails to capture essential parts of the construct. Example: A depression scale measuring only sadness misses other symptoms like sleep disturbance or anhedonia.

Item Bias

When individuals from different groups with equal ability do not have equal probability of getting an item correct. Example: A sports trivia item favoring students who watch American football.

Fairness vs. Bias

Bias refers to systematic, construct-unrelated score differences; fairness refers to equitable interpretation and use of scores. A test may have no biased items but still be used unfairly.

Cultural Bias

When test items assume knowledge, practices, or values more common in one culture than another. Example: Asking about Thanksgiving traditions disadvantages non-U.S. students.

Linguistic Bias

When wording, translations, idioms, or reading level unfairly affect certain groups. Example: “Hit the books” confuses English-language learners.

Response Bias

Consistent tendency to respond in certain ways regardless of item content. Common forms include acquiescence and social desirability.

Acquiescence Bias

Tendency to agree with items regardless of meaning. Consequence: Inflated correlations, misleading factor structures. Example: A respondent answering “agree” to all items.

Social Desirability Bias

Responding in a way that is socially approved rather than truthful. Example: Underreporting prejudice or overreporting charitable behavior.

Extreme Response Bias

Tendency to choose only the highest or lowest response options. Seen in some cultural groups; affects scale reliability.

Moderacy Bias

Opposite of extreme responding; preference for middle/neutral options. Example: Students from harmony-valuing cultures avoiding extreme answers.

Speededness Bias

Occurs when time limits prevent examinees from demonstrating true ability. Example: A reading test with too little time disadvantages slower processors.

Stereotype Threat

Bias that occurs when awareness of stereotypes affects performance. Example: Women underperform on math tests when gender stereotypes are highlighted.

Group Differences vs. Bias

Group differences alone do NOT indicate bias. Bias exists only if differences arise from construct-irrelevant factors.

Differential Item Functioning (DIF)

Evidence that an item functions differently for different groups after controlling for ability. DIF is a statistical indicator of potential item bias.

Uniform DIF

One group consistently performs better on the item across all ability levels. Example: A vocabulary item using culturally specific terms.

Non-Uniform DIF

Item favorability changes depending on ability level; involves interaction between trait level and group membership.

Mantel–Haenszel Method

A statistical technique used to detect DIF between two groups by comparing odds of correct responses at matched ability levels.

Logistic Regression DIF

Detects DIF by modeling the probability of a correct response using group membership and ability. Identifies both uniform and non-uniform DIF.

IRT-Based DIF Analysis

Uses item response theory parameters (difficulty, discrimination) to evaluate whether items behave differently across groups.

Translation Bias

Errors introduced during translation such as literal translations, loss of nuance, or cultural mismatch. Example: A mental health term lacking direct equivalent in another language.

Content Bias

Items measuring content more familiar or relevant to one group. Example: “Snow-related hazards” items for students in warm climates.

Format Bias

When the layout, response options, or modality disadvantages a group. Example: Computer-based tests disadvantaging those unfamiliar with computers.

Administration Bias

When test administration conditions affect groups differently. Example: Testing ESL students in noisy rooms where language processing is harder.

Scoring Bias

When subjective scoring is influenced by stereotypes or expectations. Example: Essay graders scoring names like “Emily” higher than “Lakisha.”

Test Sampling Bias

Occurs when performance depends heavily on exposure to test-specific strategies. Example: Students from wealthy schools trained in test-taking strategies scoring higher.

Predictive Bias

Occurs when test scores predict outcomes differently for different groups. Example: SAT underpredicting GPA for women or overpredicting for men.

Impact vs. Bias

Impact = group differences that reflect real differences in the construct; Bias = differences driven by construct-irrelevant factors.

Why Bias Matters

Bias reduces validity, undermines fairness, and leads to inaccurate decisions about individuals or groups.

Sources of Bias (Summary)

Test content, language, culture, administration, scoring, response styles, anxiety, test-wiseness, stereotype threat, and translation issues.

Reducing Bias Strategy 1: Diverse Item Writing

Include diverse writers to prevent culturally narrow perspectives. Application: Avoids items relying on specific cultural knowledge.

Reducing Bias Strategy 2: Clear Wording

Use simple, unambiguous wording. Example: Replace idioms with literal language.

Reducing Bias Strategy 3: Pilot Testing

Test items with people from diverse backgrounds to detect unexpected difficulty or misunderstanding.

Reducing Bias Strategy 4: Statistical DIF Screening

Use Mantel–Haenszel, logistic regression, or IRT to identify biased items.

Reducing Bias Strategy 5: Remove or Revise Items

Items showing DIF or cultural disadvantage should be rewritten or removed.

Reducing Bias Strategy 6: Standardized Administration

Ensure identical instructions, timing, environment, and procedures.

Reducing Bias Strategy 7: Accommodations

Provide accommodations (extended time, translated instructions) when they do not alter the construct being measured.

Bias and Validity

Bias threatens validity by introducing systematic error unrelated to the construct.

Bias in Self-Report Measures

Self-report scales are particularly vulnerable due to response styles, social desirability, and interpretation differences.

Bias in Performance Tests

Performance tests are vulnerable to speededness, stereotype threat, and differential familiarity with content.

Example of Bias in Schools

A reading comprehension test using culturally specific stories may penalize immigrant students unfamiliar with the narrative context.

Example of DIF Analysis

Application: If a math item shows higher success for boys than girls at the same ability level, that item displays DIF and should be reviewed.

Real-World Application: Hiring Tests

A job test requiring advanced vocabulary may unfairly disadvantage capable applicants with lower education access.

Real-World Application: Clinical Measures

A PTSD scale written using Western symptom definitions may under-detect symptoms in collectivist cultures.