Lecture 11: Item Response Theory | Psychometric Theory

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/70

encourage image

There's no tags or description

Looks like no tags are added yet.

71 Terms

1
New cards

What two types of Item Response Theory

Dichotomous and Non-Dichotomous Item Response Theory.

2
New cards

What is Factor Analysis used for?

To determine the quality of items and identify which items load onto a specific latent construct.

3
New cards

What is Item Response Theory (IRT) used for?

To determine which items are best for specific groups and examine how items operate within those groups.

4
New cards

Why is IRT useful across different demographic groups?

not all items are created equally; some work better for certain groups (e.g., men vs. women, different ages).

5
New cards

What is the Classical testing theory equation?

X = T + E

6
New cards

What is the major limitation of Classical testing theory?

All error is lumped into one term (E), so we cannot separate random error from systematic error.


It cannot identify sources of error at the item level.

7
New cards

How does IRT differ from CTT?

IRT examines scores on each item and uses hierarchical mathematical modeling to determine scoring patterns

8
New cards

What major advantage does IRT have over CTT?

IRT allows error to be examined at the item level.

9
New cards

What are the two components of Item Analysis?

Item Difficulty and Item Discriminability.

10
New cards

What is a test floor?

A test with many easy items.

11
New cards

What is a test ceiling?

A test with many hard items

12
New cards

When would you design a test with a high ceiling?

When testing medical students because the test should contain difficult items.

13
New cards

When would you design a test with a floor?

When testing special-needs students because the test should contain easier items

14
New cards

What standardized test is an example of a test with a high ceiling?

The SAT

15
New cards

What does test sensitivity measure?

How sensitive a test is in determining the presence of a disorder.

16
New cards

What happens when a test has low sensitivity?

It produces many false positives

17
New cards

What is an example of a low-sensitivity test discussed in class?

Home COVID tests (approximately 50-60% sensitivity).

18
New cards

What happens when a test has high sensitivity?

It produces few false negatives.

19
New cards

If a highly sensitive test says "negative," what can you conclude?

You can trust that negative result

20
New cards

What are examples of highly sensitive tests?

pregnancy tests

21
New cards

What important caution did the professor give about sensitivity?

Sensitivity only matters if the test is administered properly

22
New cards

What does a negative skew indicate?

(Tail on the left)

An easy test where most examinees score high.

23
New cards

What does a positive skew indicate?

(Tail on the right)

A difficult test where most examinees score low.

24
New cards

What determines the direction of skew?

The tail. Tail = skew.

25
New cards

How is item difficulty defined (Dichotomous?

The percentage of people who answer an item correctly.

26
New cards

If 84% of people answer an item correctly, what is its difficulty value?

.84

27
New cards

In item difficulty, what does a higher value mean?

The item is easier

28
New cards

In item difficulty, what does a lower value mean?

The item is harder

29
New cards

What is the ideal average item difficulty for most tests?

.50

30
New cards

What item difficulty range is generally preferred?

Between .30 and .70.

(So it averages to .50 for entire test)

31
New cards

Why might medical school exams have difficulty values around .20-.30?

The questions need to be difficult to ensure competency

32
New cards

Why might tests for disabled students have values around .80-.90?

Easier items are needed for the population being tested

33
New cards

What does item discriminability measure?

Whether people who do well on an item also do well on the entire test

34
New cards

What are the groups in discriminability for dichotomous

Upper

Middle

Lower

35
New cards

How is discrimination calculated using the extreme group method? (Dichotomous)

U - L

36
New cards

What does a higher discrimination value indicate? (Dichotomous)

Greater distinction between upper and lower groups

37
New cards

How is item difficulty calculated in the extreme groups example? (Dichotomous)

U + M + L

38
New cards

What is item discriminability? (Dichotomous)

It determines whether people who do well on an item also do well on the entire test

39
New cards

What does a higher discrimination index mean? (Dichotomous)

etter discrimination between high and low performers.

40
New cards

How is discriminability found in dichotomous scoring (methods)

  1. extreme groups method

  2. Point Biserial method


41
New cards

What does point biserial range from?

-1 to +1

42
New cards

What does a point biserial value that is positive and closer to 1 tell us?

Those scoring higher on the exam got this question correct

43
New cards

What does a negative point biserial value tell us?

There is likely a problem with the item

44
New cards

Can point biserial values be seen as percentages?

No

45
New cards

What does a good discriminating ICC curve look like? (Dichotomous)

A smooth upward curve where higher scorers are more likely to answer correctly.

46
New cards

What does a flat ICC curve indicate? (dichotomous)

No discrimination across score groups

47
New cards

What does the "overthinking" ICC pattern show? (dichotomous)

Lower and middle scorers improve, but upper scorers perform worse.

48
New cards

What types of constructs are measured with non-dichotomous IRT?

Personality traits, attitudes, depression, and other continuous psychological constructs

49
New cards

Why doesn't the traditional bell curve approach work well for non-dichotomous constructs?

Psychological traits like depression do not fit neatly into lower, middle, and upper groups.

50
New cards

How are non-dichotomous groups typically conceptualized?

Mild, moderate, and severe levels of symptom severity

51
New cards

What is an Item Response Function?

A graph showing the relationship between trait level and probability of endorsing an item

52
New cards

What is the basic unit of non-dichotomous IRT?

The Item Response Function

53
New cards

What indicates item difficulty on a non-dichotomous ICC curve?

The inflection point of the curve

54
New cards

Which curve represents the most difficult item?

(non-dichotomous)

The curve furthest from the Y-axis.

55
New cards

What does a higher level of difficulty mean in

non-dichotomous IRT?

A person must have a higher level of the trait/severity to endorse the item

56
New cards

How is discrimination determined on a nondichotomous ICC?

Steepness of the slope

57
New cards

What does a steeper slope indicate in nondichotomous ICC ?

Greater discrimination

58
New cards

What is being discriminated in non-dichotomous IRT?

Different levels of symptom severity (e.g., mild vs. severe).

59
New cards

What does the A parameter represent?

(Non-dichotomous)

Item discrimination

how well a question can tell the difference between people with high ability/severity and people with low ability/severity

60
New cards

What does a higher A value indicate (non-dichotomous)

Higher discrimination.

61
New cards

What does a higher B value indicate (non-dichotomous)

Higher difficulty or greater severity

62
New cards

What does the B parameter represent?

Item difficulty

the degree of severity the question is. 

63
New cards

For dichotomous tests, what is the key advantage of IRT?

Item difficulty remains stable across different groups of test-takers.

Difficulty is matched to the aptitude of the test-takers ability

64
New cards

For non-dichotomous tests, what is the key advantage of IRT?

It allows comparison of how different demographic groups respond to the same items and helps identify test bias.

(across gender, age, other demographic affiliations)

65
New cards

What is test bias?

A scoring difference between demographic groups.

66
New cards

What does ICC stand for?

Item characteristic curves

67
New cards

What are you looking for when looking at ICC curves (discriminability wise)

Are the groups performing as expected?

68
New cards

What points does Item response theory hit?

  • analyze items and scales

  • measure psychological constructs

  • compare individuals on psychological constructs


(AMC)


69
New cards

Why are the groups changed for Dichotomous and Nondichotomous.

Mild, moderate, severe, maps better onto psychological constructs (nondichotomous)

70
New cards

What would happen if we were to harness IRT in nondichotomous scoring systems?

We could get over test bias - we aren’t there yet in the field


Reduce over pathologizing people and provide accurate diagnoses based on demographic affiliations

71
New cards