Psychometrics Lecture #7

What is a standardization group?

A group of test takers who represent the population for whom the test was designed

What is a norm?

The performance of the standardization group

Why are norms important?

Tests are only as good as their norms. If the norms are problematic, the test is problematic.

Why is the BMI problematic?

A BMI is normed on the general pop. so many athletes (Lebron) are classified as obese despite having high muscle mass

Why must clinicians know the norms of their tests?

To interpret and apply scores correctly

What are developmental norms?

Norms that compare performance to expected developmental progress

What is mental age?

How far along the normal developmental path an individual has progressed

What is the absolute threshold for determining the mental age?

80%

What is the major problem with mental age scores?

Parents often interpret them literally

What does a mental age of 10 actually mean for a 7-year-old?

The child performs like the average 10-yr-old on that task, not that they literally function as a 10 yr old

What is our role when discussing mental age scores?

To demystify the results and explain their meaning

What is tracking?

Determining growth along a specific biological pathway

example of tracking

Growth charts

What is an ordinal scale?

A scale designed to identify the developmental stage reached in a sequence of behaviors or functions

Ordinal scares are most commonly used during what period?

Infancy

Example of ordinal scale behaviors

Developmental milestones

Why are developmental milestones useful?

They generally occur in a predictable order regardless of SES or culture

What is a grade equivalent?

A score determined by comparing a child’s raw score to the average scores earned by students in various grades

what does a grade equivalent of 3.9 mean?

The score resembles the average performance of a strudent in the 9th month of 3rd grade

What is the biggest problem with grade equivalents?

They are frequently misinterpreted

T or F: Grade equivalents are percentile ranks

False

A student has a grade equivalent of 3.9 in reading and 3.9 in math. Does this mean equal standing in both subjects? Why?

No. A 3.9 in reading might be in the 95th percentile while a 3.9 in math might be in the 68th percentile

What does a grade equivalent tell you?

Which grade level’s avg performance matches the student’s score

What does a grade equivalent NOT tell you?

Percentile rank or standing relative to peers

If a 4th grader earns a grade equivalent of 9.0 in math, what does this mean?

Their score resembles the average raw score of a 9th grader. This does not mean they are doing 9th grade math

What are within-group norms?

comparing a person’s performance to a similar reference group

Example of within group norms?

Comparing a 10 yr olds score to scores of other 10 yr olds

What is percentile?

The percentage of individuals in the norm group who scored below a specific score

What does the 90th percentile mean?

The person scored higher than 90% of the standardization sample

Advantage of percentiles?

Easy to understand

Major limitation of percentiles?

Unequal units at the extremes of the distribution

What is the most important population parameter?

The census

Why is the census important?

It tells us who is represented in the US population

Why was the WISC-V sample constructed to measure census data?

To create a nationally stratified representative sample

Approximately how many childen were in the WISC-V standardization sample?

2,200

Why must a sample resemble the population?

Otherwise the measure may not accurately represent the population

What is a standard score?

A score expressing a person’s distance from the mean in terms of the mean and SD W

What do standard scores tell us?

How far above or below average someone is

What is another name for a linear transformation

Z-score

Mean and SD of a T score?

Mean = 50

SD = 10

Mean and SD of WAIS IQ scores?

Mean = 100

SD = 15

When should Raven Progressive Matrices be considered instead of the WAIS?

When someone does not speak English fluently - because it is a nonverbal test measuring abstract reasoning and general intelligence

What do scaled scores account for?

Grade-lvl norms and test difficulty

Can scaled scores increase while percentile ranks stay the same?

Yes. Children are expected to perform better as they age and their peers improve as well

In the metropolitan achievement test example, why did scaled scored rise from 571 to 644 to 674?

The child was making normal progress and taking more difficult tests

Why did the percentile remain 50?

The child remained avg relative to peers

What is a Stanine?

A norm referenced score that places performance into one of nine categories

Stanine range?

1-9

Mean and SD of Stanines

Mean = 5

SD = 2

What are the ranges for stanines?

1-3: Below avg

4-6: avg

7-9: above avg

What percentile range typically corresponds to a stanine of 5?

a 40th-59th percentile

What percentile range is typically assigned a stanine of 6?

60th-76th percentile

What stanine is assigned to the top 4%

9

What stanine is assigned to the bottom 4%?

1

What is domain referenced testing?

A test describing specific skills or tasks that a person can perform

Does domain-referenced testing use norms?

No

Who is the child compared to in domain referenced testing?

themselves

What question is domain referenced testing trying to answer?

“What can this child do and are they improving?"

Example of domain referenced testing?

Tracking improvement in tying shoes, using utensils, or dressing independently

What are the 2 requirements for standardization?

large sample and representative sample

Why is comparing different tests difficult?

Different tests may have different standardization groups

Should tests with very different standardization groups be direclty compared?

Ideally no, although it is often done

What is an overlap sample?

giving 2 different tests to the same group of people

Why use overlap samples?

To compare scores from tests that have different normative samples

Who is associated with overlap samples?

Cicchetti

What 2 areas did Cicchetti examine?

Adaptive functioning and cognitive functioning

What is adaptive functionng?

Daily living skills (dressing, grooming, using utensils, tying shoes)

What is cognitive functioning?

A problem solving, perceptual reasoning, and verbal comprehension skills

Why compare adaptive and cognitive functioning?

to determine whether they are synchronized

Can children have strong cognitive skills and weak adaptive skills?

yes

What is criteron referenced testing?

Comparing scores to a specified content standard rather than to other people

Does criterion referenced testing use norms?

no. cutoff scores are used

what is the focus of criterion reference testing

what the person knows or can do

Example of criterion referenced testing

Mastery testing

What are 2 major components of mastery testing?

1. Determining the proportion correct needed for mastery

2. Determining how many items are needed to establish master

Advantage of cutoff scores?

they require a specific skill level to pass

Disadvantage of cutoff scores?

increased error judgements or misclassification

example of cutoff score error?

a student scoring 70 passes while a student scoring 69 fails despite nearly identical performance

Why should multiple measures be used when determining mastery?

To reduce classification errors

What is a fixed reference group?

a group from a specific point in time that serves as the comparison group for future examinees

What types of tests often use fixed reference groups?

GRE and SAT

before 1995, SAT scores were compared to which group?

individuals who took the SAT in 1941 (11k). primarily white men attending ivy leage institutions - limited educatonal access for other groups

After 1995, SAT scores were compared to which reference group?

infividuals who took the SAT in 1990 (1 mil)

How did SAT prep companies contribute to concerns about bias?

expensive preparation courses were more accessible to wealthier students

Why is SES important when evaluating fixed reference goups?

the reference group may not represent individuals from all socioeconomic backgrounds,

According to the lecture, the SAT is…

1. reliable

2. valid

3. biased

Takeaway from lecture

a measure can be accurate and consistent while still creating unequal opportunities becase of how norms, reference groups, or access to resources are structured

What are the 3 norm concepts most commonly misinterpreted?

1. mental age

2. grade equivalents

3. percentile ranks

What are 2 testing approaches that do NOT use norms?

1. domain referenced testing

2. criterion referenced testing (mastery testing)