tests/measurements exam 2

construct test overlap- construct underrepresentation

construct not covered by the test

construct test overlap- construct irrelevance

measuring unwanted characteristics

content validity- what is it

the degree to which the questions, tasks, or items on a test are representative of the content it was designed to test

content validity- how it's determined

-by comparing test items to all possible items if available

-with expert judges

content validity- typical uses

-educational achievement tests

-job related tests

criterion-related validity- criterion

a standard or benchmark used to evaluate the validity of a test

criterion-related validity- external criterion

-is the test effective for what it's measuring?

-whether the test can predict or be associated with an outcome that is already measured

Sensitivity vs. Specificity (HIT RATE)

Sensitivity: identifies individuals who meet the criterion (hit for meeting criterion)

Specificity: identifies individuals who do not meet the criterion (hit for not meeting criterion)

criterion-related validity- group contrasts

-does the test yield good separation between two groups

-test this by testing one group that meets criteria and one that does not and see the separation

criterion-related validity- another test

compare test to another test that measures the same thing

concurrent validity

the degree to which test scores correlate with scores from other measures taken at the same time

predicitive validity

the degree to which a test predicts future performance

convergent validity

compare a test to other tests that measure the same thing - want a positive correlation

discriminant validity

give two tests that are unrelated to see how they correlate - correlation should be low

multiple regression for prediction

statistical technique that uses multiple tests to make a prediction

construct validity- construct

a hypothetical, abstract concept or idea that is not directly observable but is inferred from observable data or behaviors

construct validity- factor analysis

statistical techniques that help to identify the common dimensions underlying performance on many different measures

-Generate a long list of items

-Administer these items to a large group of people

-Using statistical technique of factor analysis, identify which items group together

-Name the factor by interpreting what the items that group together have in common

statistical vs clinical predicition

- statistical method: using a formal algorithm, like a statistical equation, to help make decisions. Sometimes judgment is not enough, and equations process info more effectively

- Clinical prediction would be a person, like a graduate admission employee, who relies on judgment and intuition for decision making. Computers replacing clinicians, dehumanizing, not all have the education for these tests

decision theory

-Hits are correctly predicting whether they will not meet the criterion or if they will meet the criteria

-Misses are incorrectly predicting (false positive or false negative). False negative: they say you won't meet the criterion, and they do. False positive: they say you will meet the criteria, and you do not (think false covid or pregnancy test)

validity coefficient- acceptable level

.3-.4, rarely above .6

test development

1) defining the test's purpose

2) preliminary design issues

3) item preparation

4) item analysis

5) standardization and ancillary research programs

6) preparation of final materials and publication

selected vs constructed response

- selected response (multiple choice/Likert scale - selecting from the options)

- constructed response (short answer/ constructing their own response)

training levels

- Level A: minimal training

- Level B: knowledge of test construction and training in statistics and psychology

- Level C: substantial understanding of testing and supporting topics

Spearman's theory: g factor and specific factors

- Overall intelligence is best represented by a single underlying factor (g)

- Specific factors - we have overall intelligence that plays into aspects like creativity, organization, communication

Thurstone's theory: primary mental abilities

- Primary mental abilities - saw them as distinct separate factors that are not related

- Some of these abilities:

Word fluency

Spatial reasoning

Memory

Perceptual speed

Etc.

Hierarchical model

- An organization of different mental abilities with levels - combines Spearman's and Thurstone's theory

- Overall mental abilities that plays into primary abilities that plays into assessing these abilities

Cattell-Horn-Carroll Theory

the theory that our intelligence is based on g as well as specific abilities

crystallized intelligence

one's accumulated knowledge and verbal skills; tends to increase with age

fluid intelligence

our ability to reason and think abstractly; tends to decrease during late adulthood

Wechsler tests: 4 indices (VCI, WMI, POI, PSI)

- Verbal Comprehension (VC1): vocabulary, similarities, information

- Working memory (WMI): digit span, arithmetic, letter-number sequencing

- Perceptual Reasoning (POI): block design, matrix reasoning, picture completion

- Processing speed (PSI): coding, symbol search

Generational Changes - Flynn Affect

- What is the Flynn effect? IQ scores have been increasing over time, 3 IQ point change over each decade

- Why do we see these generational changes in IQ scores? They are not updating tests to what is currently being taught, younger generations have more test preparation or practice, better nutrition or prenatal care helps more with brain development

-What are the implications of the Flynn effect? Standards have to change

age changes

cross sectional- data taken at one point in time from multiple groups

longitudinal- follow same group of people over life

cross sequential- combines elements of both longitudinal and cross-sectional research designs, examining multiple groups of individuals of different ages over a period of time to study development and change