PSY333: Measurement & Testing (some level of finals)

Cattell

Cattell

"First used the term ""mental tests"""

Binet

Associated with the first modern-day intelligence test (measure higher mental processes)

Wundt

First psychological laboratory that used experimental research

Terman

First use of the term intelligence quotient (IQ); revised Binet

Thorndike

Associated with the Stanford Achievement Test

WWI

What was the era that first widely used group testing?

Army Alpha

Group administation of intelligence test for the military; reading literacy

Army Beta

Used as an intelligence test, but is the language-free version

Thorndike

Research on vocational assessments

Miner

Person involved in occupation selection for large groups of high school students

Strong

First -- much more general career counseling for the future aptitude tests

Woodworth's Personal Data Sheet

First modern personality inventory (WWI); measured suspectibility to mental health problems

aptitude

measure whether or not you're ready for something

Step 1: Determine the goals of your client

"Defines purpose of test; demographics are considered; what context the test is on

Step 2: Choose instrument types to reach client goals

Asks the questions: What behaviors, content, skills is it intended to measure? What is the <u\>theory </u\>that the trait is based on? What about subsets/domains it is based on? Operationalization of test forms.

Step 3: Access information about possible instruments

Item formats are determined; test is written and item reviewers make sure it measures what is intended to measure

Step 4: Examine validity, reliability, cross-cultural, fairness, and practicality of the possible instruments

Before this is done, a pilot test is done to make sure the items are valid, reliable, and fair, among other items. <i\>Then</i\>, this happens.

pilot test

Validation process

Step 5: Choose an Instrument Wisely

Determines test length, testing time, scoring approaches, and test procedures, administers test materials.

Level A

Tests which can be administered, scored, and interpreted by laypeople

Level B

Tests that require a psychology degree or coursework in testing

Level C

Tests that require an advanced psychology degree, a license and/or advanced training for that particular test

cognitive sources of construct-irrelevant variance

Knowledge or skill not related to the purpose of the test is required to answer an item correctly.

affective sources of construct-irrelevant variance

Language or images causes strong emotions that may interfere with the ability to respond to an item correctly (i.e. political opinions, beliefs)

physical sources of construct-irrelevant variance

aspects of tests interfere with the test takers' ability to attend to, see, hear, or sense the items or stimuli (consider disabled people!)

correlation coefficient

statistical relationship between two variables

scatter plot

"Used to visually examine data, especially to discover patterns (such as curvilinear relationships)<br\>"

positive relationship

"an increase in one variable is related to an increase in the other variable<br\>"

negative relationship

"an increase in one variable is related to a decrease in the other variable<br\>"

no relationship

"two variables that are not related to each other<br\>"

strong correlation

"<div\><font color\=""\#000000""\>±0.70 - 1.00</font\></div\>"

moderate strength

<font color\=""\#000000""\>±0.30 ~ 0.69</font\></div\>"

no strength

<font color\=""\#000000""\>±.00 ~ 0.29</font\></div\>"

scores from a test is a consistent measure of individuals’ true scores

reliability

correlation coefficient

To measure reliability, we use

method error

caused by test administrators or the testing environment

trait error

Error associated with test takers, subjects themselves

test-retest reliability

Relationship between scores on the same test administered twice with a time interval between the administration

practice effects

e.g., subjects may get better at second testing, subjects knowing how they answered in a similar test form

alternate-forms reliability

Coefficients of two equivalent tests are compared (time interval)

internal consistency

obtaining a reliability coefficient by assessing how items are correlated as a group

split-half reliability

internal consistency; correlation between scores from even-numbered items and scores from odd-numbered items

validity

whether a test measures what it is supposed to measure (<i\>accuracy)</i\>

content validity

Does the ______ ______ cover a representative sample of behaviors to be measured in its entirety? Content experts

criterion validity

Does a test predict the target trait it is intended to measure?

concurrent validity

Focuses on the prediction of current performance or psychological behavior

predictive validity

Focuses on the prediction of future performance or psychological behavior

construct validity

Does an assessment measure a theoretical construct that it is designed to measure (e.g., intelligence)?

convergent validity

Are two assessments measuring the <u\>same (or similar)</u\> construct related?

discriminant validity

Are two asssessments measuring different constructs <u\>not related</u\>?

factor analysis

Found construct you want to measure from the test scores

fairness

whether an individual's score is not affected by potential bias inherent in a test, test procedure and interpretation

the 1960s (civil rights movement)

Fairness did not get much attention until

fairness in testing process

Equal testing condition + proctors

fairness as lack of measurement bias

Idea that all items should behave equally across all examinees

fairness in access to the construct as measured

accessibility in testing; showing their status on target without being advantaged or disadvantaged by their individual characteristics or opportunity to learn

differential item functioning

Statistical approach to examine test fairness by identifying items that perform differentially across subgroups of test takers while controlling for test takers' ability

cognitive interview

examining response processes through probing questions

achievement testing

tests that measure what one has learned<br\>e.g., high school exit exams

aptitude testing

measure what one is capable of learning<br\>e.g., intelligence tests

personality assessment

used to assess habits, temperament, likes and dislikes, character, and similar behaviors

diagnostic tests

tests that assess problem areas of learning; often used to assess learning disabilities

cognitive ability tests

tests that measure a broad range of cognitive ability<br\>e.g., SATs

intellectual and cognitive functioning

tests that measure a broad range of cognitive functioning in general intelligence, intellectual disabilities, giftedness, changes in overall cognitive functioning

special aptitude tests

tests that measure one aspect of ability; likelihood of success in a vocation

multiple aptitude tests

tests that measure many aspects of ability; likelihood of success in multiple vocations

interest inventories

tests that measure likes and dislikes as well as one's personality orientation toward the world of work; career counseling

classification methods

a tool whereby an individual identifies whether he or she has, or does not have, specific attributes or characteristics

readiness tests

tests that measure one's readiness for moving ahead in school. used to assess readiness to enter first grade

mental age/chronological age x 100

How do you calculate IQ (use / as a division sign)?

Spearman-Brown formula

What formula is used for split-half reliability due to the test being cut in half?

"bar graph<br\>"

"visual for a categorical, discrete variable</span\>"

"histogram<br\>"

visual for continuous variables

"frequency polygon<br\>"

used to see the distributional shape of data

positively skewed

"(Type of curve)<br\>"

negatively skewed

"(Type of curve)<br\>"

Mode < Median < Mean

"Left to right, how are measures of central tendency distributed in positively skewed distributions?<br\>"

Mode > Median > Mean<br\>

"Left to right, how are measures of central tendency distributed in negatively skewed distributions?<br\>"

variance

avg of squared distance from the mean

deviation score

the difference between an individual score and the mean

norm referenced

scores that are compared to a set of test scores called the norm group

criterion-referenced scores

scores are compared to a predetermined standard; i.e. mastering a certain level of knowledge, used for diagnoses

percentile

proportion of people falling at and below a score in a standard normal distribution

T-scores

µ = 50, σ = 10; used for personality tests

deviation IQ

µ = 100, σ = 15; used for tests of intelligence

Stanines

µ = 5, σ = 2, round to nearest whole number; used for achievement testing

Sten scores

µ = 5.5, σ = 2, round to nearest whole number; used for personality inventories and questionnaires

NCE scores

µ = 50, σ = 21.06; used for educational tests

SAT scores

µ = 500, σ = 100

ACT scores

µ = 21, σ = 5

Publisher type scores

µ and σ are artbitrarily set by publisher

SEM

σ of test scores x √1 - reliability of a test

standard error of measurement

Tells us how much error there is in the test and ultimately how much any individual's score might fluctuate due to this error

comprehension

problems with the _____ of questions

information retrieval

failure in the information retrieving to answer (related to background characteristics)

decision process

low motivation/intention of faking or impression enhancement

response process

mismatch in the choice of response option; difference in interpretation of option meanings

interquartile range formula

Deviation score

X (raw score) - M (mean score)

Variance

Deviation score squared