Psychometrics Final Review

Construct validity

evidence that the test is related to other tests and behaviors as predicted by a theory

Content validity

degree to which a content of a test is representative of the domain it is supposed to cover

Criterion related validity

the extent to which the scores on a test correlate with scores on a measure of performance or behavior

Face validity

extent to which respondents can tell what the items are measuring

Stems

a statement, question or partial sentence that is the stimulus in multiple choice questions

Distractors

incorrect responses in multiple choice questions

Pilot test

a scientific investigation of a new test's reliability and validity for its specified purpose

Survey

research tools to describe people's knowledge, behavior, or attitudes

Concurrent validity

a form of criterion related validity in which test administration and criterion measurement happen at approximately the same time

Predictive validity

a form of criterion related validity that shows a relationship between test scores and a future behavior.

Criterion contamination

when the criterion in a validation study measures more dimensions than those measured by the test

Linear regression

the statistical process used to predict one set of test scores from one set of criterion scores

Multiple regression

a statistical technique that includes two or more predictor variables in a prediction equation

Intercept

the place where the regression line crosses the Y-axis

Slope

the expected change in one unit of Y for every change in X

Construct explication

defining or explaining a psychological construct

Hypothesis

educated guess or prediction based on a theory

Discriminant validity

evidence that the test scores are not correlated with unrelated construct

Convergent validity

evidence that the scores on a test correlate strongly with the scores on other tests that measure the same construct

Factor analysis

an advanced statistical procedure based on the concept of correlation that helps investigators identify the underlying constructs or aspects being measured.

Multitrait-multimethod matrix

a design for test validation that gathers evidence of reliability, convergent validity, and discriminant validity in one study.

Goodness-of-fit test

a statistical test of significance that provides evidence that the factors obtained empirically in a confirmatory factor analysis are similar to those proposed theoretically

Steps in developing a psychological test

define, plan, items, instructions, manual

Test plan

a design for developing a new test that specifies that characteristics of the test

Objective test format

a test format that has one response that is designated as "correct" or that provides evidence of a specific construct

Subjective test format

test format that doesn't have a designated answer as correct.

Example of objective

multiple choice

Example of subjective

essay

Cumulative model of scoring

the more the test taker responds in a particular fashion, the more the test taker exhibit the attribute being measured

Categorical model of scoring

places test takers in a particular group or class

Ipsative model of scoring

compares the test taker's scores on various scales within the inventory to yield a profile

Multiple choice questions

closed-ended questions that ask the respondent to choose among several answers

Projective techniques

unstructured and require test takers to respond to ambiguous stimuli

Forced-choice questions

requires test takers to choose one of two or more words or phrases that appear unrelated but equally acceptable

Acquiescence

agreeing with everything/saying yes to every question

Social desirability

test taker provides answers that are socially acceptable

Faking

answer items in a way that will cause a desired diagnosis

Phi coefficient

statistic showing relationship between two variables

Item characteristic curve (ICC)

line that results when we graph the probability of answering an item correctly with the level of ability on the construct being measured

Item difficulty

percentage of test takers who answer correctly

Item discrimination

extent to which items differentiate high from low scorers

Item response theory

related performance on each question with ability of what you are measuring

Discrimination index

indicates the extent to which students who get a particular test item correct are also likely to get a high score on the entire test

Cut score

decision points for dividing test scores into pass/fail groups

Inter-item correlations

displays the correlation of each item with every other item

Authentic assessment

measures the students ability to apply in real world settings

Formative assessment

helps teachers determine what information students are or are not learning

Summative assessment

determines whether students have learned what the teacher intended them to learn

Placement assessment

used to determine whether students have the skills or knowledge necessary to understand new material

Diagnostic assessment

in depth evaluation of an individual to identify characteristics for treatment or enforcement

Norm-referenced test

standardized tests given to a large representative group of test takers

Criterion-referenced test

involve comparing an individual's test scores to an objectively stated standard of achievement

Information-gathering model of assessment

goal is to collect data that will aid in communication and decision making

Therapeutic model of assessment

getting information to make changes in life; clients are actively encouraged to participate in discussions

Different treatment model

used to conduct research and evaluate programs

Structured clinical interview

interviewer asks the same question of all job candidates

Non-directive clinical interview

leading questions, those that indicate a particular answer as 'preferred' by the interviewer

Single construct test

measures one specific idea

MMPI (Minnesota Multiphasic Personality Inventory)

widely used structure personality test designed to assess symptoms of mental disorders

NEO personality inventory

a test that measured the Big Five traits: extraversion, openness to experience, agreeableness, conscientiousness, and neuroticism

Thematic apperception test (TAT)

a projective test in which people express their inner feelings and interests through the stories they make up about ambiguous scenes

WAIS

most widely used intelligence test; contains verbal and performance subtests

Neuropsychology

branch of psychology that concentrates on the relation between how the brain functions and the behavior it produces