Psychometrics Exam 3 content

Validity

Evidence that the interpretations derived from the scores on the test are appropriate

Traditional Model of Validity

Measures what its designed to measure

Contemporary Model of Validity

Are we making appropriate interpretations with scores?

Sources of Validity Evidence

Evidence based on test content, response process, internal structure, relationships with other variables, consequences of testing

Content Validity

Are items in your test representative of the construct intended?

Construct

Attribute, trait, or other characteristic thats abstracted from observable behavior

Criterion-related Validity

The extent to which the scores on a test correlate with scores on a measure of behavior

Construct Validity

Evidence that a test relates to other tests and behaviors as predicted by theory

Convergent Validity

The test correlates with measures of the same or similar constructs

Discriminant Validity

The test is not correlated with measure of dissimilar constructs

Front End (during test dev.)

Clearly operationalized construct; Items are carefully crafted to ensure the measure is representative of the concept/construct

Back End (after test dev.)

Evaluation of content after the test is developed; Allows us to quantify

Content Validity Ratios (-1.00 - 1.00)

Experts review and rate how essential test items are to the target construct; Measure of agreement among the judges; Items that dont meet minimum cut off are removed

Face Validity

Perceptions of the test takers that the test measures what it intended to measure

Traditional View of Validity

Content and criterion-related

Abstract Construct

Underlying attitudes that exist in thought only, but underlie behaviors

Construct Explication

1. Identify behaviors related to construct

2. Identify other related constructs

3. Identify behaviors related to other constructs

Concrete Constructs

Something we can observe, behaviors

Nomological Network

A method of defining a construct by identifying its relationships with as many other relevant constructs as possible

States

A temporary condition often brought on by situational circumstances

Traits

A long standing individual quality that has become an enduring part of a person’s personality

Multitrait (Multimethod Matrices)

Researcher chooses 2 or more constructs that are unrelated in theory and 2 or more types of tests- to measure each of the constructs

Heterotrait-hetero

Evidence of discriminant validity; Different things measured different ways shouldnt be correlated

Heterotrait-mono

Evidence of common method variance; Different things measured same way shouldnt be correlated (but typically are to some degree)

Monotrait-hetero

Evidence of convergent validity; Same thing measured different ways should be correlated

Monotrait-mono

Evidence of reliability; Same thing measured same way correlated with itself should highly correlate

Reliability Induction

Practice of explicitly referencing the reliability coefficients from prior reports as the sole warrant for presuming the score integrity of entirely new data

Factors

Underlying concepts of constructs that the tests or groups of test questions are measuring

Exploratory Factor Analysis

No formal hypothesis about the factors; How many underlying factors are there?

Confirmatory Factor Analysis

Factor structure specified in advance based on theory; How well does my data replicate the theoretical data?

Residual

Distance between the trendline and each observed data point (=observed score - predicted)

Linear Regression

Standardized statistical technique used for the finding best fitting straight line for a set of data

Ordinary Least Squares

Goal is to reduce residuals between data points and regression line

Regression Coefficients

Describe the direction and magnitude of the relationship between X and Y

Ttest

Evaluate whether the X significantly predicts Y

Coeffiecent of Determination

Describes proportion of variance explained in the Y by X

ANOVA Table with F value

Tests whether the proportion of variance explained is statistically significant

Temporality

In predictive validity, some meaningful period must pass; In concurrent test administration and criterion measurement occur at similar time

Multiple Regression

The correlation between the manifold (or combination) of predictors and the criterion

Range Restriction

The reduction in the range of scores that results when some people are dropped from a validity study

Versimilitude

Similarity between test items and tasks and similarity between real world everyday tests

Veridicality

How well do test scores predict real world functioning?

Sensitivity

Correctly classify people with the disorder (identify true pos)

Specificity

Correctly screen out people without the disorder (identify true neg)

Classification Accuracy

How well does our regression equation work?

Cross Validation

Process of administering a test to another sample of test takers, representative of a target population

Measurement Bias

Scores on a test taken by different subgroups in the population need to be interpreted differently because of some characteristic of the test not related to the construct being measured

Differential Validity

Different validity coefficients for subgroups

Single-group Validity

Test has evidence of validity for only one group