2.2 - Validity

Validity

Agreement between a test score or measure and the quality it is believed to measure.

Validity

Asks the question of whether the test scores measure what they purport to measure

• Face

• Criterion-Related

• Content

• Construct

• Treatment

(5) ASPECTS of Validity:

F-C-C-C-T

Face Validity

mere appearance that a measure has validity

Face Validity

determines if the assessment “looks like” it is measuring what it is supposed to measure

Face Validity

not really a validity at all

• because it does not offer evidence to support conclusions drawn from test scores

Face Validity

A driving test asks questions about traffic rules. It looks valid for testing driving knowledge, but we can’t be sure it truly measures driving ability until we see actual evidence.

Criterion-Related Validity

pertains to validity evidence that is obtained by comparing test scores with performance on a criterion measure

Criterion-Related Validity

May include scores on other assessment or may be some external criteria such as grades, diagnosis, or job satisfaction

• Concurrent

• Predictive

2 Types of Criterion-Related Validity:

Concurrent Validity

refers when test scores and the criterion performance scores are collected at the same time

Concurrent Validity

A new depression questionnaire is given to patients today, and their current clinical diagnosis is checked at the same time.

Predictive Validity

client’s performance or criterion measure is obtained some time after the test score

Predictive Validity

test is going to be used in the future to predict some type of behavior

Predictive Validity

A college entrance test is given this year. Students’ first-year GPA is checked next year.

Concurrent Validity

Test and criterion measured together.

Predictive Validity

Test forecasts future behavior or outcome.

Content Validity

refers to the representativeness of items from a “population” of items

Content Validity

involves defining the domain of interest

• identifying items that refers to that domain, involving judges to ensure that those items correspond to the domain and that items are sampled adequately

Content Validity

A history test aims to cover the entire syllabus of World War II. Experts check that the test items include key events, dates, and people.

• If all areas are covered, the test has good ________

Construct Validity

evidence that pertains to the theoretical basis of a test

Construct Validity

has been used as a means of explaining the psychological meaning

of the variable measured by the test

• Convergent

• Discriminant

Construct Validity (2)

Convergent Validity

test scores should be expected to show a substantial correlation with other tests and assessments that measure similar characteristics

Convergent Validity

A new anxiety scale should correlate highly with existing anxiety tests.

Discriminant Validity

test scores should not be substantially correlated with other tests from which they are supposed to differ

Discriminant Validity

An anxiety scale should not correlate much with a physical fitness test.

Treatment Validity

Do the results obtained from the test make a difference in the treatment?

Treatment Validity

A psychologist administers a stress questionnaire. Based on the results, the client is given relaxation exercises.

• If the exercises improve the client’s stress levels, the test shows _________.

Validity Scales

are tools used to determine three types of response distortions

• Faking Bad

• Faking Good

• Random responding

Validity Scales (3)

Faking bad

A client pretending to have some problem or disorder

Faking good

A client responding in a socially desirable manner to appear more favorable

Random responding

answering inconsistently or carelessly, intentionally or unintentionally.

Faking bad

A client taking a personality test reports extreme symptoms of depression that are unlikely given their history. The validity scale flags ______

Faking good

Another client exaggerates positive traits to impress the psychologist

Random responding

A client randomly selects answers on a long questionnaire

Criterion-Related Validity

Quiz scores compared to their examination scores

Face Validity

The test looks like it measured what it is supposed to measure

Treatment Validity

Results of the test helped the counselling process

Construct Validity

Neo-Personality Inventory (Big 5 factors)

Construct Validity (Convergent)

Ramon’s MBTI result correlates to his 16 PF test result