Week 4 Module 1– Validity of Measurements

What is reliability?

The extent to which a measurement is consistent and free from error

Reproducibility

Dependability

What is validity?

Assures that a test is measuring what it is intended to measure

It’s necessary for drawing meaningful inferences from data and for determining how the results of a test can be used

What are the goals of reliability and validity?

Limit the amount of error involved in measurement

A test must be _______ before it is considered _________

Reliable, valid

What does validity assess?

What we are able to do with the test results; is the information transferable

What is face validity?

When an instrument appears to test what it is supposed to; ‘face value’

What is the least rigorous method for assessing validity?

Face validity

Face validity should not be considered…

Sufficient for scientific purposes

What is content validity?

The adequacy with which the theoretical domain of the intended construct/variable is measured

When does a construct have content validity?

Of it covers all parts of the construct/variable’s content and reflects the relative importance of each part

What are examples of instruments that it is very important to have content validity for?

Questionnaires

Examinations

Inventories

Interviews

Content validity demands that a test is free from the influence of…

Factors that are irrelevant to the test’s purpose and contains all elements of the variable/construct being studied

What is criterion-related validity?

Based on the ability of one test to predict results obtained on an external criterion

Criterion-related validity is when the _____ test is compared with the _______ standard

Target, gold

It’s important to demonstrate or establish ______ and ________ of the criterion

Reliability, validity

When is it more difficult to define the criterion or gold standard? (When evaluating criterion validity)

If the constructs being evaluated are more abstract

What is concurrent validity?

A form of criterion-related validity studied when the test measure and criterion measure are taken at relatively the same time (concurrently)

When is concurrent validity often used?

To establish the validity for diagnostics, screening tests, measures of presence/absence of disease/condition

What is predictive validity?

Test that aims to establish that a measurement is consistent will be a valid predictor of some future criterion score

What are types of criterion-related validity?

Concurrent validity

Predictive validity

How is information regarding predictive validity gathered?

A target test is given, and after a period of time, the criterion score is collected

Predictive validity is important for….

Screening and risk assessments, prognosis, setting long-term goals, potential for change

What is construct validity?

Reflects the ability of an instrument to measure an abstract concept or construct

Part of construct validity is based on _______ validity

Content

What is the known groups method?

Determines if a test can discriminate between individuals who are known to have a trait and those that do not

What is convergent validity?

Indication that two measures believed to reflect the same phenomenon will yield similar results or will correlate highly

What is discriminant validity?

Indicates that different results, or low correlations, are expected from measures that are believed to assess different characteristics

What are different types of validity or components that fall under construct validity?

Convergent validity

Known Groups Method

Discriminant validity

Factor analysis

Hypothesis testing

Criterion validation

What is factor analysis?

Based on the idea that a construct contains one or more underlying dimensions; separates correlated variables into subsets of test items or behaviors that are related to each other but not to other factors

What is hypothesis testing?

Ability of a test to statistically support hypotheses related to the underlying theoretical construct

What is criterion validation?

Construct validity can be determined using methods in criterion-related validity to determine correlations with criterion measures or sets of criterion measures

What is ‘measuring change’?

The difference between outcome and initial score may be referred to as the change score or difference score

What are four issues that influence the validity of change?

Level of measurement (data type)

Reliability (measurement error over time, MDD)

Stability (true scores changes over time

Baseline scores (floor and ceiling effects)

What is responsiveness?

The ability to detect minimal change over time

Score must change in proportion to the client’s status over time (and can’t change if the client is unchanged)

What does ‘minimally clinically important difference’?

Smallest difference in a measured variable that signifies an important (rather than trivial) difference in the client’s condition

What is a criterion-referenced test?

A test that is interpreted relative to a fixed standard that represents an acceptable level

What is a norm-referenced test?

A test that is interpreted relative to a reference group often evaluated on an average

What is cross validate?

A test with a second group that has similar characteristics as the original validation group; it can also be used with a group that is different but experiences or exhibits the same construct

What is a measurement?

The process of assigning numerals to variables to represent quantities of characteristics according to certain rules