Lecture 8: Overview of Assessment and Prediction

What is Diagnosis/Case Formulation?

Specification of DSM clinical diagnosis and conceptualization of clinically significant difficulties

What is Prognosis/Prediction?

Estimation of the likelihood of clinically significant difficulties

What is Treatment Monitoring and Evaluation?

Examination of how well prevention or intervention is working

What is Screening?

Identification of those experiencing clinically significant difficulties

What is Diagnosis/Case Formulation?

Specification of DSM clinical diagnosis and conceptualization of clinically significant difficulties

What is Prognosis/Prediction?

Estimation of the likelihood of clinically significant difficulties

What is Treatment Monitoring and Evaluation?

Examination of how well prevention or intervention is working

What are three types of reliability that are evaluated for assessment and prediction approaches?

Internal consistency, Test-retest, and Inter-rater reliability

What is the AUDIT?

The best-available brief screener for alcohol abuse, alcohol dependence, and high-risk drinking (Alcohol Use Disorders Identification Test)

What is Content Validity?

Extent to which item content reflects concept of interest

What is Predictive Validity?

Association of measure with another presumably related measure at FUTURE point in time

What is Incremental Validity?

Extent to which measure predicts more than what already could predict

What does the Sensitivity of an assessment/prediction approach tell us?

Proportion of those with a disorder or condition who are correctly identified

What does the Specificity of an assessment/prediction approach tell us?

Proportion of those without a disorder/condition who are correctly identified

What does it mean if a measure of Bulimia shows a Sensitivity value of .60?

60% of those with Bulimia are correctly identified as Bulimic by the measure

What does it mean if a measure of Bulimia shows a Specificity value of .80?

80% of those without Bulimia are correctly identified as not having Bulimia by the measure

If a measure of Bulimia shows a Sensitivity value of .60, what percentage of people with Bulimia are NOT identified as Bulimic by the measure?

40%

If a measure of Bulimia shows a Specificity value of .80, what percentage of people without Bulimia are WRONGLY identified as Bulimic by the measure?

20%

What does a Cutoff Value determine for an assessment/prediction approach?

Whether the approach makes a positive or negative assessment or prediction

Suppose that you administer a measure of Social Anxiety to 1000 college students, where higher scores are associated with a greater likelihood that the person would be diagnosed with Social Anxiety Disorder in a comprehensive clinical interview. If you increase the clinical cutoff fromm 20 to 24, will MORE or FEWER people receive a Social Anxiety Disorder diagnosis?

FEWER

Suppose that you administer a measure of Social Anxiety to 1000 college students, where higher scores are associated with a greater likelihood that the person would be diagnosed with Social Anxiety Disorder in a comprehensive clinical interview. If you increase the clinical cutoff from 20 to 24, will the measure's sensitivity INCREASE or DECREASE?

DECREASE

Suppose that you administer a measure of Social Anxiety to 1000 college students, where higher scores are associated with a greater likelihood that the person would be diagnosed with Social Anxiety Disorder in a comprehensive clinical interview. If you increase the clinical cutoff fromm 20 to 24, will the measure's SPECIFICITY INCREASE or DECREASE?

INCREASE

What is a True Positive?

Person with disorder/condition classified by screen/questionnaire as having disorder/condition

What is a False Positive?

Person without disorder/condition classified by screen/questionnaire as having disorder/condition

What is a True Negative?

Person without disorder/condition classified by screen/questionnaire as not having disorder/condition

What is a False Negative?

Person with disorder/condition classified by screen/questionnaire as not having disorder/condition

Suppose that you administer a measure of Social Anxiety to 1000 college students, where higher scores are associated with a greater likelihood that the person would be diagnosed with Social Anxiety Disorder in a comprehensive clinical interview. What do we call those who are correctly identified by the measure as meeting diagnostic criteria for Social Anxiety Disorder?

True Positives

Suppose that you administer a measure of Social Anxiety to 1000 college students, where higher scores are associated with a greater likelihood that the person would be diagnosed with Social Anxiety Disorder in a comprehensive clinical interview. What do we call those who are correctly identified by the measure as not meeting diagnostic criteria for Social Anxiety Disorder?

True Negatives

Suppose that you administer a measure of Social Anxiety to 1000 college students, where higher scores are associated with a greater likelihood that the person would be diagnosed with Social Anxiety Disorder in a comprehensive clinical interview. What do we call those who are wrongly identified by the measure as meeting diagnostic criteria for Social Anxiety Disorder?

False Positives

Suppose that you administer a measure of Social Anxiety to 1000 college students, where higher scores are associated with a greater likelihood that the person would be diagnosed with Social Anxiety Disorder in a comprehensive clinical interview. What do we call those who are wrongly identified by the measure as not meeting diagnostic criteria for Social Anxiety Disorder?

False Negatives

Does Specificity increase or decrease as Sensitivity increases?

Decrease

What does a Prediction Equation provide us?

An overall risk score from a weighted combination of risk factors

What does it mean for an assessment/prediction approach to be completely clinical?

Clinician selects, measures, and combines risk factors and produces risk estimate solely according to clinical experience and judgment

What does it mean for an assessment/prediction approach to be completely actuarial or statistical?

Risk assessment completely determined by prediction equation and established cutoff scores

What does it mean that the Violence Risk Appraisal Guide (VRAG) shows a sensitivity value of .65?

The proportion of those correctly predicted by the VRAG to show violent recidivism

What does it mean that the Violence Risk Appraisal Guide (VRAG) shows a specificity value of .65?

The proportion of those correctly predicted by the VRAG not to show violent recidivism?

Are clinical or actuarial methods more accurate at making clinically relevant predictions?

Actuarial methods

What are Base Rates?

The proportion of people who have a particular condition or meet a particular set of criteria

When Base Rates are low, do our assessments/predictions tend to be MORE or LESS accurate?

LESS