6 - Risk Assessment

risk assessment is a _____ problem

risk assessment is a _____ problem

practical

first generation risk assessment

expert opinion

• psychologist or psychiatrist gives their professional judgement

• involves interview and file review

• nature of information is gathered, and interpretation is subject to personal discretion

pros and cons of first generation RA

pros

• always available to clinician

• incorporates clinician’s experience

• clinician has freedom to consider any data as relevant

• may have short-term validity

cons

• questionable accountability and fairness

• difficult to explicate decision rules

• poor reliability and validity for medium and long term predictions

second generation risk assessment / actuarial

adds objectivity

• ex., VRAG, PCL-R

• identify variables empirically associated with risk and assign values, or weight, to each

• may include dynamic/clinical variables but often do not

• designed to predict a particular outcome

historical variables

static, cannot be undone like previous convictions

dynamic variables

fluctuate such as housing or peer influences

clinical varaibles

subtype of dynamic variables

fluctuate but reflect clinical conditions

easier to use second generation for a ____ group rather than an _____

large, individual

pros and cons of second generation RA

pros

• objective, scientific, lend well to appropriate scrutiny

• affords consistency and accuracy (reliability and validity)

• less dependent on rater’s clinical experience

cons

• can be insensitive to changes if only static variables are included

• optimization can restrict generalization

violence risk appraisal guide

assigns patient into one of nine categories or bins

• each bin is associated with a specified probability of recidivism within given period of time

• creates a Forward Stepwise Regression Analysis

• variables correlate to recidivism and to each other

individuals need to have ____ ____ to do a 2nd gen risk assessment on them

previously offended

what needs to be specified in 2nd gen risk assessment

• risk cohort

• type of recidvism

• time frame / opportunity

• underestimate (any reason to override the first gen risk assessment)

what does VRAG not touch on?

the type of crime, severity of crime, risk in the short term, when in the interval recidivism is most likely, or what might reduce the overall risk

underlying risk model (2nd gen)

tells us the risk over time compared to other people with the same offense

• the individual risk level does not change unless person accumulates more static items

• individual case can be higher or lower than reference population

third-generation risk-needs assessment / SPJ

SPJ (structured professional judgment)

• certain need are linked to criminal behaviour (criminogenic needs)

  • therefore, altering the needs will change the likelihood of criminal behaviour

• emphasis on repeated assessments

• three categories: historical, clinical, risk

• tool that guides treatment

criminogenic needs

dynamic risk factors

underlying risk model (3rd gen)

risk = static risk + dynamic risk. If threshold is exceeded, offending becomes highly likely

• most of the time, stays under the offending threshold

actuarial (2nd gen) approaches are good for ____ ____ prediction and stratifying offenders by risk level

long term

SPJ (3rd gen) approaches are also good for stratification, but better at identifying the _____ under which further offending will take place

circumstances

• needs to be re-assessed after big life events

which RA is best?

uses 2nd and 3rd in combination

NEVER just use 1st

pros and cons of third gen RA

pros

• imposes formal structure on evaluation but flexibility around decision-making

• can be sensitive to changes

• better than unaided clinical judgement

cons

• less evidence on reliability

• less evidence on validity

• no empirical optimization of item selection or weighting

• accuracy unclear when scores adjusted by clinicians

the higher the number of the AUC the ____ the predictions

better

base rate problem

difficult to evaluate instruments on the basis of how often they are accurate in their classification

base rate

proportion of a large reference population that exhibits has/undergoes a target attribute or event

true positive

success

correct identification that they will recidivate

false positive

incorrect identification, did not recidivate

false negative

incorrect prediction, actually did recidivate

true negative

success

correct prediction that they would not recidivate

as base rate decreases, false positives _____

increase

as base rate increases, false negatives _____

increase

sensitivity

ability to pick something up without missing anything

= # of predicted recidivists / total # of recidivists

= probability of correctly classifying a future recidivist

specificity

ability to reject stuff that we are not trying to pick up

= # of predicted non-recidivists / total # of non-recidivists

= probability of correctly classifying a future non-recidivist

we would like to maximize both sensitivity and subjectivity, but in practice…..

one comes at the cost of the other (ex., as sensitivity increases, specificity decreases)

a good risk assessment tool will…

1. produce a larger mean score difference for the two groups

2. produce reasonably small variance as to minimize variance

the AUC is ______ to the base rate problem

immune

AUC / ROC score meanings

0.5 = no better than chance

0.6-0.65 = some accuracy (barely statistically significant)

0.65-0.69 = better accuracy

0.70+ = much better accuracy

which generation RA has the best predictive accuracy?

second gen / actuarial

which generation of assessment is most used in practice?

first generation, often due to unforeseen circumstances like a general clinician performing the RA

do aging offenders pose less of a risk overtime?

in most cases, if no further offending occurs, risk goes down 10% each year

accuracy of VRAG is maintained when controlled for age and time risk

the two foci that 3rd gen instruments have added

1. relevance: risk factors may be more or less influential based on the individual

2. case formulation: movement away from using RA instruments purely as means of estimating risk, and more to detail a range of possible outcomes

pitfalls to avoid with RA

1. failing to stick to the publisher’s scoring criteria

2. inappropriate instrument selection

3. vague reporting (failing to describe instrument, not specifying reference groups)

risk

one’s tendency to offend

threat

likelihood of being able to follow through with the offense

offender risk x target availability x vulnerabilities

what does the AUC value reflect?

proportion of non-recidivists that would have a lower test score than a randomly chosen recidivist