PS365 -- Limitations of Neuropsychological Assessment

History of Neuropsychological Assessments

• A.R. Luria developed a theory of brain-behaviour relationships and instruments for studying higher cortical functions from his study of WWII soldiers

  • Core father of neuropsychological assessments

Limitations of Frequently Used Tests

• Although assessments of memory is one of the most frequent concerns, many tests do not have alternative versions equated for difficulty with the original that can be used for re-test purposes

• Certain tests can be failed for various reasons:

  • Cultural

  • Random responding

  • Lack of initiation or persistence of an effective strategy

  • Too long (e.g., WCST)

Lack of Range of Test Meaures

• Emotional influences are minimized as much as possible to reduce interference with cognitive control

• A variety of rating scales focus on regulation of behaviour and emotions, but few tests are designed to address these questions

• Timed tests may evoke feelings of anxiety, affecting performance

• Very few test examine social cognition

Language and Cultural Considerations

• Administer and score tests for diverse languages and cultures

  • MOCA now available in 16 languages

• Familiarity of items will vary between cultures

Tests’ Reliability and Validity

• It is extremely crucial that individual test results are compared to data obtained from a normative or diagnostic group using the same procedures

• Four important test characteristics:

  • Reliability

  • Validity

  • Sensitivity

  • Specificity

Reliability

• The consistency or repeatability of measurements or results

• A research instrument, such as a survey, test, or experiment, is considered reliable if it produces consistent results over time

  • Test-Retest

  • Inter-Rater

  • Internal Consistency

  • Parallel-Forms (different versions)

Construct Validity

• The extent to which a research instrument, such as a test or survey, accurately measures the theoretical concept or “construct” it is intended to measure

Convergent Validity

• The degree to which the measure correlates with other measures that it is theoretically expected to relate to

Divergent Validity

• Degree to which the measure does not correlate with variables that are theoretically unrelated

Factorial Validity

• Whether the structure of the instrument matches the theoretical structure of the construct

Content Validity

• Whether the measurement tool fully represents the content or domain of the construct

Predictive Validity

• How well the construct can predict future outcomes that are theoretically linked to the construct

Sensitivity

• Also known as the “True Positive Rate“

• The ability of a test to correctly identify individuals who what the condition or characteristics

  • Measures how well the test detects the presence of a condition when it is truly there

Specificity

• Also known as “True Negative Rate”

• Ability of a test to correctly identify individuals who do not have the condition

  • Measures how well the test detects the absence of the condition when it is truly not there

False Results

• Type 1 Error -- False Positive

• Type 2 Error -- False Negative

Ecological Validity

• In an attempt to control the examination, procedures often deviate from real-world task demands

• Tasks tend to be administered one at a time and in isolation from other activities

• Free of distractions

Neuroscientific Basis

• Creates stimuli sets that truly represent the real world

Psycho-Legal Issues

• Common reason for evaluation include the:

  • Assessment of disabilities

  • Determination of capacity to make personal decisions

  • Civil and criminal competencies

• Evaluation response bias (malingering) is central to many evaluations

  • Easier to easier to establish that a disability exists than it does not exist

  • Malingering -- Intentional fabrication or exaggeration of symptoms

• Limited understanding of what constitutes as an “invalid response”