Psychometrics in Assessments

Psychometrics in Assessments

Overview of Psychological Measurements

  • Understanding Psychometric Properties

    • Essence: Essential for interpreting and utilizing scores from standardized assessments.

    • Function: Help determine the level of evidence of the tools/instruments, promoting evidence-based practices.

    • Importance: Ability to read and understand psychometric properties in assessment manuals is crucial for clinical application.

    • Variety of Tools: The course introduces multiple assessment tools; awareness of a broader range is necessary for practice.

Considerations for Selecting an Assessment Tool

  1. Purpose

    • Definition: Measures the specific construct intended for evaluation.

  2. Theoretical Approach

    • Definition: Must align with the practical setting in which it will be applied.

  3. Clinical/Research Utility

    • Question: How can result findings be applied in practice?

  4. Population

    • Question: Is the assessment designed for a population that matches the intended subjects?

Psychometric Properties

  • Reliability

    • Definition: Precision and accuracy of measurement procedures. Consistency ensures repeatable results under identical conditions.

    • Types of Reliability:

    • Test-retest reliability: Stability of scores with repeated testing using the same conditions.

    • Parallel forms or equivalence: Comparing scores from different versions of the same instrument.

    • Intrarater reliability: Consistency of scores from a single rater across multiple measurements.

    • Interrater reliability: Stability among scores from different raters.

    • Internal consistency: Degree of agreement between items measuring a single construct.

  • Validity

    • Definition: Assessment measures what it claims to measure. Questions to ask include:

    • Does it accurately measure the intended concept?

    • Types of Validity:

    • Face validity: Appearance of test items in relation to test purpose.

    • Content validity: Extent to which test items represent the construct.

    • Criterion-related validity: Relationship with an external criterion, including:

      • Concurrent validity: Correlation with existing measures, especially gold standards.

      • Predictive validity: Ability to forecast future abilities or outcomes.

      • Construct validity: Relationship between underlying theoretical constructs and scores obtained. This includes:

      • Convergent validity: Scores correlate with similar constructs.

      • Divergent validity: Scores differ from unrelated constructs.

      • Known-groups validity: Distinction between groups (e.g., presence vs. absence of a condition).

Types of Evidence Used in Assessment

  • Responsiveness

    • Change in scores beyond measurement error.

    • Captures clinically significant change in individuals.

    • Floor and Ceiling Effects:

    • Floor effect: Inability to detect low-level changes.

    • Ceiling effect: Inability to detect high-level changes.

    • Sensitivity and Specificity:

    • Sensitivity: Accuracy in detecting conditions.

    • Specificity: Accuracy in rejecting non-conditions.

Levels of Measurement

  • Constructs: Ability assessed via different scales.

  • Types of Measurement Levels:

    1. Nominal: Non-ordered categories (e.g., Yes/No).

    2. Ordinal: Ordered categories without consistent intervals (e.g., Likert-type scales).

    3. Interval: Equal intervals but no true zero (e.g., Degrees Fahrenheit).

    4. Ratio: Equal intervals with a true zero (e.g., Weight, Muscle Strength).

Examples of Constructs Across Measurement Levels

  1. Balance:

    • Nominal: "Stable" vs. "Unstable".

    • Ordinal: Berg Balance Scale (0-4 scale).

    • Interval: Time (in seconds) maintaining a position.

    • Ratio: Percentage of successful attempts (e.g., standing on one leg).

  2. Hand Grip Muscle Strength for a male 72 years old:

    • Nominal: Present vs. Absent.

    • Ordinal: Scale indicating range of motion (e.g., 2/5).

    • Interval: Grip strength in lbs. (75 lbs indicates fair strength).

    • Ratio: Percent of muscle hand mass (27% for age, below normal).

Rehabilitation Measures Database

  • Resource: Shirley Ryan Ability Lab's Database

    • Purpose: Curated resources for rehabilitation assessment tools.

    • How to Use: Sift through hundreds of tools relevant to clinical needs. Not exhaustive but essential for familiarity with assessment psychometrics.

  • Examples of Searches:

    • General health assessment for Alzheimer’s (free).

    • Performance measure for joint pain/fracture.

    • Performance measure for cerebral palsy (not free).

    • Vision and perception measure for older adults.

Conclusion

  • Summary: Importance of evidence in assessments and the types of psychometric properties in standardized tools.

    • Reference: Kramer, P., & Grampurohit, G. (2020). Hinojosa and Kramer's Evaluation in Occupational Therapy: Obtaining and Interpreting Data. AOTA Press.