127-129

Validity in Measurement

  • Definition of Validity

    • Validity refers to how well an empirical measure reflects the intended concept.

    • Measurement validity addresses whether a measurement truly captures what it claims to measure.

  • Operational Definitions

    • Operational definitions specify how concepts are measured.

    • A valid measure accurately reflects the concept in question.

  • Example of Validity

    • A laser speed detector accurately measures vehicle speed.

    • A wind velocity indicator does not validly measure vehicle speed due to different displacement mechanics.

Assessing Validity

  • Face Validity

    • Face validity assesses whether a measure intuitively appears to be valid based on common understanding and logical reasoning.

    • Example: Counting citizen complaints may or may not adequately measure satisfaction with police services.

    • Illustration: A study by Ditte Johansen (2006) where electronic records of food purchases indicated healthiness based on drink choices highlighted face validity.

  • Criterion-Related Validity

    • Involves comparing a measure to an established external criterion to assess its validity.

    • Convergent Validity: Shows a measure's ability to predict cases on other, accepted measures.

    • Example: College Board exams' effectiveness in predicting college success.

    • Example: Voas et al. (2009) validated a measure of alcohol-related auto accidents by comparing it to blood alcohol test results from drivers.

  • Discriminant Validity

    • This demonstrates that a measure can differentiate between similar but distinct concepts.

    • Example: Research by Jacinta Gau (2011) distinguishes between obligations to obey legal authorities and trust in police.

Construct Validity

  • Definition

    • Assessing construct validity involves examining the logical relationships among different variables.

    • Example: The measure of fear of crime should relate logically to other related measures in expected ways.

Content Validity

  • Definition

    • Content validity evaluates whether a measure captures the full range of meanings within a concept.

    • Example: A question on safety after dark captures some aspects of fear but lacks comprehensiveness for the broader concept of fear of crime (which includes burglary, auto theft, etc.).

Multiple Measures

  • Approach

    • Comparing multiple measures within a given concept can support the validation of individual measures.

    • Example: Examining the relationship between fear of crime and the tendency to limit outings at night.

    • Validity is strengthened if theoretical expectations about relationships hold true, as shown in studies by Ditton and Farrall (2007).

Reliability and Validity Relationship

  • Understanding Reliability vs. Validity

    • Illustrative analogy:

      • A reliable measure consistently hits a 'target.'

      • A valid measure accurately hits the target (bull's-eye).

  • Implications

    • Validity failures indicate systematic error, while reliability failures result in random error.

    • Utility of a measure diminishes if it lacks both reliability and validity.

Composite Measures

  • Utility of Combining Measures

    • Composite measures, such as indices, often provide more reliable and valid indicators than individual measures.

    • Example: A property crime index could combine burglary, larceny, and vehicle theft; a violent crime index would include murder, robbery, etc.

  • Reasons for Composite Measures

    1. Sometimes, concepts are complex, requiring more than one indicator for valid measurement (e.g., fear of crime).

    2. Ordinal measures might need a refined range of categories, achievable through indexes or scales.

    3. Increased efficiency in data analysis through multiple data items leads to more comprehensive evaluations.