Different types of validity

VALIDITY

No single measure of personality is perfect. Instead, its better to use multiple techniques/principles of “triangulation”

  • Internal reliability: participants to give consistent responses, no matter how the question is phrased

  • In developing a good questionnaire, we recognize that people that might answer questions differently because of slight differences in phrasing (there are 2 factors that influence someone’s response to a questionnaire item:

  1. Their true score on some construct

  2. Random error

  • How do we account for this?

  1. Ask a lot of questions slightly different and average together

  2. Usually one or two item measures are not considered enough to accurately measure a construct

  • How do we assess internal reliability?

  1. Use statistic called Croncbach’s alpha

  2. Calculated like a correlation (ranges from 0-1) but can be negative

  3. Higher numbers indicate greater reliability

  4. Usually want an alpha>.70

  • Inter-rater reliability: measures of consistency used to evaluate the extent to which different judges agree in their assessment decisions

  1. Only applies to behavioral/observational measures

  • Construct Validity: Does your measure accurately assess what you think it does (and only what you think it does)(is your operational definition accurately measuring your concept)

  1. Face validity: your measures appears “on its face” to measure what it says it does (face value). In other words, it just looks like what you want to measure

  • Is considered “weak evidence” for validity of a measure, and it is NOT a necessary component of a good measure

  • In fact, sometimes LOW face validity may be a good thing

  1. Criterion (also predictive) validity: the extent to which a measure is correlated with a relevant observable outcome or behavior

  • Criterion- a standard on which a judgment or decision may be based (singular of criteria)

  • Considered the best evidence for validity

  • What is an “observable outcome or behavior”: gpa, x number of times doing blank, brain activity, things that are observable in general, something OUTSIDE of yourself

  • What does not count as concrete outcome or behavior: Another self report measure, self-esteem, internal feelings/subjectiveness

  1. Convergent validity: When a scale correlates with similar self report scales (measuring the same/similar constructs)

  2. Discriminant validity: A measure does NOT correlate (too highly) with unrelated constructs (very little confounding variables)(measure only ONE thing and ONLY one thing)

Content validity: Covering ALL parts of a construct (as defined by the theory) (DID YOU COVER ALL THE CONTENT)