Quiz 2 - Validity

what is the difference between internal and external validity?

• internal validity:

  • the extent that a change in the DV can be attributed to the IV; the extent that a study can rule out alternate explanations for the relationship between two variables

• external validity:

  • the extent that findings from a sample can be generalized to the corresponding population

Name the different types of test validities.

• Face Validity

• Content Validity

• Criterion Validity

  • Predictive validity

  • Concurrent validity

  • Retrospective validity

• Construct Validity

  • Convergent validity

  • Divergent validity

what is face validity?

• Does it look like it measures what it claims to measure?

  • it’s subjective

  • ex. Does a measure of locus of control seem to measure locus of control?

what is content validity? (AKA logical validity)

• Does it measure all factors of a construct?

  • it’s measured with exploratory factor analysis and other multivariate statistics

    • ex. Does a measure of locus of control measure both internal and external locus of control?

what is criterion validity?

• does it have predictive, concurrent and retrospective validity?

  • does it predict future (predictive), current (concurrent), and past (retrospective) behavior?

  • ex. a measure of criminal behavior has criterion validity if it predict past, present, and future acts of breaking the law

what is construct validity?

• does it have convergent and divergent validity?

  • convergent validity: are scores correlated with scores from another measure of the SAME construct?

  • divergent validity: are scores uncorrelated with scores from another measure of a DIFFERENT construct

what three criteria are needed for establishing causation between variables A & B?

• 1. Covariance

  • The study’s results show that as A changes, B changes, either consistently in the same direction or the opposite direction.

• 2. Temporal Precedence

  • The study’s method ensures that A comes first in time before B

• 3. Internal Validity

  • The study’s method ensures that there are no alternate explanations for the change in B.

• what is reliability?

  • how is this different from validity?

• reliability: the degree of stability or consistency of measurements

  • if the same individuals are measured under the same conditions, a reliable measure will produce nearly identical measurements

• a valid measure is reliable but a reliable measure is NOT necessarily valid

• ex. we could measure your height and claim that it is a measure of intelligence. It would be reliable but not valid.

what is accuracy?

• the degree to which a measure conforms to the established standard

• ex. the speedometer on a car may consistently read 10mph faster than the actual speed.

  • It reliable BUT inaccurate

• what are the characteristics of good survey items?

• BRUSO

  • Brief

    • Poor: Have you at any point in your life been the owner of a gun?

    • Good: Have you ever owned a gun?

  • Relevant

    • Poor: What is your sexual orientation?

    • Good: Would you like to own a gun?

  • Unambiguous

    • Poor: Are you a gun person?

    • Good: Do you own a gun?

  • Specific

    • Poor: How much have you read about the new gun control measure and sales tax?

    • Good: How much have you read about the new gun control measure? How much have you read about the new sales tax?

  • Objective

    • Poor: How much do you support the new gun control measure?

    • Good: What is your view of the new gun control measure?

what are the type of survey questions?

• 1. Open-ended questions:

  • allow respondents to answer how they choose

  • these are similar to short-answer or long-answer quiz questions

• 2. Closed-ended questions.

  • these are similar to multiple-choice quiz questions. They can also include Likert-type questions

what are the strengths of open-ended questions vs. closed-ended questions?

• open-ended questions

  • good for gathering rich, detailed information

  • better when you don’t know or don’t want to limit how participants will respond

  • more work to code and analyze

• closed-ended questions

  • good when you want to control / limit the type of data you get from responses

  • better when the construct or variable is well-defined and you have a clear idea about the types of responses people are likely to give

  • more work to write good questions and options