Chapter 3 Notes: Three Claims, Four Validities

Chapter 3 Notes: Three Claims, Four Validities

  • Chapter title: Three Claims, Four Validities: Interrogating Research

  • Source context: Slides from a psychology/textbook chapter introducing the three types of claims, the four big validities, and how to interrogate research.

Agenda

  • Main topics covered:

    • Types of Variables

    • Operational Definitions

    • 3 Types of Claims

    • The Big 4 Validities

  • Quick cue: The module emphasizes how to evaluate research quality and the relationships among conceptual variables, their operationalizations, and the strength and limits of conclusions.

Variables and constants

  • Example prompt: Variables in the statement “Study shows that college students who watch cat videos are happier”

    • Question: What are the variables?

    • Question: What is the constant?

  • Note: An image credit line is shown: "This Photo by Unknown Author is licensed under CC BY-SA-NC 3" (copyright/credit note).

Measured vs. Manipulated Variables

  • Measured variables:

  • Manipulated variable: _

  • Important nuance: Some variables can only be measured; other variables can be manipulated or measured.

  • Page reference: 4

From Conceptual to Operational Definition

  • Concept:

  • Conceptual definition: _

  • Operational definition:

  • Page reference: 5

Example – Life satisfaction

  • Page 6 content highlights:

    • Concept: “Satisfaction with life”

    • Conceptual definition: “A person’s cognitive evaluation of his or her life”

    • Operational definition: 5 questionnaire items on the Satisfaction with Life scale, answered from 1 (strongly disagree) to 5 (strongly agree)

Example – Persistence in young children

  • Page 7 content highlights:

    • Concept: “Persistence” (young children specifically)

    • Conceptual definition: “The ability to push through when confronted with … obstacles”

    • Operational definition: How long a child will choose to engage in a slow-paced, boring activity that involves pressing a button when they see a picture of cheese on a screen and not pressing when they see a cat on the screen

From Conceptual to Operational Definition (continued)

  • Construct: - Weight gain in dogs - Anxiety

  • Operational definition: Good; Bad

    • Dog’s weight - Anxious; relaxes; uneasy, worried

    • Sweating palms, increased heart rate, etc. or a validated survey

  • Page 8

From Conceptual to Operational Definition (summary)

  • A single conceptual variable can have many different operationalizations.

  • Example prompt: “What grades do you get?”

    • All As

    • Mostly As and Bs

    • Mostly Bs

    • Mostly Bs and Cs

  • Self-report questionnaire as an operationalization of "school achievement"; other options include checking records, teachers’ observations.

  • Left image: Christina K. Betz; Right image: Sean Gallup/Getty Images

  • Key terms: Conceptual variable; Operational variables

  • Page 9

Table 5.1: Dictionary and Operational Definitions of Terms Commonly Used by Psychologists

  • For each concept, other operational definitions are possible.

  • Terms and examples:

    • Punishment: Dictionary definition — Harsh or injurious treatment for an offense; Operational definition — Presentation of 3 milliamp shock for .5 second following certain (specified) behavior

    • Learning: Dictionary definition — Acquiring knowledge or skill; Operational definition — Change in behavior (specify kind of behavior) as a function of practice

    • Anxiety: Dictionary definition — State of being uneasy, apprehensive, or worried; Operational definition — Self-reported anxiety on a scale of 1 to 7; or physiological measures (sweat gland activity, heart rate, etc.)

    • Intelligence: Dictionary definition — Ability to learn or understand from experience; Operational definition — Score on standardized tests (e.g., Stanford-Binet, Wechsler scales)

    • Thirst: Dictionary definition — Distressful feeling caused by a desire or need for water; Operational definition — Eighteen hours (or other value) without access to water

    • Sleep: Dictionary definition — Recurring condition of rest, no conscious thought, eyes closed, etc.; Operational definition — Specific brain wave frequencies (EEG) for different sleep stages

    • Guilt: Dictionary definition — A painful feeling of self-reproach; Operational definition — Score on a personality inventory, self-reported guilt on a scale of 1 to 10

  • Source: http://uca.edu/psychology/files/2013/08/Ch5-Measurement.pdf

  • Page 10

Three Claims

  • Frequency Claims

  • Association Claims

  • Causal Claims

  • Table 3.3: Examples of Each Type of Claim (sample headlines)

    • Frequency claims examples include percentages or rates (e.g., “39% of Teens Admit to Texting While Driving”)

    • Association claims examples include links between variables (e.g., “Playing an Instrument Is Linked to Better Cognition”)

    • Causal claims examples include actions that imply causation (e.g., “Pretending to Be Batman Helps Kids Stay on Task”)

  • Page 11

Frequency Claims details

  • Page 12 content:

    • The rate or degree of a _

    • Examples: 39%39\% of people lying, amount of books read, average crime rate

  • Page 12

Association Claims

  • Page 13 content:

    • One level of a variable is likely associated (correlated) with a particular level of a different variable

    • At least

  • Page 13

Association Claims – 3 types

  • Positive association: __

  • Negative association (inverse association): ___

  • Zero association:

  • Page 14

Correlation examples (visuals described in slides)

  • Positive correlation example: Years of education vs. height? (illustrative data showing a positive trend)

  • Negative association example: Dental problems requiring care vs. Hours of sleep; or Sleep vs. Annual income (illustrative)

  • Zero association example: Shoe size vs. daily hours of video game play (no relation)

  • Pages 15-17

Causal Claims

  • Page 18 content highlights:

    • Causal claim: _

    • At least

    • Independent variable (IV) is manipulated

    • Dependent variable (DV) is measured

    • Control for confounding variables

  • Related through the four big validities framework

Word Choice: Association vs Causal

  • Table 3.4: Verbs distinguishing association vs causal claims

    • Association claim verbs: is linked to, is associated with, is correlated with, etc.

    • Causal claim verbs: causes, promotes, increases, reduces, improves, prevents, enhances, etc.

    • Other example verbs listed include: affects, exerts, may lead to, may predict, etc.

  • Page 19

Practice items: Identifying claim types and study design

  • Page 20: Which of the following is an association claim? options include a mix of statements; correct one must reflect non-causal language.

  • Page 21: Variables in the claim exercise – identify the variables and their operationalization.

  • Page 22: What kind of study was it likely based on? correlational vs. experimental.

  • Page 23: Which phrases indicate a causal claim? examples include curbs, may enhance, etc.; note: some phrasing like may imply causation depending on context.

  • Page 23

Four Big Validities

  • Table 3.5: The Four Big Validities

    • Construct validity: How well the variables in a study are measured or manipulated; extent to which operational variables approximate conceptual variables

    • Statistical validity: How strong the effect is, precision, confidence intervals, and replication

    • Internal validity: For causal claims; whether A causes B after accounting for confounds

    • External validity: Generalizability to populations, settings, and times

  • Page 24

Interrogating Frequency Claims

  • Construct validity: Do levels of a variable correspond to real differences? Are variables measured reliably?

  • Statistical validity: What is the margin of error of the estimate? Confidence interval?

  • External validity: Are results generalizable to populations/settings/times?

  • Page 25

Interrogating Association Claims

  • Construct validity: How well are the variables operationally defined?

  • Statistical validity: Significance of the correlation coefficient; strength of the association; replication

  • External validity: Generalizability of the association to other populations/settings/times

  • Page 26

Interrogating Causal Claims

  • Internal validity: Key criteria

    • Covariance: _

    • Temporal precedence: _

    • Internal validity (causal explanation): _

  • Also consider Construct validity, Statistical validity, and External validity

  • Page 27

Example studies for validity interrogation

  • Does social media use harm adolescents? (Two example sources)

    • The Impact of Social Media on Teen Mental Health (University of Utah Health, 2022)

    • Fewer Likes on Social Media Causes Real Emotional Distress in Adolescents (CBS Austin, 2020)

  • For each, assess:

    • Covariance: is there a relationship between social media use and distress or well-being?

    • Temporal precedence: does the cause precede the effect?

    • Internal validity: could another variable explain the relationship?

  • Page 28-29

Table 3.6: Interrogating the Three Types of Claims Using the Four Big Validities

  • Frequency Claims: typical in surveys/polls; focus on construct validity, statistical validity (margin of error, confidence intervals); not usually internal validity because causality is not claimed; external validity concerns generalizability

  • Association Claims: typically correlational; assess measurement quality and strength/replication; internal validity less central to the claim itself, but important for causal inference if pursued; external validity concerns generalizability

  • Causal Claims: require experimental evidence; assess measurement/manipulation quality; effect size; precision; replication; temporal precedence; control for alternative explanations via random assignment; internal validity threats addressed

  • Page 30

Prioritizing Validities

  • Question: Which validity is most important? Depends on goals; some validities are hard to achieve simultaneously

  • Note: Often one validity is deprioritized given constraints

  • Page 31

Four big validities in practice

  • Question: Which validity is appropriate to interrogate for every study? Options:

    • a. construct

    • b. statistical

    • c. internal

    • d. external

  • Page 32

Trade-offs among validities

  • Question: Which two validities are most often in a trade-off (researchers give up one to prioritize the other)?

    • a. construct and statistical

    • b. internal and external

    • c. internal and statistical

    • d. construct and external

  • Page 33

Variable in Context (Table style) – Conceptual vs Operational

  • Demonstrates how a single variable can have multiple operational definitions

  • Examples from the slide:

    • Demographic information: level of education

    • Conceptual Level: Level of education

    • Operational Definition: Highest level of education from the list: High school diploma, Some college, College degree, Graduate degree

    • Levels: High school diploma; some college; college degree; graduate degree

    • Measured

    • Anxiety: Measured with a 20-item Spielberger Trait Anxiety Inventory

    • Readability study: fonts (sans-serif vs serif)

    • SAT score as a measure of college readiness

    • Self-control & blood sugar: lemonade with sugar vs sugar-free

  • Page 34

Let’s Review! – Chapter 3 recap and practice prompts

  • Suggested activities:

    • Read the Chapter 3 summary and answer review prompts

    • Study questions include:

    • What is the difference between a variable and its levels?

    • What might be the levels of the variable “favorite color”?

    • Why can some variables only be measured, not manipulated? Could “history of trauma” be manipulated? Could “level of eye contact” be manipulated?

    • Difference between conceptual variable and operational definition

    • How might conceptual variables like “level of eye contact,” “intelligence,” and “stress” be operationalized?

    • How many variables are there in a frequency claim? an association claim? a causal claim?

    • Which part of speech in a claim helps differentiate association vs causal claims?

    • How are causal claims special compared with the other two claim types?

    • What are the three criteria that causal claims must satisfy?

    • What questions would interrogate a study’s construct validity?

    • Summarize in your own words at least three things that statistical validity addresses.

    • Define external validity using the term generalize.

    • Why is a correlational study not sufficient to support a causal claim?

    • Why don’t researchers usually aim to achieve all four big validities at once?

  • Page 35

Lecture take-homes

  • Key questions and distinctions:

    • What is the difference between a constant and a variable?

    • What is the difference between a measured and a manipulated variable? What types of claims are each used to assess?

    • What distinguishes a frequency claim from the other types?

    • What distinguishes an association claim from a causal claim?

    • Provide examples of a main conclusion from a frequency claim, an association claim, and a causal claim.

    • Briefly summarize the four main types of validity introduced in the chapter.

  • Page 36

Summary of the four big validities and their roles

  • Construct validity: Are the variables properly defined and measured/manipulated to capture the intended concepts?

  • Statistical validity: Are the results precise and reliable? What are effect sizes, confidence intervals, and replication status?

  • Internal validity: Does A cause B, after ruling out confounds? Are there temporal precedence and covariance evidence, and are alternative explanations controlled?

  • External validity: Can findings generalize beyond the study context (population, setting, time)?

Quick reference: key terms and distinctions

  • Variable vs. levels: A variable can have multiple levels (e.g., education level has multiple categories).

  • Conceptual vs. Operational definitions: The abstract construct (conceptual) vs. how it is measured or manipulated in a study (operational).

  • Claim types: Frequency (descriptive rates), Association (correlations), Causal (experimental causation).

  • Verbs as clues: association verbs (is linked to, correlates with) vs causal verbs (causes, increases, reduces, promotes).

  • Why not all four validities are maximized simultaneously: trade-offs are common due to practical constraints in research design and resources.

"