Title: Three Claims, Four Validities: Interrogation Tools for Consumers of Research
Focus: Understanding how to evaluate claims in psychology research articles and popular media.
Introduction
Psychology-related media articles garner interest due to topics like happiness and social interaction. However, readers must critically evaluate these claims as they may be misrepresented.
Objective: Equip psychology students with tools to assess the credibility of research claims.
Learning Objectives
Differentiate between three types of claims: frequency, association, and causal.
Ask targeted questions related to four validities: construct, statistical, external, and internal validity.
Identify which validities are significant for each claim type.
Recognize the research type required to support causal claims.
Variables
Definition and Importance
Variable: Any entity that can take on different values. It has at least two levels or values.
Examples of Variables:
Smiling Yesterday: Yes/No format, classified under the variable as seen in the headline "72% of the world smiled yesterday."
Income: Low, medium, high - in studies observing its effects on social behaviors.
Constants vs. Variables
Constant: A feature that does not change across measurements; e.g., nationality in a study that focuses solely on Americans.
Measured and Manipulated Variables
Measured Variables: Observed and noted without manipulation, e.g. height, weight, gender.
Examples:
Weight measured by scales, gender noted via surveys.
Abstract constructs like stress evaluated via specific questionnaire items.
Manipulated Variables: Experimenter controls this variable by assigning participants to different levels, e.g. dosage of medication.
Examples: Participants receiving varying doses of the same medication (10mg, 20mg, 30mg).
Certain variables (like age) can only be measured due to ethical concerns about manipulation.
Conceptual Variables vs. Operational Definitions
Conceptual Variables
Abstract constructs used in theories, e.g., social engagement, school achievement.
Operational Definitions
Specific, observable, measurable definitions of variables for research purposes.
Example:
Social Engagement: Could be operationalized by the frequency of social gatherings or meal instances with friends.
Three Types of Claims
1. Frequency Claims
Definition: Claims that report the rate at which a certain behavior or characteristic appears within a population.
Examples:
"4 in 10 teens admit to texting while driving."
The claim indicates a single variable's frequency, focusing solely on that variable.
Characteristics of Frequency Claims
Focus on one variable.
Always measured variables, never manipulated.
2. Association Claims
Definition: Claims stating one variable is associated with another, demonstrating a relationship between two or more variables.
Examples:
"People with higher incomes spend less time socializing."
These claims suggest correlations without implying causation.
Types of Associations
Positive Association: Both variables increase together.
Example: Higher gratitude correlates with relationship longevity.
Negative Association: One variable increases while the other decreases.
Example: High multitasking correlates with lower multitasking skills.
Zero Association: No relationship exists between variables.
Example: Dinner time has no relation to childhood obesity.
3. Causal Claims
Definition: Claims asserting one variable directly influences another.
Examples:
"Music lessons enhance IQ."
Implies direct causation between variables.
Causal Claim Requirements
Covariance: The variables must show a correlation.
Temporal Precedence: The purported cause must precede the effect in time.
Internal Validity: Other potential explanations for the relationship must be ruled out (only a properly designed experiment can achieve this).
Validities in Research
Overview
Validity: Refers to the extent to which a study accurately reflects or assesses the concept it intends to measure.
Four Big Validities
Construct Validity
Assesses how accurately concepts are operationalized and measured.
Example frame: How accurately was anxiety measured?
External Validity
Examines the generalizability of findings across populations, settings, and times.
Example frame: How well can the results be generalized?
Statistical Validity
Evaluates how well the conclusions drawn from the data are supported statistically.
Example frame: What is the margin of error?
Internal Validity
Focuses on whether the study properly establishes a cause-effect relationship.
Example frame: Are there alternative explanations for the results?
Interrogating Claims Using Validities
Evaluating Frequency Claims
Focus on construct and external validity primarily, but consider statistical validity.
Evaluating Association Claims
Assess construct, external, and statistical validity detailing the degree of association strength.
Evaluating Causal Claims
Requires interrogating all four validities focusing on internal validity.
Applying the Framework
The structured approach can assist in assessing claims made both in academic settings and in popular media, encouraging a critical mindset while consuming information.
Example of Practical Application
To evaluate a claim like "stories told of brilliant scientists affect kids’ interest in science":
Determine Claim Type: Causal claim due to use of affect.
Investigate Constructs: Assess how well each variable was manipulated and measured.
Evaluate Validities: Apply frameworks to check each aspect of validity as detailed above.
Conclusion
Learning to recognize and assess these claims will empower students to consume research critically and refine their research methodologies when conducting their studies.
Key Terms
Variable: Core unit of research; something that can vary and be measured.
Claim: Assertion made based on observed facts.
Construct Validity: Accuracy of operational definitions.
External Validity: Generalization potential of study findings.