Methods in Behavioral Research: Chapter 1 Study Guide
Foundations of Scientific Understanding and Research Consumption
Learning about research methods provides the necessary foundation to become an educated, critical consumer of behavioral research and scientific findings.
Occupational domains across multiple fields rely directly on the evaluation and application of empirical research findings, including:
Mental health professionals, who must evaluate treatment efficacy and clinical evidence.
Business practitioners, who evaluate employee behavior, consumer choices, and organizational data.
Educators, who assess pedagogical strategies and learning outcomes.
Scientific research, behavioral science, developmental psychology, and neuroscience play a critical role in public policy and judicial decision-making, directly influencing landmark decision-making by the U.S. Supreme Court.
Understanding scientific methodology is essential for designing, evaluating, and refining targeted social, organizational, and educational programs.
Research methods provide systematic tools to satisfy intellectual curiosity regarding human behavior, societal dynamics, and personal interactions.
Non-Scientific Ways of Knowing
Intuition and Anecdotal Evidence:
Intuition involves accepting one's personal judgment, singular experiences, or isolated stories unquestioningly.
Drawing general conclusions about human behavior based on intuition or anecdotal evidence is fundamentally flawed due to cognitive and motivational biases.
Human perception is routinely systematically skewed by these biases, leading directly to incorrect conclusions.
Illusory Correlation: A specific cognitive bias that occurs when an individual focuses selectively on two salient events that stand out and happen to occur together, leading to the false conclusion that the events are linked when no actual relationship exists.
Authority:
People are naturally inclined to accept claims made by individuals who appear prestigious, trustworthy, highly educated, or respectable.
Uncritical acceptance occurs across numerous external sources of authority, including the internet, news media, books, government officials, celebrities, religious figures, and university professors.
The scientific approach explicitly rejects the notion that statements from any authority figure can or should be accepted on faith without empirical evidence.
Core Principles of the Scientific Approach
Empiricism:
While intuition, personal anecdotes, and statements of authority can serve as initial sources of ideas or hypotheses about behavior, all such ideas may be entirely wrong, including those proposed by scientists themselves.
Knowledge must be derived from systematic empiricism, which requires that conclusions be grounded strictly in structured observations and analyzed data.
Scientific Skepticism: The foundational mindset requiring that all ideas and hypotheses be evaluated strictly through careful logic and the results of well-executed scientific investigations.
Goodstein's Evolved Theory of Science:
Goodstein (2000) defined an evolved theory of science outlining four fundamental characteristics of scientific inquiry:
Data Play a Central Role: Empirical observations and structured data collection form the essential basis for drawing conclusions, taking precedence over speculation or unverified assertion.
Scientists Are Not Alone: Science is a public and communal enterprise where observations and procedures are reported to and evaluated by the broader scientific community.
Science Is Adversarial: Scientific ideas contend against one another in open debate and rigorous testing.
Falsifiability: The mandatory principle that a scientific idea, hypothesis, or theory must be formulated in such a way that it can be proven false when tested using scientific methods.
Scientific Evidence Is Peer Reviewed:
Peer Review: The process of judging the merit, validity, and methodology of research through detailed evaluation by expert scientists within the relevant field prior to publication.
Advantages of Scientific Inquiry:
The primary advantage of the scientific approach over other ways of knowing is its reliance on an objective set of rules for gathering, evaluating, and reporting empirical observations.
Opinions and personal beliefs are acceptable provided they are presented strictly as opinions or beliefs; however, scientific inquiry requires determining whether an opinion can be empirically tested and whether scientific evidence exists to support or refute it.
Identification and Analysis of Pseudoscience
Understanding Pseudoscience:
Skepticism is defined as maintaining an attitude of doubt regarding the truth of assertions or conclusions.
Skepticism must be applied to media reports of research, scientists' unverified assertions, and pseudoscientific claims.
Pseudoscience: The practice of using seemingly scientific terminology, concepts, and physical demonstrations to substantiate nonscientific, unverified claims.
Facilitated Communication: A prominent historical example of a pseudoscientific procedure used in an attempt to assist non-verbal individuals, which failed under empirical testing.
Indicators of Pseudoscientific Claims:
Claims that are inherently untestable and structured such that they cannot be refuted or proven false.
Reliance on imprecise, biased, or intentionally vague language.
Reliance on anecdotal evidence, testimonials, and personal success stories rather than objective scientific data.
Assertions made by "experts" who possess vague or dubious qualifications and fail to support their claims with rigorous scientific evidence.
Selective presentation of confirmatory evidence while actively ignoring, dismissing, or suppressing conflicting scientific evidence.
Reliance on purported "scientific" evidence that cannot be independently verified or replicated because the methodology used to gather the evidence is kept secret or inadequately described.
Consumer Guidelines for Media and Online Claims:
Exercise extreme skepticism toward scientific claims supported only by vague, improbable, or extraordinary evidence without empirical backing.
Perform independent searches for verifiable, peer-reviewed scientific evidence before accepting media statements regarding scientific findings.
Framework for Evaluating Behavioral Research
Critical evaluation of a research study requires systematically answering eight fundamental questions:
What is the primary goal of the study (Description, Prediction, Determining Cause, or Explanation), and did the authors successfully achieve that goal?
What specific methodology and procedural design did the researchers utilize?
What specific variables and constructs were measured?
To what broader populations, settings, or conditions can the results of the study be validly generalized?
What were the exact empirical findings and statistical results of the study?
Have other independent researchers successfully replicated these results or found consistent outcomes?
What are the specific methodological and theoretical limitations of the study?
What ethical issues were present in the study, and how were they addressed?
Four Primary Goals of Behavioral Science
1. Description of Behavior:
Scientific research aims to systematically describe behavioral phenomena, which may include directly observable actions or less observable internal states (e.g., mental states, cognitive processes).
Description involves detailing how events or variables systematically relate to one another (e.g., investigating whether jurors render more lenient sentences to attractive defendants compared to unattractive defendants).
Empirical Descriptive Example (Adolescent Screen Time Use During COVID-19 Pandemic):
Measurement Scale: Daily screen time measured on a continuous horizontal axis ranging from to , structured in increments of .
Subgroup Descriptive Means:
Entire Sample Average: per day.
Black Adolescents: per day.
Hispanic or Latinx Adolescents: per day.
White Adolescents: per day.
Asian Adolescents: per day.
2. Prediction of Behavior:
Once a consistent, systematic relationship between two events or variables is identified through descriptive observation, researchers can form predictions about future outcomes.
Example: Establishing a systematic empirical link between early-childhood poverty and lower academic achievement enables predicting that children experiencing early-childhood poverty will display lower academic achievement metrics.
All behavioral predictions are probabilistic in nature rather than deterministic guarantees.
3. Determination of Cause:
Identifying the cause of a behavior is essential for developing effective interventions to modify or change behavior.
Establishing causal relationships requires meeting three precise criteria defined by Cook and Campbell (1979).
4. Explanation of Behavior:
The ultimate goal of behavioral science is to explain why a behavior occurs and why specific relationships exist between variables.
Example: Beyond demonstrating that childhood poverty predicts lower academic achievement, science must elucidate the underlying mechanisms and underlying theoretical reasons explaining why childhood poverty causes lower academic achievement.
Scientific explanations are dynamic; empirical findings continually generate new research questions, requiring theoretical explanations to be discarded or revised as new data accumulate.
Inferring Causation: Cook and Campbell's Criteria
According to Cook and Campbell (1979), inferring cause and effect requires satisfying three distinct evidentiary criteria:
Temporal Precedence:
The proposed cause must precede the observed effect in time.
Example: A researcher must demonstrate that playing violent video games occurs prior to the subsequent display of aggressive behavior over time.
Covariation of Cause and Effect:
When the proposed cause is present, the effect occurs; when the proposed cause is absent, the effect does not occur.
Example: Individuals who play violent video games display aggressive behavior, whereas individuals who do not play violent video games do not display aggressive behavior.
Elimination of Alternative Explanations:
All third variables or potential confounding factors that could account for the observed relationship must be ruled out.
Example: Ruling out alternative causes such as a lack of parental supervision, which could independently cause both increased violent video game play and increased aggressive behavior.
Distinguishing Basic and Applied Research
Basic Research:
Seeks to answer fundamental theoretical questions concerning the essential nature of human behavior, cognition, and underlying psychological phenomena.
Typically focuses on developing, testing, and refining theoretical frameworks rather than solving immediate real-world problems.
Applied Research:
Designed to address practical problems in real-world settings and identify immediate, functional solutions.
Program Evaluation: A major subfield of applied research that systematically assesses the implementation, effectiveness, and impact of social reforms, public policies, and educational or governmental innovations.
Applied research findings are frequently proprietary, conducted for client organizations or internal company decision-making, and remain unpublished in scientific journals.
Interdependence of Basic and Applied Science:
Neither basic nor applied research is superior; both are fundamental components of scientific progress.
Applied research reliance: Applied interventions and evaluation studies are directly grounded in theories and foundational knowledge developed through basic research.
Basic research stimulation: Unexpected findings, challenges, and outcomes discovered in applied settings frequently generate new theoretical questions that prompt basic research.
Unpredictable long-term value: The ultimate real-world applications of basic research cannot be predicted in advance; studies with no immediate practical application frequently yield foundational knowledge that proves invaluable over time.
Methodological unity: Researchers across basic research, applied research, and program evaluation utilize identical scientific methods and empirical standards.