Experimental Research Notes

Validity in Experimental Designs

External Validity

• Definition: The degree to which an experimental design reflects real-world issues.
• Focus: How well lab analogues represent real-world contexts.
• Assessment: Determines if experimental methods and results generalize to the real world.
• Example: A study on stress and creativity.
  • Independent variable: Stress (manipulated with loud noises).
  • Dependent variable: Creative problem solving (listing uses for a cardboard box).
  • Questioning external validity:
  • How similar are random loud noises to everyday stress?
  • Is listing uses for a cardboard box a good indicator of creativity?
  • Conclusion: Need to consider if results generalize to the real world.
• Core Question: Do operational definitions effectively represent real-world processes?

Internal Validity

• Definition: The degree to which changes in the dependent variable are genuinely due to the manipulation of the independent variable.
• Focus: Ensuring experimental methods are free from biases and logical errors.
• Goal: To ensure that results are trustworthy and not influenced by extraneous factors.
• Importance: Experimental research requires safeguards to maintain internal validity.

Experimenter Bias

• Definition: The influence of the experimenter's expectations on the outcome of the research.
• Mechanism: Experimenters may subtly (and often unknowingly) influence research participants.
• Demand Characteristic: Any aspect of a study that communicates to participants how the experimenter wants them to behave.
• Classic Study: Robert Rosenthal (1966).
  • Method: College students assigned as experimenters with rats.
  • Groups: Half told their rats were "maze bright," half told "maze dull."
  • Results: "Maze-bright" rats performed better due to students' expectations.
• Impact: Experimenter bias can introduce systematic differences between experimental and control groups.
• Consequence: Cannot definitively attribute outcomes to the independent variable alone.
• Confounds: Factors that "ride along" with the experimental manipulation, influencing the dependent variable undesirably.

Research Participant Bias and the Placebo Effect

• Research participant bias: Participants' expectations about what they are supposed to do and how they should behave influencing the results of experiments.
• Placebo Effect: Participants' expectations, rather than the experimental treatment, produce a particular outcome.
• Drug Study Example:
  • Experimental group: Receives a pill containing an actual painkiller.
  • Control group: Receives a placebo pill (harmless substance with no physiological effect).
  • Purpose of Placebo: To treat both groups identically except for the active agent (painkiller).
  • Goal: Determine whether changes in the experimental group are due to the drug or participants' expectations.

Double-Blind Experiment

• Definition: An experimental design in which neither the experimenter nor the participants are aware of which participants are in the experimental group and which are in the control group until the results are calculated.
• Purpose: To ensure that neither the experimenter's nor the participants' expectations affect the outcome.
• Mechanism: Prevents subtle gestures or signals from the experimenter.
• Benefit: Distinguishes the specific effects of the independent variable from the possible effects of expectations from both experimenters and participants.
• Example: COVID-19 vaccine studies used double-blind protocols.
  • Neither the person giving the shots nor the people getting them knew if the shots contained vaccine or not.
  • Ensures unbiased results regarding vaccine efficacy and safety.

Applications of the Three Types of Research

• Descriptive, correlational, and experimental research can be used to address the same topic.
• Example: Social Media Use
  • Descriptive Research:
  • Goal: Determine the basic dimensions of a phenomenon.
  • Observation: How much time individuals spend on social media each day.
  • Interviews/Surveys: How people describe their social media use.
  • Case Studies: Analyze social media pages of individuals to reveal important information.
  • Strengths and Weaknesses: Lays groundwork for future research but doesn't explain processes or provide generalizable conclusions.
  • Correlational Research:
  • Goal: Determine how variables change together.
  • Example: Relationship between hours spent on social media and face-to-face interactions.
  • Strengths and Weaknesses: Provides information about how variables change together but doesn't allow causal conclusions.
  • Experimental Research:
  • Goal: Determine whether a causal relationship exists between two variables.
  • Example: How going "cold turkey" on social media influences stress levels.
  • Strengths and Weaknesses: Permits causal conclusions, but potential artificiality might raise concerns about external validity.

Experimental and Control Groups

• Experimental group: Participants exposed to the change that the independent variable represents.
• Control group: Participants as much like the experimental group as possible, treated in every way like the experimental group except for the change.
• Purpose of Control Group: Provides a comparison against which the researcher can test the effects of the independent variable.
• Music and Intelligence Study: Experimental group listens to classical music; control group listens to no music.
• Happiness and Meaning in Life Study: Experimental group listens to happy music; control group hears neutral music.
• Risk Taking and Trust Study: Risk-taking and risk-avoiding groups were the experimental groups; the third group was the control group.
• Experiments are the key way that researchers determine causation.

Quasi-Experimental Designs

• Definition: Similar to an experiment, but does not randomly assign participants to conditions.
• Reason for Use: Random assignment is either impossible or unethical.
• Examples:
  • Soldiers who have seen combat versus those who have not.
  • Children whose school was destroyed by a tornado versus those in a neighboring town where the school was not affected.
• Group Assignment: Not determined randomly; might be determined by natural events or by participants placing themselves in groups of interest.
• Example Study: People exposed to natural disasters early in life compared to those who were not.
  • Found that people who had experienced a natural disaster before their first birthday finished fewer years of school and experienced poorer socioeconomic functioning in adulthood.
• Pitfalls: Cannot definitively determine causality due to lack of random assignment.
• Example: Researcher interested in the influence of using online learning tools on performance in introductory psychology.
  • Compares students from two different sections of a class—one that uses online tools and one that does not.
  • Differences might be due to other factors (e.g., whether students are morning people or not).
• Conclusion: Do not allow for the strong causal conclusions that can be drawn from true experiments with random assignment.

Analyzing and Interpreting Data

• Measures of Central Tendency: Used to describe data sets.
  • Mean: The average score, calculated by summing all scores and dividing by the number of scores.
  • Median: The score that falls exactly in the middle of the distribution of scores after they have been ranked from highest to lowest.
  • Mode: The score that occurs most frequently in a distribution.
• Effects of Extreme Scores:
  • Outliers (extreme scores) can greatly affect the mean.
  • In such situations, the median or the mode may provide a more accurate picture of the data.

Examples of Means, Medians, and Modes:

• Group 1: Earnings of five ordinary people (39,000, 53,000, 54,000, 39,000, 55,000)
  • Mean: 48,000
  • Median: 53,000
  • Mode: 39,000
• Group 2: Earnings of four ordinary people (same as above) plus Steven Spielberg's annual earnings (150,000,000)
  • Mean: 30,037,000
  • Median: 53,000
  • Mode: 39,000