Social Science Research: Manipulations and Quasi-Manipulations Study Guide

Core Concepts of Reliability and Validity

  • Reliability vs. Validity Overview:     * Reliability: Defined by the concept of consistency. It refers to the extent to which a measure produces consistent results across repeated applications or over time.     * Validity: Defined by the concept of accuracy. It refers to the correctness of the information or whether the tool measures what it is intended to measure.

  • The Interdependence of Reliability and Validity:     * Reliability is considered a prerequisite for validity.     * It is possible to have a measure that is reliable but not valid.         * Example: Chris's Scale. If a scale (Chris's scale) consistently weighs someone at the same weight every day (reliability) but that weight is actually 10 pounds off the actual weight, the scale is reliable but invalid.     * Conversely, you cannot have a valid measure unless it is also reliable. A tool that provides inconsistent results cannot be accurate.

Understanding Validity in Social Science Research

  • Definition and Correctness:     * Validity concerns the accuracy and correctness of information.     * The transcript provides a comparative example to illustrate validity:         * Statement A: "Dr. Poepsel is 77 feet tall."         * Statement B: "Dr. Poepsel is 66 feet, 11 inch tall."         * Statement B is the valid statement.

  • Concrete vs. Abstract Variables:     * Establishing validity is relatively straightforward when dealing with concrete variables (like height).     * In psychology, establishing validity is significantly more difficult because research often involves constructs (abstract concepts).

  • Challenges in Psychological Measurement:     * High-level questions regarding validity include:         * "How can we validly measure altruism?"         * "How can we validly predict whether providing help to others increases one’s own well-being?"

  • Levels of Validity Consideration:     * Validity must be addressed at both the measurement level (the tools used) and the study level (the overall research design).

Internal Validity and Causality

  • Internal Validity Definition:     * This refers to the ability of the research to make stringent inferences of causality.     * It ensures there is an accurate relationship between the Independent Variable (IV) and the Dependent Variable (DV).     * It confirms that the IV is the actual cause of the observed changes in the DV.

  • Requirements of Causality: To establish that one variable causes another, three criteria must be met:     1. Correlation: There must be a correlation between variable XX and variable YY.     2. Temporal Precedence: A change in one variable must precede the change in the other (e.g., XX must occur before YY).     3. Rule out Confounds: The researcher must ensure that XX is the only factor in the study that could cause changes in YY.

Types of Research Designs

  • Experimental Designs:     * The researcher manipulates the Independent Variable.     * Includes random assignment of participants to groups.     * The Dependent Variable (DV) is then measured to observe effects.

  • Quasi-experimental Designs:     * The researcher manipulates the Independent Variable.     * Crucially, there is no random assignment. Research often utilizes pre-existing groups.     * The Dependent Variable (DV) is then measured.

  • Experimental Research Defined: This involves manipulating one variable to look for an effect on another variable. Manipulate is defined as systematically varying how people experience the experiment.

Features of Quasi-Experimental Designs

  • Quasi Independent Variable:     * Involves pre-existing groups rather than groups created via random assignment.     * Can involve comparing a naturally occurring event vs. no naturally occurring event.     * Deals with a wide range of naturally occurring events or human characteristics.

  • Median Split: A method used in quasi-experiments to categorize participants based on whether they fall above or below the median score of a specific characteristic.

Independent Variable (IV) Levels

  • Definition of "Levels": This refers to the number of different ways participants will experience the experiment (the different conditions or groups within the IV).

  • Examples of IVs and Levels:     * IV = Temperature (The number of levels would depend on how many specific temperatures are being compared).     * IV = Mood (The number of levels depends on how many mood states are being induced or measured).     * IV = Amount of depression (Levels are determined by the categorization of depression severity).

Case Study: Eberhardt, Davies, Purdie-Vaughns, & Johnson (2006)

  • Data Source: The study analyzed 600600 death-eligible cases originating from Philadelphia between the years 19791979 and 19991999.

  • Methodology:     * Photographs were obtained of Black defendants.     * These photographs were presented to 4444 naive raters.     * The raters were Stanford undergraduates who were non-Black.     * Raters judged the "stereotypicality of features" on a scale ranging from 11 to 1111.

  • Application of IV Levels: The ratings served as the levels of the Independent Variable based on a median split (50%50\% cutoff).

Types and Considerations of Manipulations

  • Types of Manipulations:     * Environmental: Changing the physical setting or surroundings.     * Scenario: Presenting different hypothetical situations.     * Instructional: Giving different directions or information to different groups.     * Physiological: Directly inducing physical changes in the body. Examples include:         * Inducing pain.         * Requiring exercise.         * Inducing smoking behaviors.         * Inducing alcohol ingestion.

  • Considerations for Effective Manipulations:     * Effectiveness: Ensuring the manipulation will actually have the desired effect on participants.     * Strength: While stronger manipulations might show clearer results, they decrease external and ecological validity.     * Ecological Validity: A specific type of external validity that deals with whether the study findings can generalize to the real world.     * Manipulation Check: An extra step or measurement inserted into the study to verify that the manipulation worked as intended.