Social Science Research: Manipulations and Quasi-Manipulations Study Guide
Core Concepts of Reliability and Validity
Reliability vs. Validity Overview: * Reliability: Defined by the concept of consistency. It refers to the extent to which a measure produces consistent results across repeated applications or over time. * Validity: Defined by the concept of accuracy. It refers to the correctness of the information or whether the tool measures what it is intended to measure.
The Interdependence of Reliability and Validity: * Reliability is considered a prerequisite for validity. * It is possible to have a measure that is reliable but not valid. * Example: Chris's Scale. If a scale (Chris's scale) consistently weighs someone at the same weight every day (reliability) but that weight is actually 10 pounds off the actual weight, the scale is reliable but invalid. * Conversely, you cannot have a valid measure unless it is also reliable. A tool that provides inconsistent results cannot be accurate.
Understanding Validity in Social Science Research
Definition and Correctness: * Validity concerns the accuracy and correctness of information. * The transcript provides a comparative example to illustrate validity: * Statement A: "Dr. Poepsel is feet tall." * Statement B: "Dr. Poepsel is feet, inch tall." * Statement B is the valid statement.
Concrete vs. Abstract Variables: * Establishing validity is relatively straightforward when dealing with concrete variables (like height). * In psychology, establishing validity is significantly more difficult because research often involves constructs (abstract concepts).
Challenges in Psychological Measurement: * High-level questions regarding validity include: * "How can we validly measure altruism?" * "How can we validly predict whether providing help to others increases one’s own well-being?"
Levels of Validity Consideration: * Validity must be addressed at both the measurement level (the tools used) and the study level (the overall research design).
Internal Validity and Causality
Internal Validity Definition: * This refers to the ability of the research to make stringent inferences of causality. * It ensures there is an accurate relationship between the Independent Variable (IV) and the Dependent Variable (DV). * It confirms that the IV is the actual cause of the observed changes in the DV.
Requirements of Causality: To establish that one variable causes another, three criteria must be met: 1. Correlation: There must be a correlation between variable and variable . 2. Temporal Precedence: A change in one variable must precede the change in the other (e.g., must occur before ). 3. Rule out Confounds: The researcher must ensure that is the only factor in the study that could cause changes in .
Types of Research Designs
Experimental Designs: * The researcher manipulates the Independent Variable. * Includes random assignment of participants to groups. * The Dependent Variable (DV) is then measured to observe effects.
Quasi-experimental Designs: * The researcher manipulates the Independent Variable. * Crucially, there is no random assignment. Research often utilizes pre-existing groups. * The Dependent Variable (DV) is then measured.
Experimental Research Defined: This involves manipulating one variable to look for an effect on another variable. Manipulate is defined as systematically varying how people experience the experiment.
Features of Quasi-Experimental Designs
Quasi Independent Variable: * Involves pre-existing groups rather than groups created via random assignment. * Can involve comparing a naturally occurring event vs. no naturally occurring event. * Deals with a wide range of naturally occurring events or human characteristics.
Median Split: A method used in quasi-experiments to categorize participants based on whether they fall above or below the median score of a specific characteristic.
Independent Variable (IV) Levels
Definition of "Levels": This refers to the number of different ways participants will experience the experiment (the different conditions or groups within the IV).
Examples of IVs and Levels: * IV = Temperature (The number of levels would depend on how many specific temperatures are being compared). * IV = Mood (The number of levels depends on how many mood states are being induced or measured). * IV = Amount of depression (Levels are determined by the categorization of depression severity).
Case Study: Eberhardt, Davies, Purdie-Vaughns, & Johnson (2006)
Data Source: The study analyzed death-eligible cases originating from Philadelphia between the years and .
Methodology: * Photographs were obtained of Black defendants. * These photographs were presented to naive raters. * The raters were Stanford undergraduates who were non-Black. * Raters judged the "stereotypicality of features" on a scale ranging from to .
Application of IV Levels: The ratings served as the levels of the Independent Variable based on a median split ( cutoff).
Types and Considerations of Manipulations
Types of Manipulations: * Environmental: Changing the physical setting or surroundings. * Scenario: Presenting different hypothetical situations. * Instructional: Giving different directions or information to different groups. * Physiological: Directly inducing physical changes in the body. Examples include: * Inducing pain. * Requiring exercise. * Inducing smoking behaviors. * Inducing alcohol ingestion.
Considerations for Effective Manipulations: * Effectiveness: Ensuring the manipulation will actually have the desired effect on participants. * Strength: While stronger manipulations might show clearer results, they decrease external and ecological validity. * Ecological Validity: A specific type of external validity that deals with whether the study findings can generalize to the real world. * Manipulation Check: An extra step or measurement inserted into the study to verify that the manipulation worked as intended.