COM 205: Research Methods Test 2 Study Guide

Foundations of Experimental Research

  • Appropriate Research Topics for Experiments:

    • Experimental research is best suited for studies investigating causal relationships (cause-and-effect) between clearly defined variables.
    • Ideal for explanatory research rather than exploratory or purely descriptive inquiries.
    • Particularly well adapted for micro-level social phenomena, such as small-group interactions, persuasion, attitude change, cognitive processing, and media effects.
    • Effective when the researcher can manipulate the exposure to the independent variable (IVIV) while isolating subjects from extraneous environmental variables.
  • Core Components of Experimental Design:

    • Independent Variable (IVIV) vs. Dependent Variable (DVDV):
      • Independent Variable (IVIV): The variable that is manipulated, varied, or applied by the experimenter (the hypothesized cause or stimulus).
      • Dependent Variable (DVDV): The variable that is measured to determine the effect of the independent variable (the hypothesized effect or outcome).
    • Experimental Group vs. Control Group:
      • Experimental Group: The group of subjects exposed to the experimental stimulus, treatment, or manipulation of the IVIV.
      • Control Group: A group of subjects identical to the experimental group in all respects, except they are not exposed to the experimental stimulus (or they receive a placebo/neutral stimulus). Serves as a baseline for comparison.
    • Pre-testing vs. Post-testing:
      • Pre-testing: The baseline measurement of the dependent variable (DVDV) across subjects prior to the introduction of the independent variable (IVIV).
      • Post-testing: The measurement of the dependent variable (DVDV) across subjects after the independent variable (IVIV) has been introduced or manipulated.
  • The Double-Blind Experiment:

    • Definition: An experimental setup in which neither the human subjects nor the researchers who interact with them know which participants belong to the experimental group and which belong to the control group.
    • Mitigation of Bias:
      • Eliminates experimenter expectancy bias (where researchers unconsciously treat participants differently or interpret ambiguous behaviors based on expected outcomes).
      • Mitigates demand characteristics and placebo effects (where participants alter their behaviors or symptoms because they know they are receiving an active treatment).
  • Random Assignment vs. Participant Matching:

    • Random Assignment (Randomization):
      • A procedure where every participant has an equal probability of being placed into any experimental condition or control group.
      • Controls for both known and unknown confounding variables by distributing individual differences evenly across conditions by chance.
    • Participant Matching:
      • A technique where subjects are paired on one or more specific background characteristics (e.g., age, gender, prior knowledge, cognitive ability) that could confound the DVDV.
      • One member of each matched pair is assigned to the experimental group and the other to the control group.
      • Requires knowing in advance which variables are critical to match, leaving unmeasured variables potentially unbalanced.

Experimental Designs and Methodological Variations

  • Pre-Experimental Designs:

    • Characterized by a lack of random assignment and/or a lack of a comparison control group, resulting in weak internal validity.
    • One-Shot Case Study:
      • Structure: A single group is exposed to a treatment (XX) and then measured on the dependent variable (OO).
      • Notation: X→OX \rightarrow O
      • Limitation: Lacks a pre-test and a control group, making it impossible to determine if the post-test score resulted from the treatment.
    • One-Group Pretest-Posttest Design:
      • Structure: A single group is measured on the DVDV (O1O_1), exposed to the treatment (XX), and then re-measured (O2O_2).
      • Notation: O1→X→O2O_1 \rightarrow X \rightarrow O_2
      • Limitation: Lacks a control group; differences between pre-test and post-test may be caused by maturation, history, or testing effects rather than XX.
    • Static-Group Comparison:
      • Structure: Includes an experimental group exposed to XX and measured (O1O_1), and a comparison group that does not receive XX and is measured (O2O_2).
      • Notation:
        • Experimental: X→O1X \rightarrow O_1
        • Control: O2O_2
      • Limitation: Groups are not formed via random assignment, leading to selection bias.
  • True Experimental Designs:

    • Characterized by both random assignment (RR) of subjects to conditions and the presence of a control or comparison group.
    • Pretest-Posttest Control Group Design (Classic Experiment):
      • Structure: Participants are randomly assigned to either the experimental or control group. Both groups complete a pre-test, the experimental group receives the treatment (XX), and both complete a post-test.
      • Notation:
        • Experimental: R→O1→X→O2R \rightarrow O_1 \rightarrow X \rightarrow O_2
        • Control: R→O3→O4R \rightarrow O_3 \rightarrow O_4
      • Controls for most threats to internal validity, but vulnerable to testing-treatment interaction.
    • Posttest-Only Control Group Design:
      • Structure: Participants are randomly assigned to either the experimental or control group, the experimental group receives XX, and both groups are measured only after treatment.
      • Notation:
        • Experimental: R→X→O1R \rightarrow X \rightarrow O_1
        • Control: R→O2R \rightarrow O_2
      • Prevents testing sensitization effects while maintaining high internal validity due to randomization.
    • Solomon Four-Group Design:
      • Structure: Combines the Pretest-Posttest Control Group design with the Posttest-Only Control Group design using four randomly assigned groups:
        • Group 1: R→O1→X→O2R \rightarrow O_1 \rightarrow X \rightarrow O_2
        • Group 2: R→O3→O4R \rightarrow O_3 \rightarrow O_4
        • Group 3: R→X→O5R \rightarrow X \rightarrow O_5
        • Group 4: R→O6R \rightarrow O_6
      • Explicitly isolates and evaluates the main effect of the treatment, the main effect of pre-testing, and the interaction effect between the pre-test and the treatment.
  • Environmental Variations of Experimentation:

    • Laboratory Experiments:
      • Conducted in highly controlled, artificial settings.
      • High degree of internal validity due to strict control over extraneous variables; lower ecological and external validity due to artificiality.
    • Field Experiments:
      • Conducted in naturalistic, real-world settings where the researcher actively manipulates the independent variable.
      • Higher external validity; reduced control over extraneous variables and ethical constraints regarding informed consent.
    • Natural Experiments:
      • Observational studies where manipulations occur naturally due to environmental events, policy shifts, or disasters without direct researcher intervention.
      • High ecological validity, but the researcher cannot randomly assign subjects or strictly control confounding factors.
  • Strengths and Weaknesses of Experimental Research:

    • Strengths:
      • Optimal design for establishing causal relationships (time precedence, association, non-spuriousness).
      • High level of internal validity and control over variables.
      • High replicability due to standardized procedures.
    • Weaknesses:
      • Artificiality of laboratory settings, which can inhibit generalizability.
      • Vulnerability to experimenter bias and subject reactivity (e.g., Hawthorne effect).
      • Ethical and practical limitations regarding which variables can be manipulated in human populations.

Validity in Experimental Research

  • Internal Validity:

    • Definition: The degree of certainty that observed changes in the dependent variable (DVDV) are directly caused by the independent variable (IVIV) rather than confounding factors.
    • Threats to Internal Validity:
      • History: Unplanned, external events that occur between the pre-test and post-test that influence the dependent variable.
      • Maturation: Biological or psychological changes occurring within participants as a natural function of time (e.g., growing older, becoming fatigued, bored, or hungry).
      • Testing (Sensitization): The process of taking a pre-test enhances performance or sensitizes participants, altering their scores on the subsequent post-test independent of the treatment.
      • Instrumentation: Changes or inconsistencies in the calibration of measuring instruments, observational criteria, or grading rubrics between pre-test and post-test.
      • Statistical Regression (Regression to the Mean): The tendency of participants selected on the basis of extreme (very high or very low) pre-test scores to score closer to the average on subsequent testing.
      • Selection Bias: Non-equivalent groups formed through non-random assignment, leading to baseline differences that explain post-test outcomes.
      • Experimental Mortality (Attrition): Differential, non-random dropout of participants across groups over the course of the study.
      • Causal Time-Order: Ambiguity or lack of clarity regarding whether the independent variable preceded the dependent variable in time.
      • Diffusion or Imitation of Treatments: The control group inadvertently learns about or accesses the experimental stimulus from experimental group participants.
      • Compensatory Rivalry (John Henry Effect): Control group subjects work unusually hard to overcome the perceived disadvantage of not receiving the treatment.
      • Demoralization: Control group subjects experience feelings of deprivation or resentment, reducing their effort or performance on the post-test.
  • External Validity:

    • Definition: The extent to which experimental findings can be generalized across diverse populations, settings, treatment variables, and measurement instruments.
    • Limiting Factors: Artificial laboratory conditions, reactive measurement, unrepresentative volunteer participant pools (e.g., college student samples), and pre-test sensitization.

Foundations of Content Analysis

  • Content Analysis as an Unobtrusive Research Method:

    • Unobtrusive Research: Research methodologies that study social behavior without interacting directly with human subjects.
    • Eliminates reactive measurement effects (such as the Hawthorne effect or social desirability bias) because the creators of the texts are unaware their artifacts are being studied.
  • Scope and Topics for Content Analysis:

    • Addresses the foundational communication question: "Who says what, to whom, why, to what extent, and with what effect?"
    • Suitable for analyzing recorded human communication, including books, news articles, television broadcasts, advertisements, political speeches, social media posts, song lyrics, and legal documents.
    • Well-suited for longitudinal studies examining cultural shifts or historical trends in representation over extended timeframes.
  • Manifest Content vs. Latent Content:

    • Manifest Content:
      • The concrete, observable surface content directly visible in the text.
      • Involves counting explicit words, visual elements, or quantifiable characteristics (e.g., number of times a female protagonist appears, frequency of specific profanities).
      • Yields high coding reliability; may lack depth regarding underlying semantic context.
    • Latent Content:
      • The underlying, implicit, deeper thematic meaning conveyed by a message.
      • Requires interpretive reading of tone, subtext, ideology, or symbolic meaning (e.g., evaluating whether an article is overall "optimistic" or "cynical").
      • Provides high conceptual validity, but suffers from lower inter-coder reliability due to subjective coder interpretations.

Conducting Content Analysis: Stages and Methodological Protocols

  • Stages of Content Analysis:

    • 1. Formulating the Research Question or Hypotheses: Clarifying the specific concepts, representations, or trends under study.
    • 2. Defining the Population (Universe): Identifying the exact boundaries of the textual universe to be investigated (e.g., front-page headlines of the New York Times between 2010 and 2020).
    • 3. Sampling: Selecting a representative sample from the defined population using probability sampling methods (e.g., random sampling, stratified sampling, or constructed week sampling).
    • 4. Establishing Units of Analysis:
      • The discrete physical or symbolic element that is analyzed and coded.
      • Can be individual words, sentences, paragraphs, entire articles, visual scenes, or whole media episodes.
      • Unlike survey research where the unit of analysis is almost always an individual person, content analysis defines units based on textual segments.
    • 5. Operationalizing Variables and Constructing a Coding Scheme:
      • Coding Scheme / Codebook: A formalized, exhaustive set of instructions, rules, operational definitions, and category boundaries guiding coders on how to score text.
      • Categories must be mutually exclusive (each unit fits into only one category per variable) and exhaustive (a category exists for every unit).
    • 6. Coder Training and Pilot Testing (Inter-Coder Reliability):
      • Multiple independent coders apply the codebook to a shared sub-sample of texts.
      • Statistical metrics (e.g., percent agreement, Scott's Pi, Cohen's Kappa, or Krippendorff's Alpha) are calculated to confirm that different coders categorize the material consistently.
    • 7. Executing Coding: Independent coding of the entire textual sample.
    • 8. Analyzing Data: Statistical analysis of frequencies, percentages, cross-tabulations, or longitudinal patterns.
  • A-Priori Coding vs. Emergent Coding:

    • A-Priori (Deductive) Coding:
      • Categories and operational codes are established prior to data analysis, derived directly from preexisting theories, conceptual frameworks, or literature.
      • Coders apply these predetermined rules strictly to the empirical text.
    • Emergent (Inductive) Coding:
      • Categories are established during a preliminary examination of the empirical data.
      • The researcher reviews a subset of texts, identifies recurring motifs, and iteratively refines coding categories before final deployment.
  • Strengths and Weaknesses of Content Analysis:

    • Strengths:
      • Unobtrusive and completely non-reactive.
      • Cost-effective compared to large-scale surveys or clinical laboratory trials.
      • Enables longitudinal analyses covering decades or centuries of recorded history.
      • High safety margin: coding errors can be corrected by re-analyzing the original preserved texts.
    • Weaknesses:
      • Restricted purely to recorded communications; cannot study unrecorded human thoughts or behaviors.
      • Cannot directly establish causal claims regarding audience reception or real-world behavioral impact.
      • Subjective assessments of latent content can degrade inter-coder reliability.

Univariate Quantitative Data Analysis

  • Definition of Univariate Analysis:

    • The statistical analysis of a single variable at a time, evaluating the distribution of its attributes independently of other variables.
    • Purpose: To organize, summarize, and describe the characteristics of a sample or population on that specific measure.
  • Descriptive Statistics:

    • Techniques used to organize, summarize, and visualize data collected from a sample without drawing conclusions beyond the observed sample.
    • Two Primary Descriptive Functions:
      • Central Tendency: Identifies the single most typical, representative, or central score in a data distribution.
      • Dispersion: Identifies the extent of spread, variability, or heterogeneity of scores around the central value.
  • Measures of Central Tendency and Measurement Levels:

    • Mode (MoMo):
      • The most frequently occurring attribute or value in a distribution.
      • Only measure of central tendency appropriate for Nominal data (e.g., religious affiliation, biological sex, political party).
      • Can also be reported for Ordinal, Interval, and Ratio data.
    • Median (MdnMdn):
      • The middle value in an ordered distribution of scores, dividing the data into two equal halves (50th percentile).
      • Requires