COM 205: Research Methods Test 2 Study Guide
Foundations of Experimental Research
Appropriate Research Topics for Experiments:
- Experimental research is best suited for studies investigating causal relationships (cause-and-effect) between clearly defined variables.
- Ideal for explanatory research rather than exploratory or purely descriptive inquiries.
- Particularly well adapted for micro-level social phenomena, such as small-group interactions, persuasion, attitude change, cognitive processing, and media effects.
- Effective when the researcher can manipulate the exposure to the independent variable () while isolating subjects from extraneous environmental variables.
Core Components of Experimental Design:
- Independent Variable () vs. Dependent Variable ():
- Independent Variable (): The variable that is manipulated, varied, or applied by the experimenter (the hypothesized cause or stimulus).
- Dependent Variable (): The variable that is measured to determine the effect of the independent variable (the hypothesized effect or outcome).
- Experimental Group vs. Control Group:
- Experimental Group: The group of subjects exposed to the experimental stimulus, treatment, or manipulation of the .
- Control Group: A group of subjects identical to the experimental group in all respects, except they are not exposed to the experimental stimulus (or they receive a placebo/neutral stimulus). Serves as a baseline for comparison.
- Pre-testing vs. Post-testing:
- Pre-testing: The baseline measurement of the dependent variable () across subjects prior to the introduction of the independent variable ().
- Post-testing: The measurement of the dependent variable () across subjects after the independent variable () has been introduced or manipulated.
- Independent Variable () vs. Dependent Variable ():
The Double-Blind Experiment:
- Definition: An experimental setup in which neither the human subjects nor the researchers who interact with them know which participants belong to the experimental group and which belong to the control group.
- Mitigation of Bias:
- Eliminates experimenter expectancy bias (where researchers unconsciously treat participants differently or interpret ambiguous behaviors based on expected outcomes).
- Mitigates demand characteristics and placebo effects (where participants alter their behaviors or symptoms because they know they are receiving an active treatment).
Random Assignment vs. Participant Matching:
- Random Assignment (Randomization):
- A procedure where every participant has an equal probability of being placed into any experimental condition or control group.
- Controls for both known and unknown confounding variables by distributing individual differences evenly across conditions by chance.
- Participant Matching:
- A technique where subjects are paired on one or more specific background characteristics (e.g., age, gender, prior knowledge, cognitive ability) that could confound the .
- One member of each matched pair is assigned to the experimental group and the other to the control group.
- Requires knowing in advance which variables are critical to match, leaving unmeasured variables potentially unbalanced.
- Random Assignment (Randomization):
Experimental Designs and Methodological Variations
Pre-Experimental Designs:
- Characterized by a lack of random assignment and/or a lack of a comparison control group, resulting in weak internal validity.
- One-Shot Case Study:
- Structure: A single group is exposed to a treatment () and then measured on the dependent variable ().
- Notation:
- Limitation: Lacks a pre-test and a control group, making it impossible to determine if the post-test score resulted from the treatment.
- One-Group Pretest-Posttest Design:
- Structure: A single group is measured on the (), exposed to the treatment (), and then re-measured ().
- Notation:
- Limitation: Lacks a control group; differences between pre-test and post-test may be caused by maturation, history, or testing effects rather than .
- Static-Group Comparison:
- Structure: Includes an experimental group exposed to and measured (), and a comparison group that does not receive and is measured ().
- Notation:
- Experimental:
- Control:
- Limitation: Groups are not formed via random assignment, leading to selection bias.
True Experimental Designs:
- Characterized by both random assignment () of subjects to conditions and the presence of a control or comparison group.
- Pretest-Posttest Control Group Design (Classic Experiment):
- Structure: Participants are randomly assigned to either the experimental or control group. Both groups complete a pre-test, the experimental group receives the treatment (), and both complete a post-test.
- Notation:
- Experimental:
- Control:
- Controls for most threats to internal validity, but vulnerable to testing-treatment interaction.
- Posttest-Only Control Group Design:
- Structure: Participants are randomly assigned to either the experimental or control group, the experimental group receives , and both groups are measured only after treatment.
- Notation:
- Experimental:
- Control:
- Prevents testing sensitization effects while maintaining high internal validity due to randomization.
- Solomon Four-Group Design:
- Structure: Combines the Pretest-Posttest Control Group design with the Posttest-Only Control Group design using four randomly assigned groups:
- Group 1:
- Group 2:
- Group 3:
- Group 4:
- Explicitly isolates and evaluates the main effect of the treatment, the main effect of pre-testing, and the interaction effect between the pre-test and the treatment.
- Structure: Combines the Pretest-Posttest Control Group design with the Posttest-Only Control Group design using four randomly assigned groups:
Environmental Variations of Experimentation:
- Laboratory Experiments:
- Conducted in highly controlled, artificial settings.
- High degree of internal validity due to strict control over extraneous variables; lower ecological and external validity due to artificiality.
- Field Experiments:
- Conducted in naturalistic, real-world settings where the researcher actively manipulates the independent variable.
- Higher external validity; reduced control over extraneous variables and ethical constraints regarding informed consent.
- Natural Experiments:
- Observational studies where manipulations occur naturally due to environmental events, policy shifts, or disasters without direct researcher intervention.
- High ecological validity, but the researcher cannot randomly assign subjects or strictly control confounding factors.
- Laboratory Experiments:
Strengths and Weaknesses of Experimental Research:
- Strengths:
- Optimal design for establishing causal relationships (time precedence, association, non-spuriousness).
- High level of internal validity and control over variables.
- High replicability due to standardized procedures.
- Weaknesses:
- Artificiality of laboratory settings, which can inhibit generalizability.
- Vulnerability to experimenter bias and subject reactivity (e.g., Hawthorne effect).
- Ethical and practical limitations regarding which variables can be manipulated in human populations.
- Strengths:
Validity in Experimental Research
Internal Validity:
- Definition: The degree of certainty that observed changes in the dependent variable () are directly caused by the independent variable () rather than confounding factors.
- Threats to Internal Validity:
- History: Unplanned, external events that occur between the pre-test and post-test that influence the dependent variable.
- Maturation: Biological or psychological changes occurring within participants as a natural function of time (e.g., growing older, becoming fatigued, bored, or hungry).
- Testing (Sensitization): The process of taking a pre-test enhances performance or sensitizes participants, altering their scores on the subsequent post-test independent of the treatment.
- Instrumentation: Changes or inconsistencies in the calibration of measuring instruments, observational criteria, or grading rubrics between pre-test and post-test.
- Statistical Regression (Regression to the Mean): The tendency of participants selected on the basis of extreme (very high or very low) pre-test scores to score closer to the average on subsequent testing.
- Selection Bias: Non-equivalent groups formed through non-random assignment, leading to baseline differences that explain post-test outcomes.
- Experimental Mortality (Attrition): Differential, non-random dropout of participants across groups over the course of the study.
- Causal Time-Order: Ambiguity or lack of clarity regarding whether the independent variable preceded the dependent variable in time.
- Diffusion or Imitation of Treatments: The control group inadvertently learns about or accesses the experimental stimulus from experimental group participants.
- Compensatory Rivalry (John Henry Effect): Control group subjects work unusually hard to overcome the perceived disadvantage of not receiving the treatment.
- Demoralization: Control group subjects experience feelings of deprivation or resentment, reducing their effort or performance on the post-test.
External Validity:
- Definition: The extent to which experimental findings can be generalized across diverse populations, settings, treatment variables, and measurement instruments.
- Limiting Factors: Artificial laboratory conditions, reactive measurement, unrepresentative volunteer participant pools (e.g., college student samples), and pre-test sensitization.
Foundations of Content Analysis
Content Analysis as an Unobtrusive Research Method:
- Unobtrusive Research: Research methodologies that study social behavior without interacting directly with human subjects.
- Eliminates reactive measurement effects (such as the Hawthorne effect or social desirability bias) because the creators of the texts are unaware their artifacts are being studied.
Scope and Topics for Content Analysis:
- Addresses the foundational communication question: "Who says what, to whom, why, to what extent, and with what effect?"
- Suitable for analyzing recorded human communication, including books, news articles, television broadcasts, advertisements, political speeches, social media posts, song lyrics, and legal documents.
- Well-suited for longitudinal studies examining cultural shifts or historical trends in representation over extended timeframes.
Manifest Content vs. Latent Content:
- Manifest Content:
- The concrete, observable surface content directly visible in the text.
- Involves counting explicit words, visual elements, or quantifiable characteristics (e.g., number of times a female protagonist appears, frequency of specific profanities).
- Yields high coding reliability; may lack depth regarding underlying semantic context.
- Latent Content:
- The underlying, implicit, deeper thematic meaning conveyed by a message.
- Requires interpretive reading of tone, subtext, ideology, or symbolic meaning (e.g., evaluating whether an article is overall "optimistic" or "cynical").
- Provides high conceptual validity, but suffers from lower inter-coder reliability due to subjective coder interpretations.
- Manifest Content:
Conducting Content Analysis: Stages and Methodological Protocols
Stages of Content Analysis:
- 1. Formulating the Research Question or Hypotheses: Clarifying the specific concepts, representations, or trends under study.
- 2. Defining the Population (Universe): Identifying the exact boundaries of the textual universe to be investigated (e.g., front-page headlines of the New York Times between 2010 and 2020).
- 3. Sampling: Selecting a representative sample from the defined population using probability sampling methods (e.g., random sampling, stratified sampling, or constructed week sampling).
- 4. Establishing Units of Analysis:
- The discrete physical or symbolic element that is analyzed and coded.
- Can be individual words, sentences, paragraphs, entire articles, visual scenes, or whole media episodes.
- Unlike survey research where the unit of analysis is almost always an individual person, content analysis defines units based on textual segments.
- 5. Operationalizing Variables and Constructing a Coding Scheme:
- Coding Scheme / Codebook: A formalized, exhaustive set of instructions, rules, operational definitions, and category boundaries guiding coders on how to score text.
- Categories must be mutually exclusive (each unit fits into only one category per variable) and exhaustive (a category exists for every unit).
- 6. Coder Training and Pilot Testing (Inter-Coder Reliability):
- Multiple independent coders apply the codebook to a shared sub-sample of texts.
- Statistical metrics (e.g., percent agreement, Scott's Pi, Cohen's Kappa, or Krippendorff's Alpha) are calculated to confirm that different coders categorize the material consistently.
- 7. Executing Coding: Independent coding of the entire textual sample.
- 8. Analyzing Data: Statistical analysis of frequencies, percentages, cross-tabulations, or longitudinal patterns.
A-Priori Coding vs. Emergent Coding:
- A-Priori (Deductive) Coding:
- Categories and operational codes are established prior to data analysis, derived directly from preexisting theories, conceptual frameworks, or literature.
- Coders apply these predetermined rules strictly to the empirical text.
- Emergent (Inductive) Coding:
- Categories are established during a preliminary examination of the empirical data.
- The researcher reviews a subset of texts, identifies recurring motifs, and iteratively refines coding categories before final deployment.
- A-Priori (Deductive) Coding:
Strengths and Weaknesses of Content Analysis:
- Strengths:
- Unobtrusive and completely non-reactive.
- Cost-effective compared to large-scale surveys or clinical laboratory trials.
- Enables longitudinal analyses covering decades or centuries of recorded history.
- High safety margin: coding errors can be corrected by re-analyzing the original preserved texts.
- Weaknesses:
- Restricted purely to recorded communications; cannot study unrecorded human thoughts or behaviors.
- Cannot directly establish causal claims regarding audience reception or real-world behavioral impact.
- Subjective assessments of latent content can degrade inter-coder reliability.
- Strengths:
Univariate Quantitative Data Analysis
Definition of Univariate Analysis:
- The statistical analysis of a single variable at a time, evaluating the distribution of its attributes independently of other variables.
- Purpose: To organize, summarize, and describe the characteristics of a sample or population on that specific measure.
Descriptive Statistics:
- Techniques used to organize, summarize, and visualize data collected from a sample without drawing conclusions beyond the observed sample.
- Two Primary Descriptive Functions:
- Central Tendency: Identifies the single most typical, representative, or central score in a data distribution.
- Dispersion: Identifies the extent of spread, variability, or heterogeneity of scores around the central value.
Measures of Central Tendency and Measurement Levels:
- Mode ():
- The most frequently occurring attribute or value in a distribution.
- Only measure of central tendency appropriate for Nominal data (e.g., religious affiliation, biological sex, political party).
- Can also be reported for Ordinal, Interval, and Ratio data.
- Median ():
- The middle value in an ordered distribution of scores, dividing the data into two equal halves (50th percentile).
- Requires
- Mode ():