Notes on Research Methods in Social Science

Experiments

• What is an experiment?
  • A research design that manipulates one variable (the independent variable) to observe its effect on another variable (the dependent variable), while keeping all other factors constant (the control condition).
  • Purpose: to determine cause and effect. Experiments are considered our best method for inferring causality because they isolate a single changing factor.
• Key terms from the transcript
  • Manipulation: deliberately changing one aspect of a participant’s experience to see its effect on another outcome.
  • Keeping everything else constant (ceteris paribus): all other potential influences are held steady so they cannot explain any observed change.
  • Example from the transcript:
  • Two groups: one asked to scroll social media while listening to a story, the other not allowed to use social media.
  • Outcome measured: attention/listening to the story via four questions.
• The Shell Silverstein demo (story as an experimental stimulus)
  • Story used: "Smart" by Shel Silverstein (narrated while some participants could scroll on social media).
  • Outcome: four comprehension/listening questions; performance used to infer attention differences between groups.
  • Values used in the story (illustrative math from the tale):
  • Starting amount: $A_0 = 1.00$ (one dollar)
  • Swap for two quarters: $A_1 = 2 imes 0.25 = 0.50$
  • Trade for three dimes: $A_2 = 3 imes 0.10 = 0.30$
  • Trade for four nickels: $A_3 = 4 imes 0.05 = 0.20$
  • Trade for five pennies: $A_4 = 5 imes 0.01 = 0.05$
  • Resulting sequence: $A<em>0 = 1.00,\, A</em>1 = 0.50,\, A<em>2 = 0.30,\, A</em>3 = 0.20,\, A_4 = 0.05$
  • Takeaway: the manipulation (social media use) is tied to differences in attention, though the results were described as tentative/undetermined.
• Why experiments are valuable
  • They are the best way to infer cause and effect because they attempt to control for other variables.
• Downsides of experiments (as discussed in the transcript)
  • Can feel contrived or artificial; not always naturalistic.
  • Real-world behavior might differ from lab-like setups.
  • If you rely on self-report after an experimental manipulation, you still inherit self-report biases.
• Important caveat about self-report data (tying into experiments)
  • Even in experiments, participants may lie, misremember, or respond in socially desirable ways.
• Quick recap to memorize
  • Manipulation = change one thing, measure its effect on another.
  • Control for other variables to claim causality.
  • Experiments ≈ best (not perfect) for causal inference; trade-off: sometimes artificial settings.

Self-Report Methods

• Two self-report approaches: surveys and interviews.
• Surveys
  • Define: structured data collection where respondents answer fixed questions.
  • Data type: typically quantitative (numbers, scales) but can include qualitative open-ended items in some surveys.
  • Examples from the transcript:
  • Hours per day on social media (numerical, quantitative).
  • Rating impact of social media on life on a 1–10 scale (quantitative, numbers).
  • Key point: marginal issues include lying, memory errors, nonresponse bias, and sampling biases.
  • Noted advantage: easy to administer to large samples; relatively efficient for getting broad trends.
• Interviews
  • Define: qualitative data collection via open-ended, in-depth questions.
  • Data type: qualitative (words, descriptions, stories).
  • Purpose: to gain in-depth understanding of complex experiences that numbers alone cannot capture.
  • Example questions from the transcript:
  • Tell me about a time when social media had a positive influence on your life.
  • Strengths: rich, contextual insights; flexible questioning.
  • Trade-offs: time-consuming; more demanding in analysis; potential interviewer influence.
• Key distinction between surveys and interviews
  • Surveys quantify experiences with fixed choices or scales.
  • Interviews explore processes, meanings, and personal narratives through open-ended questions.
• When to use self-report methods
  • To capture subjective experiences, perceptions, or attitudes.
  • When depth (interviews) or breadth (surveys) is desired.
• Downsides of self-report methods
  • Social desirability bias, recall bias, and willingness to participate.
  • Not always objective; results depend on respondent honesty and memory.

Textual Analysis

• What it is
  • Analyzing existing texts or artifacts (not generated via new self-report) to extract information.
  • Not self-report data; uses previously produced materials.
• Examples in the transcript
  • Advertising texts and social media ads: analyze how ads address problems, create relatability, and offer solutions.
  • Textual analysis might use social media posts, advertisements, or other artifacts as data sources.
• Data type and implications
  • Primarily qualitative; patterns, themes, and discourse are identified from texts.
  • Useful for understanding media messaging, representation, and cultural trends.
• Pros and cons
  • Pros: unobtrusive; avoids respondent bias; leverages existing content.
  • Cons: interpretation can be subjective; may require clear coding schemes to ensure reliability.
• Why use textual analysis in advertising research
  • It reveals how ads are crafted to be relatable and persuasive.
  • Helps identify target demographics, messaging strategies, and embedded values.

Ethnography

• What it is
  • An observational, often immersive method that studies people in their natural environments.
  • Emphasizes real-time behavior rather than self-reported data.
• How it differs from textual analysis
  • Textual analysis looks at existing texts; ethnography directly observes behavior and interactions.
• Examples from the transcript
  • Observing whether people quietly scroll on their phones in public settings or discuss what they see on their screens.
  • Seeing how people interact with devices in social spaces to determine the impact on social interaction.
• Data type
  • Qualitative, observational data (descriptions of behavior, interactions, contexts).
• Downsides and challenges
  • Observer effect: people may change behavior when being watched (or that change might be subtle or invisible).
  • Misinterpretation: researchers may misread interactions or intentions.
  • Logistical difficulties: time, access, and consistency of observation can be challenging.
• Strengths
  • Real-world behavior; less reliant on self-report.
• Practical implications
  • In ethnographic work, researchers can provide nuanced descriptions of how technology shapes daily life and social interactions.

Mixed Methods and Triangulation

• Mixed method studies
  • Combine qualitative and quantitative data in a single study (e.g., survey plus interview, survey plus textual analysis, etc.).
  • Rationale: leverage strengths of multiple approaches and offset weaknesses of any single method.
• Triangulation
  • Using multiple methods to examine the same research question to gain more confidence in findings.
  • If different methods converge on similar conclusions, researchers gain higher confidence in the results.
• Takeaway from the transcript
  • There is no perfect single method; triangulation helps mitigate individual method weaknesses.
• Practical applications
  • In real-world research, teams often use multiple methods to build a more robust evidence base.

The Big Picture: What Each Method Offers (Pros/Cons Snapshot)

• Self-report methods (surveys, interviews)
  • Pros: direct access to people’s thoughts, experiences, and perceptions; scalable (surveys).
  • Cons: memory and social desirability biases; subject to respondent interpretation.
• Observational methods (ethnography, textual analysis)
  • Pros: reduces reliance on memory and self-report; captures natural behavior or existing artifacts.
  • Cons: interpretation challenges; potential observer bias; ethical/logistical constraints.
• Mixed methods/triangulation
  • Pros: complementary strengths; greater confidence if results align.
  • Cons: more complex design and analysis; time-consuming.

Critical thinking and real-world relevance

• How to analyze research critically
  • Recognize that surveys often include a margin of error and potential sampling biases.
  • Understand that social media polls or political polls are not always perfectly representative.
  • Consider how the method used shapes what is known about a phenomenon.
• Real-world examples mentioned
  • UK Health Care Marketing patient satisfaction surveys used to drive changes within an organization.
  • Social media trend analysis as a tool for understanding trends in marketing or public perception.
• Ethical and philosophical notes
  • No single method can perfectly capture reality; researchers must acknowledge biases and limitations.
  • Mixed methods and triangulation encourage humility and methodological pluralism.
  • Critical evaluation of data sources (e.g., political polls) is essential, especially during election seasons.

Quick math and concepts from the transcript (LaTeX-format examples)

• Quantitative values in the Shell Silverstein example:
  • Starting amount and subsequent exchanges:
  • $A_0 = 1.00$
  • $A_1 = 2 imes 0.25 = 0.50$
  • $A_2 = 3 imes 0.10 = 0.30$
  • $A_3 = 4 imes 0.05 = 0.20$
  • $A_4 = 5 imes 0.01 = 0.05$
  • Resulting sequence: $A<em>0 = 1.00,\, A</em>1 = 0.50,\, A<em>2 = 0.30,\, A</em>3 = 0.20,\, A_4 = 0.05$
• Simple causal relation (conceptual model, not a mathematical result from the transcript but aligned with experimental logic):
  • Let
  • $X ext{ = social media use (1 = scroll, 0 = no scroll)}$
  • $Y ext{ = attention score (higher means better listening)}$
  • Then a simple linear model to illustrate causality could be written as:
  • $Y = \beta<em>0 + \beta</em>1 X + \epsilon$
  • If $\beta_1 \neq 0$ in an experiment where X is manipulated and confounds are controlled, this supports a causal effect of X on Y.
• Key takeaways to remember in exam-style questions
  • The five methods: no single method is perfect; each has strengths, weaknesses, and appropriate contexts.
  • Experimental manipulation aims to establish cause and effect but may lack naturalism.
  • Self-report methods (surveys, interviews) provide direct access to perceptions but are prone to bias.
  • Textual analysis and ethnography rely on existing artifacts or natural behavior, offering depth but requiring careful interpretation.
  • Mixed methods and triangulation increase confidence by cross-validating findings across different data sources.

Summary takeaways for exam prep

• Experiments are best for causal inference but can be contrived; manipulation of one variable and keeping others constant is the core idea.
• Self-report methods (surveys and interviews) provide access to perceptions and experiences, with surveys being quantitative and interviews qualitative.
• Textual analysis and ethnography are non-self-report methods; they analyze existing texts or observable behavior in natural settings.
• Triangulation and mixed methods strengthen research by combining strengths and compensating for weaknesses.
• Always consider margin of error, bias, and the context when evaluating data from polls or studies.
• Real-world applications include marketing analytics, health care feedback systems, and social media trend analysis, highlighting the practical relevance of understanding research methods.