Research Methods Exam 2
🔍 Research Design & Validity
Q: What is the purpose of content analysis after using open-ended questions?
A: To systematically categorize and quantify responses so patterns/themes can be analyzed statistically.
Q: Why use an even number of options on a rating scale?
A: To avoid a neutral middle option and force participants to choose a side.
Q: What is socially desirable responding, and why is it a concern?
A: When participants respond in a way that makes them look good rather than being truthful; it biases data.
Q: How do researchers combat socially desirable responding?
A: Use anonymity, indirect questioning, or validated social desirability scales.
Q: What is a response set, and why is it problematic?
A: A consistent pattern of answering regardless of content (e.g., always choosing “Agree”); it reduces validity.
Q: How do researchers reduce response sets?
A: Reverse coding some items and varying item formats.
🔁 Reliability
Q: Which reliability type measures consistency over time?
A: Test-retest reliability.
Q: When is split-halves reliability a concern, and what’s better to use?
A: When items are not equivalent across halves; internal consistency (like Cronbach’s alpha) is a better option.
Q: When is inter-rater reliability (e.g., Cohen’s kappa) used?
A: In observational studies or coding qualitative data with multiple raters.
✅ Validity
Q: Comparing a new creativity test to the Divergent Association Task tests what kind of validity?
A: Criterion validity (specifically, concurrent validity).
Q: If a personality test only measures neuroticism and ignores other traits, what validity is lacking?
A: Content validity.
🧪 Sampling
Q: What is a sampling frame?
A: A list or database from which a sample is drawn.
Q: What issues arise with sampling frames?
A: Incompleteness, outdated data, or excluding certain populations.
Q: What do all probability sampling methods have in common?
A: Each member of the population has a known, non-zero chance of being selected.
Q: Types of probability sampling:
Random Sampling: Every individual has an equal chance.
Systematic Sampling: Select every nth person from a list.
Stratified Sampling: Divide population into subgroups (strata), then sample from each.
Cluster Sampling: Randomly select entire groups.
Multistage Sampling: Combine multiple sampling methods (e.g., cluster, then random).
Q: Probability vs Non-probability sampling?
A: Probability = random selection; Non-probability = convenience or judgmental. Non-probability is acceptable when generalizability isn’t essential.
Q: What type of sampling is used when researchers recruit until they hit their target numbers per group (e.g., 20 full-time students)?
A: Quota sampling (non-probability).
🧪 Experiments vs Quasi-Experiments
Q: What is the difference between an experiment and a quasi-experiment?
A: Experiments use random assignment; quasi-experiments do not.
Q: In a study where parents choose the program (new vs standard) and there’s a significant outcome difference, what condition for causality is violated?
A: Random assignment (internal validity compromised — could be selection bias).
🧪 Lab vs Field Experiments
Q: Define lab experiment.
A: Conducted in controlled settings to eliminate confounds.
Q: Define field experiment.
A: Conducted in real-world settings.
Q: Commonalities?
A: Both manipulate IV and measure DV.
Q: Differences?
A: Control vs realism; lab = more control, field = more generalizable.
Q: Strengths and limitations?
A: Lab: high internal validity, low external. Field: lower control, higher ecological validity.
🔄 Between vs Within Subjects
Q: What defines a between-subjects experiment?
A: Different participants in each condition/level of the IV.
Q: In a study on conformity with 2, 3, or 4 confederates:
IV: Number of confederates.
Levels: 2, 3, 4.
Why between-subjects? Each participant only experiences one level.
Why an experiment? IV is manipulated, DV is measured.
DV: Conformity (agreement with group opinion).
Q: Conditions for inferring causality in this scenario?
Covariance: DV changes with IV.
Temporal precedence: IV manipulated before DV measured.
Internal validity: If random assignment is used, threats are minimized.
🎭 Factorial Design Example
Q: Two IVs: food type & condiment.
IV1: Type of food (e.g., tofu, burger).
IV2: Condiment (e.g., ketchup, mustard).
Main Effects: Look at average across levels.
Interaction: Occurs if the effect of one IV depends on the level of the other.
🧠 Within-Subjects Design (Stroop Task)
Q: Why is the Stroop Task a within-subjects design?
A: All participants experience all conditions (e.g., congruent vs incongruent trials).
Q: Benefits of within-subjects design?
A: Fewer participants needed; controls for individual differences.
Q: Limitations of within-subjects?
A: Carryover effects, fatigue, practice.
Q: What are carryover effects?
A: Performance in one condition affects performance in the next.
🌀 Design Counterbalancing
Q: Incomplete vs Complete designs?
A:
Incomplete: Not all possible orders given to each participant.
Complete: All conditions presented to each participant in every possible order.
Q: Incomplete design examples:
Counterbalancing
Latin Square
Random start with rotation
Q: Complete design examples:
Reverse counterbalancing
Block randomization
🏠 Family Structure Study Example
Q: Why can’t the researchers say family structure causes stress levels?
A: No random assignment = no internal validity. Other variables could explain results (e.g., income, parenting style).