Research Methods Flashcards: Controlling Participant Variables

Introduction to Participant Variables and Individual Differences

• Participant variables are defined as personal characteristics that make one individual different from another. The term individual differences is used interchangeably to communicate these specific variations.
• Human participants are thinking, feeling, and behaving organisms capable of drastically influencing a measurement procedure. Therefore, accounting for these characteristics is a vital aspect of research design.
• Depending on the research context, a participant variable can function as an independent variable, a dependent variable, or an extraneous variable.
• Physical differences include variables such as gender, age, metabolism, hormones, musculature, coordination, height, and weight. These differences mean physical reactions to stimuli are not identical across individuals.
• Cognitive differences include style, strategies, intelligence, and memory, which cause individuals to process stimuli differently.
• Contextual and Social differences include personal histories, experiences, social standing, economic standing, familiarity with tasks, moods, motivations, attitudes, and personality traits (e.g., competitiveness, attentiveness, or reactivity).

How Participant Variables Influence External Validity

• External validity involves accurately generalizing the results of a study to the target population identified in the hypothesis.
• Selection Criteria: This is the operational definition of participants used to select them for a study. These criteria define characteristics such as gender, age, educational level, and physical requirements. These criteria also implicitly define the population represented by the sample.
• Representative Samples: A sample should act as a miniature version of the target population. If the sample excludes individuals with certain characteristics, it becomes biased and unrepresentative, providing a distorted view of how the population behaves.
• Limitations on Representativeness:   - Identifiability: Some populations cannot be fully identified. For example, research on alcoholism has historically focused on males because female alcoholics were less willing to identify themselves.   - Accessibility: Researchers often cannot contact all identifiable members. Samples are frequently limited to those living nearby, those with listed phone numbers (excluding the very wealthy and the very poor), or university students (specifically those in psychology courses).
• Impact of Homogeneous Samples: Using college students as a sample may limit external validity for behaviors like memory or cognitive processes because students are often better educated and have higher intelligence than the general adult population. In contrast, Milgram's electric-shock study sampled the general adult population because students might have been uniquely responsive to authority.

Sample Size and Volunteer Bias

• Notation for Sample Size:   - $N$ stands for the total number of participants in a study.   - $n$ stands for the number of participants observed in a single condition.   - Adding all $n$ values equals $N$.
• General Rule for Sample Size: "The more the merrier." A larger sample is more likely to include all relevant types of participants and provide an accurate representation of the population. laboratory experiments typically feature an $N$ between $50$ and $100$, with $n$ ranging from $15$ to $30$.
• Equal $n$ Levels: While not strictly required, it is best to have nearly equal numbers of participants per condition ($n$) to maintain a comparable level of confidence across conditions. Statistical procedures are often easier and more accurate with equal $n$.
• Volunteer Bias: This arises because samples contain only individuals willing to participate. Volunteers tend to have higher social status, higher intelligence, a greater need for approval, and are less authoritarian or conforming than non-volunteers. They are also more likely to participate if the topic is personally relevant or they expect positive evaluation.
• Subject Sophistication: This bias occurs when participants are knowledgeable about research due to previous experience, having studied the topic, or having learned about manipulations during the study. This makes them less naive and more susceptible to reactivity, demand characteristics, or diffusion of treatment. This also applies to animal participants that become more relaxed or experienced over time.

How Participant Variables Influence a Research Relationship

• Extraneous participant variables can create problems when participants fluctuate within conditions or differ between conditions.
• Reliability and Variability: If participants within the same condition differ (e.g., varying inherent memory abilities), they will respond differently to the same stimulus. This produces variability (error variance) and weakens the observed relationship.
• Confounding the Results: Differences between conditions can result from the "luck of the draw" in random assignment.   - Type I Error Example: Hypnosis appears to work only because, by chance, people with good memories were assigned to the hypnosis group.   - Type II Error Example: Hypnosis appears not to work because the experimental group happened to contain people with poor memories while the control group had people with good memories.
• Identifying Variables to Control: Researchers should look for characteristics substantially correlated with the independent or dependent variables.   - Concrete stimuli/physical responses: Control physiological (height, coordination) or psychological (cognitive ability, motivation) factors.   - Social behaviors/attitudes: Control personality or cultural differences.

Controlling Participant Variables in a Between-Subjects Design

• Definition: In a between-subjects design, a different group of participants is randomly selected for each condition of the independent variable. Participants are tested under only one level of the IV.
• Random Assignment: This is the primary defense, used to balance out participant variables by randomly mixing them across conditions. This ensures that unknown variables are likely balanced. However, it is not guaranteed, works poorly with small samples, and increases error variance within conditions.
• Balancing (Counterbalancing) a Variable: This involves systematically ensuring a specific variable is represented equally in all conditions.   - Step 1: Pretest participants to measure the variable (e.g., strength, reading ability, anxiety).   - Step 2: Create separate subject pools (e.g., male/female, high/low memory).   - Step 3: Assign participants so each pool is balanced in every condition.
• Collapsing Across a Variable: This means combining scores from different levels or categories of a variable to find an overall mean ($\bar{X}$). For example, averaging the scores of male and female participants within a condition is "collapsing across gender."
• Matched-Groups Design: Each participant in one condition "matches" a participant in another condition on a specific variable.   - Process: Rank-order participants based on pretest scores and create pairs (or triplets) of people with identical or adjacent scores. One member of each pair is randomly assigned to a condition.   - Natural pairs (twins, roommates, litter mates) are often used to equate genetic or environmental factors.
• Limiting the Population: Keeping a participant variable constant by only selecting participants who meet a specific narrow criterion (e.g., only testing males with good memories). This increases internal validity and reduces error variance but restricts generalizability and can lead to a restricted range of scores.

Controlling Participant Variables in a Within-Subjects Design

• Definition: Also called a repeated-measures design, this involves measuring the same individual under all conditions of the independent variable. This ensures participants are identical across conditions.
• Advantages: It eliminates confounding from virtually all participant variables and requires a smaller $N$. It is statistically more powerful than between-subjects designs.
• Drawbacks:   - Diffusion of Treatment and Demand Characteristics: Participants may guess the hypothesis because they see all conditions.   - Subject History: External experiences occurring over the course of the study that change responses.   - Subject Maturation: Internal development or growth over time that influences responses.   - Subject Mortality (Attrition): Loss of participants before the study is complete. This is non-random; those who stay may be more motivated or skilled, leading to biased results and limited external validity.
• Pretest-Posttest Design: A specific type of repeated-measures design where participants are measured before and after a treatment (e.g., blood pressure before and after meditation).

Order Effects in Repeated-Measures Designs

• Order effects include practice effects (improvement), fatigue effects (decline), carry-over effects (previous trials influencing later ones), and response sets (habitual responding).
• Controlling Order Effects:   - Complete Counterbalancing: Testing participants using every possible order of conditions. For three conditions ($A, B, C$), there are $6$ possible orders ($ABC, ACB, BCA, BAC, CAB, CBA$). This ensures every condition appears in every position and covers every sequence.   - Partial Counterbalancing: Using only some of the possible orders, often via a Latin square design. This typically balances position (what comes first, second, etc.) but may not balance specific carry-over effects (e.g., $C$ may never follow $A$).   - Randomization: Randomly creating sequences for each participant. This is best when there are many conditions or trials.
• Blocking: Performing all trials of one condition together in a "block" before moving to the next condition to prevent confusion when procedures change.

Choosing a Design

• Within-subjects (Repeated-Measures) is preferred when:   - Individual differences (cognitive strategies, physical abilities) strongly influence responses.   - Participants are scarce ($N$ must be small).   - The study examines a sequence (learning, practice, maturation).
• Between-subjects is preferred when:   - Nonsymmetrical Carry-over Effects exist: This happens when the carry-over from order $A-B$ is not the same as $B-A$ (e.g., you cannot "unteach" a skill learned in one condition).   - Surprise elements are required.   - Subject mortality/history/maturation are high risks.   - Stimulus requirements are high: Within-subjects designs often require multiple comparable sets of stimuli (e.g., three different but equal videotaped robberies) to avoid simple practice with the same material.
• Power vs. External Validity: Researchers often choose the design that maximizes power and internal validity, even if it creates a unique situation that limits external validity.

Review and Application Questions

• Numerical Data for Recall Experiment: Stimulus is a videotape of a robbery; responses involve $30$ questions; Likert scale for another study is $1$ (strongly agree) to $5$ (strongly disagree).
• milgram study context: sampled the general adult population, not just students, to avoid bias in responsiveness to authority.

Questions & Discussion

• Question 20: (a) Why might gender differences in your sample not represent participants' views? (b) Why could a manipulation appear not to have worked although it did? (c) How can you add to a questionnaire to reduce demand characteristics (e.g., distractions)?
• Question 21: Measuring sexism by observing help given to a confederate of the opposite sex. (a) What demand characteristics might mislead you? (b) How to avoid this?
• Question 23 (The Pistol Study): Play different music, shoot a blank pistol, and measure anxiety. (a) Demand characteristic influences? (b) Informed consent info needed? (c) Risks? (d) Participant selection criteria to minimize risk? (e) Debriefing content? (f) Major flaw as the study continues? (g) Should you conduct it?
• Question 24: Drug treatment for mental illness. What conflicting ethical and design principles exist regarding the use of a control condition?
• Question 25 (Day-care Aggression): Observing children playing with dolls after an adult behaves aggressively. (a) Demand characteristic problems? (b) Dealing with biases? (c) Ethical problems? (d) Meeting APA guidelines?
• Question 26: Movie study involving $25$-minute films in black and white. Questionnaire uses $20$ Likert-type statements ($1$ to $5$). Write instructions, informed consent, and debriefing.
• Discussion Question 23 (Motivational Messages): Three types of messages, heard daily for two weeks, followed by a $20$-question well-being test. (a) Scheme for order effects? (b) Three participant problems (maturation, history, mortality)? (c) carry-over problem? (d) Ethical problems?
• Discussion Question 26 (Out sick): A participant is out sick for one week. (a) What to do with the data? (b) New selection criterion to add? (c) Impact on relationship strength? (d) Impact on external validity?