Confounding Variables, Randomization, and Research Validity Notes

Confounding Variables and Lurking Variables in Observational Research

• Core Objective of Empirical Research: The fundamental goal of research is to demonstrate that an explanatory variable is directly responsible for changes observed in a response variable.

• Confounding and Lurking Variables Defined: Extraneous variables that provide alternative explanations for the observed relationship between an explanatory variable and a response variable. Their presence prevents researchers from establishing a direct cause-and-effect relationship.

• Examples of Confounding Variables in Health Outcomes:

  • Genetics: Genetic variation is a primary confounding variable when comparing health outcomes such as weight reduction or diabetes reduction across individuals.

  • General Fitness and Health Consciousness: Individuals who voluntarily take vitamins often possess baseline health consciousness and higher fitness levels. Vitamins are routinely sold at health-conscious stores, attracting individuals who are already self-conscious and active about their health. Fitness itself may not directly drive vitamin consumption, but overall body concern links both behaviors.

  • Socioeconomic Status (SES) and Earning Potential: Higher levels of education correlate with higher income. Personalized vitamin subscription services can cost between $\$60$ and $\$80$ per month. A monthly expense of $\$60$ accumulates significantly, making it unaffordable for many individuals. However, individuals who can afford $\$60$ per month for vitamins are also more likely to afford gym memberships, shop at high-end grocers such as Whole Foods, and possess the luxury of free time to focus on personal health rather than working multiple jobs. Consequently, research showing positive health outcomes in Vitamin D users does not necessarily prove that Vitamin D caused those changes.

• Catholic Schools and Four-Year College Attendance Example:

  • Data tracking high school graduates attending four-year colleges across Catholic schools, other religious schools, non-sectarian private schools, and public schools indicates a higher rate of four-year college attendance among Catholic school graduates compared to public school graduates.

  • Lurking Variables: Valuing education, strong family expectations regarding higher education, and parental support act as lurking variables that explain both enrollment in a Catholic school and subsequent attendance at a four-year college.

  • Classification as Observational Research: This comparison is strictly observational research because participants cannot be randomly assigned to attend private or public schools. Forcing school selection on families would be completely unethical.

Methods for Controlling Confounding Variables

• Definition of Controlling Variables: Controlling a confounding or lurking variable means eliminating it as a potential alternative explanation for the observed relationship. If every alternative explanation is controlled, the explanatory variable remains as the sole explanation for changes in the response variable.

• Causal Claims in Experiments: Experiments theoretically allow researchers to make causal claims specifically because they control for all alternative explanations, confounding variables, and lurking variables.

• Method 1: Keeping All Participants Constant:

  • Procedure: Restricting the study sample strictly to individuals who share the exact same level of the confounding variable.

  • Example: Socioeconomic Status (SES) is measured on a standardized scale ranging from $0$ to $100$. A researcher selects only individuals who score exactly $80$ on the scale ($\text{SES} = 80$) and assigns some to public schools and others to private schools. If private school students show higher college attendance rates, SES is ruled out as an explanation because all participants had an identical SES score of $80$.

  • Limitation: Completely lacks generalizability. The results apply strictly to individuals with an SES score of $80$ and cannot be generalized to populations with higher or lower socioeconomic status.

• Method 2: Including the Variable in the Design of the Study:

  • Procedure: Explicitly measuring the confounding variable across a wide range of values and balancing those levels equally across all comparison groups.

  • Example: Evaluating participants across multiple SES levels (e.g., $\text{SES} = 10$ up to high SES values) and structuring two comparison groups (public school vs. private school) so that each group contains equal proportions of individuals at every SES level.

  • Limitation: Only controls for the specific variable(s) explicitly measured and included in the design (typically one or two known variables). It fails to control for unmeasured or unknown variables, such as class size, family culture, or unexpected daily routine differences (e.g., a hypothetical scenario where private school students jump rope every morning before class).

• Method 3: Random Assignment to Groups (Randomization):

  • Procedure: Using a random process to divide participants into comparison groups.

  • Mechanism: Spreads all individual characteristics—both known variables (such as SES or class size) and unknown variables (such as family culture or morning jump-rope habits)—evenly across groups on average.

  • Effect on Validity: Because randomization balances all potential confounding variables across groups on average, it isolates the explanatory variable. This makes random assignment the best method for controlling confounding variables and the defining requirement for establishing cause-and-effect in a true experiment.

  • Caveat: Randomization controls confounding variables on average, but it does not absolutely guarantee perfect equality in every single trial due to chance variation.

Distinguishing Random Assignment and Random Sampling

• Terminology Equivalences:

  • Random Assignment and Randomization are identical, interchangeable terms.

  • Random Sampling and Random Selection are identical, interchangeable terms.

• Random Assignment Defined:

  • The process used to divide an existing sample of participants into treatment groups.

  • Requires that every participant has an equal chance of being assigned to any treatment group.

  • Manipulability Requirement: Can only be performed on variables that researchers can actively manipulate (e.g., randomly assigning individuals to drink $2$ cups of caffeinated coffee versus $2$ cups of decaf coffee). Traits that cannot be manipulated (e.g., race, age, or academic class standing) cannot be randomly assigned.

  • Identification Rule: Studies utilizing random assignment will explicitly contain the word "random," "randomized," or "random assignment." If these terms are absent from the text, random assignment did not occur.

• Random Sampling Defined:

  • The process used to select participants from a broader population into a study sample (e.g., using a random number generator on a master population list).

• Complete Independence of Concepts:

  • Random assignment and random sampling are completely independent of one another. The presence or absence of one has no bearing on the presence or absence of the other.

  • A study can feature:

  1. Both random sampling and random assignment.

  2. Neither random sampling nor random assignment.

  3. Random assignment without random sampling.

  4. Random sampling without random assignment.

• Defining Experiments vs. Observational Studies:

  • An experiment is defined solely by the presence of random assignment to groups.

  • Random sampling has no bearing on whether a study is an experiment.

  • Common Student Mistake: Assuming a study is an experiment simply because the word "random" appears in the context of sampling. For example, randomly selecting $50$ freshmen, $50$ sophomores, $50$ juniors, and $50$ seniors to measure and compare stress levels is an observational study, not an experiment, because participants cannot be randomly assigned to their academic class standing.

Research Validity: Internal and External Validity

• Overview: Research validity comprises two independent dimensions: Internal Validity and External Validity. A study can be high in both, low in both, or high in one and low in the other.

• Internal Validity (Design Validity):

  • Definition: The degree to which a study successfully controls for confounding variables, allowing researchers to confidently make a causal claim between the explanatory and response variables.

  • Determining Criterion: Dependent strictly on random assignment.

  • High Internal Validity: Present when random assignment is used (all true experiments possess high internal validity).

  • Low Internal Validity: Present when random assignment is absent.

• External Validity (Sampling Validity):

  • Definition: The degree to which the results obtained from a study sample can be generalized back to the broader target population.

  • Determining Criterion: Dependent strictly on random sampling.

  • High External Validity: Present when random sampling is used, ensuring a representative sample.

  • Low External Validity: Present when random sampling is absent (e.g., relying on voluntary responses, convenience samples, or soliciting participants walking across campus areas like the Oval at OU, which excludes students currently in class).

• Four Combinations of Research Validity Illustrated:

  • High Internal Validity & High External Validity: A researcher randomly selects $200$ patients from a complete hospital patient list (random sampling $\rightarrow$ high external validity), and then randomly assigns those $200$ participants to receive either an active medication or a placebo (random assignment $\rightarrow$ high internal validity).

  • High Internal Validity & Low External Validity: An instructor solicits volunteers from a single university class (convenience sample / no random sampling $\rightarrow$ low external validity), and then randomly assigns those volunteers to receive either a medication or a placebo (random assignment $\rightarrow$ high internal validity).

  • Low Internal Validity & Low External Validity: A researcher asks for volunteers (no random sampling $\rightarrow$ low external validity) and categorizes them by their current class standing (freshmen, sophomores, juniors, seniors) to measure opinions on campus parking (no random assignment $\rightarrow$ low internal validity).