Week 8: Quantitative Experimental and Quasi-Experimental Research

Fundamentals and Key Features of Experimental Research

  • Five Foundational Pillars of Experimental Studies:

    • Adequate sample size: Ensuring an ample number of participants to detect genuine treatment effects.

    • Random assignment: Allocating participants randomly into intervention/treatment conditions versus comparison/control conditions.

    • Consistent delivery of interventions: Standardizing execution across all participants and study settings.

    • Rigorous control of extraneous influences and bias: Safeguarding internal validity against systematic distortions.

    • Low level of missing data: Preserving the integrity and balance of participant data throughout the study timeline.

Experimental Study Design and the Classic Two-Group Model

  • Theoretical Framework of Experimental Design:

    • Experimental designs, clinically termed randomized controlled trials (RCTs) or true experiments, provide the most robust empirical evidence regarding whether an active cause (an intervention) directly produces an effect (a desired outcome).

    • Occupies the highest strata of evidence hierarchies when evaluating clinical questions regarding causality and treatment effectiveness.

  • Three Essential Elements of True Experiments:

    • Intervention (Manipulation):

      • The experimenter deliberately introduces an active agent, protocol, or treatment to a selected subset of participants.

      • The experimental group serves as the designated cohort receiving the active intervention.

    • Control:

      • The experimenter introduces strict environmental and structural controls, primarily through the inclusion of an unexposed or alternate comparison group (the control group).

    • Randomization (Random Assignment):

      • The experimenter allocates participants to either the control or experimental condition purely by chance.

      • The functional objective of randomization is to ensure that all study groups are equivalent in every measured and unmeasured characteristic before the introduction of the independent variable, isolating receipt of the intervention as the sole distinguishing factor.

Random Assignment of Sample to Experimental and Control Groups
  • Mechanics of the Classic Two-Group Experimental Design:

    • Participant flow begins with defined eligibility criteria from the broader population.

    • A representative sample is recruited.

    • Random assignment directs participants into either Treatment Group A (Intervention A) or Treatment Group B (Intervention B / Control condition).

    • Both groups are tracked longitudinally to collect and compare standardized measurement outcomes.

Classic Two-Group Experimental Study Design

Intervention Fidelity, Protocol Standardization, and Sample Size Planning

  • Intervention Protocol and Treatment Fidelity:

    • Interventions require exhaustive delineation within formal, written research protocols that stipulate the exact nature, timing, duration, and operational delivery of the treatment.

    • Researchers must monitor intervention fidelity (treatment fidelity) to confirm that the planned therapeutic regimen is delivered precisely as designed and is successfully received and comprehended by the subject.

  • Operational Standards for Consistent Delivery:

    • Clear delineation: Unambiguous definitions detailing exactly who receives the intervention, what components are administered, and under what specific conditions.

    • Written intervention protocols: Highly explicit, reproducible, step-by-step procedural manuals.

    • Standardized personnel training: Delivering uniform training to a deliberately limited group of delivery agents to eliminate inter-operator variation.

    • Contamination and crossover prevention: Enforcing procedural safeguards to ensure subjects do not switch assignment arms or receive inadvertent exposure to the alternate condition.

  • Statistical Power Analysis and Sample Size Requirements:

    • Quantitative rigor demands conducting an a priori statistical power analysis to calculate the minimal sample size necessary to detect significant differences and avoid Type II errors.

    • If researchers fail to achieve the required sample size identified by the power analysis, they must explicitly document this shortfall, provide a rationale for the deficit, and examine its implications within the study limitations section of the published report.

Experimental Control Conditions

  • Variations of the Comparison / Control Condition:

    • No intervention at all: The comparison cohort receives neither experimental treatment nor contact.

    • Placebo intervention: The control group receives an inactive pseudo-intervention that appears identical to the experimental therapy but possesses zero therapeutic efficacy.

    • Usual care (standard care) intervention: Participants receive routine, standard-of-care clinical management or normal institutional procedures without experimental augmentation.

    • Alternative (different) intervention: Subjects in the control condition receive an entirely distinct therapeutic modality (e.g., comparing auditory stimulation directly against visual stimulation).

    • Dose-response comparison: The comparison group receives the same active therapeutic agent or intervention, but at a systematically modified dose, duration, or intensity.

Advantages, Disadvantages, and Identification of Experimental Designs

  • Strengths of True Experiments:

    • Functions as the most scientifically powerful design for establishing direct cause-and-effect relationships between clinical variables.

  • Limitations and Practical Challenges:

    • Feasibility and ethical barriers: Withholding beneficial treatments, administering potentially hazardous substances, or mandating behaviors is frequently impermissible or clinically unfeasible.

    • The Hawthorne effect: A form of behavioral reactivity wherein participants alter or improve their normal behaviors simply because they are conscious of being observed within a research investigation.

    • Resource intensity: Experimental trials frequently require immense financial investments, extended timelines, and extensive personnel infrastructure.

  • Criteria for Identifying Experimental Studies (Polit & Beck, p. 123):

    • Published reports generally declare their methodology explicitly within the methods section using terms such as experimental study or randomized controlled trial (RCT).

    • When explicit terminology is omitted, a study is classified as experimental if two criteria are met:

    • The stated study purpose is to test the effects of an active intervention.

    • Study participants are allocated into comparison groups using random assignment.

Hierarchy of Research Evidence

  • Stratification of Evidence from Strongest to Weakest:

    • Level 1 (Highest Quality / Strongest Evidence): Systematic reviews of experimental studies (well-designed randomized controlled trials [RCTs]).

    • Level 2: Meta-analyses of experimental (RCT) and quasi-experimental studies.

    • Level 3: Integrative reviews of experimental (RCT) and quasi-experimental studies.

    • Level 4: Single experimental studies (RCT).

    • Level 5: Single quasi-experimental studies.

    • Level 6: Meta-analyses of correlational studies.

    • Level 7: Integrative reviews of correlational and descriptive studies.

    • Level 8: Qualitative research meta-syntheses and meta-summaries.

    • Level 9: Single correlational studies.

    • Level 10: Single qualitative or descriptive studies.

    • Level 11 (Lowest Quality / Weakest Evidence): Opinions of respected authorities based upon clinical evidence, alongside reports of expert committees.

Hierarchy of Research Evidence

Quasi-Experimental Research Designs

  • Defining Characteristics:

    • Quasi-experiments involve an active manipulation or intervention introduced by the researcher, but specifically lack the element of random assignment to study arms.

  • Primary Design Categories:

    • Nonequivalent Control Group Designs:

      • Participants who receive the experimental intervention are compared directly against an existing, non-randomized comparison group.

      • Research Application (Tsai et al., 2020): Investigated the effects of a smartphone-based videoconferencing program on depression, loneliness, and quality of life (QOL) among nursing home residents using a nonequivalent control group pretest–posttest framework. Residents situated across 55 specific nursing homes received the videoconferencing intervention, whereas residents across 22 separate nursing homes served as the control group. Outcome measurements were obtained at baseline (pretest) and at multiple subsequent post-intervention intervals.

    • Within-Subjects Designs:

      • A single cohort of participants is examined repeatedly, tracking baseline indicators prior to the intervention and evaluating changes following the intervention, without utilizing an independent comparison group.

  • Evaluation of Quasi-Experiments:

    • Advantages: Markedly more practical, accessible, and logistically manageable within complex real-world clinical environments where randomization is unfeasible.

    • Disadvantages: Substantially weakens the ability to infer direct causality; vulnerable to rival hypotheses and unmeasured confounding variables explaining post-intervention changes.

    • Identification Tip: If a research report describes the testing of an intervention but makes no explicit mention of random assignment, the study design is categorized as quasi-experimental.

Control of Extraneous Confounders and Participant Factors

  • Principles of Research Control:

    • Research control focuses on maximizing the investigator's ability to isolate the independent variable and minimize extraneous variation.

    • Confounding (Extraneous) Variables: Variables that operate extraneous to the central research hypothesis but exert an independent effect on the dependent outcome.

      • Clinical Illustration: In evaluating the impact of a physical fitness intervention on physical functioning among nursing home residents, participant age, gender, and smoking history represent confounding variables. Each factor naturally alters physical functioning independent of the fitness regimen.

  • Primary Methodological Strategies for Participant Control:

    • Randomization (Random Assignment): The most comprehensive approach to distributing extraneous variables equally across all study conditions.

    • Homogeneity (Sample Restriction):

      • Intentionally restricting sample eligibility to eliminate extraneous variance associated with specific participant characteristics.

      • Research Application (Bowles et al., 2014): Examined the impact of discharge planning decision support on time to hospital readmission among older adult patients utilizing a quasi-experimental design. Confounders were rigorously controlled via sample homogeneity: limiting age strictly to individuals aged 5555 years or older, controlling medical condition by excluding any patient undergoing dialysis, and controlling admission source by excluding patients admitted from an institutional setting.

Methodological Bias, Attrition, and Data Completeness

  • Sources of Study Bias:

    • Defined as any systematic distortion or influence that leads to an error in inferential conclusions.

    • Common origins: Participant lack of candor, researcher preconceptions, and flawed data collection procedures.

  • Techniques for Bias Reduction:

    • Blinding (Masking): The intentional concealment of key study information from participants, data collectors, care providers, or outcome evaluators to maintain objectivity.

      • Involves masking participant group allocation status or withholding details regarding the primary study hypotheses.

    • Randomization: Eliminates allocation bias and systemic group differences.

    • Triangulation: Employing multiple data collection methods, perspectives, or sources to cross-verify findings (e.g., using two separate instruments to evaluate a single outcome measure to confirm result consistency).

  • Managing Participant Attrition and Missing Data:

    • Attrition: The loss of study participants over time, which compromises the equivalence of comparative groups established at baseline.

    • Attrition Bias: Systematic drop-out that leaves the remaining sample unrepresentative of the original population. Differential drop-out between study arms means group composition differences, rather than the intervention, may account for observed outcomes.

    • Protocols for Preserving Data Integrity:

      • Establish an acceptable missing data threshold a priori.

      • Implement retention strategies that incentivize and ease study continuation.

      • Continuously monitor dropout rates throughout data collection.

      • Provide the comparison group with parallel engagement, attention, or a placebo to balance retention incentives.

      • Formally disclose and discuss high missing data rates in the research report, utilizing advanced statistical handling methods designed to adjust for participant loss.

Statistical Analysis, Inferential Testing, and Interpretation

  • Translational Pipeline from Data to Conclusion:

    • Research execution follows a progressive analytical sequence:

    • Production of statistical results.

    • Statistical interpretation of quantitative values.

    • Translation of statistical patterns into clinical meaning.

    • Derivation of evidence-based conclusions.

  • Parametric Comparison: The Independent tt-Test:

    • The tt-test evaluates whether the difference between the mean scores of two distinct groups is statistically meaningful.

    • Calculates the magnitude of difference between group means relative to the sample size and the dispersion of scores across the standard deviation (SDSD).

    • Clinical Illustration (Park et al., 2015): Investigated a 1616-session Patient-Centered Environment Program (PCEP) delivered to community-dwelling patients diagnosed with dementia. PCEP produced statistically significant improvements in patient agitation (t=2.91t = 2.91, p<0.02p < 0.02) and patient pain (t=4.51t = 4.51, p<0.002p < 0.002).

  • Statistical Significance and Probability Metrics:

    • Quantitative studies must formally report the specific statistical tests utilized, the precise numerical value of the calculated statistic, and the associated level of statistical significance.

    • The pp-value functions as an index measuring the probability that an observed finding occurred solely due to chance, assuming the null hypothesis is true.

    • Smaller pp-values indicate higher inferential reliability.

    • Conventional threshold for significance is established at p=.05p = .05 or p<.05p < .05.

    • Nonsignificant outcomes indicate that any observed difference between groups could readily have emerged through random variation.

Statistical Significance Versus Clinical Significance

  • Core Distinction:

    • Statistical Significance: Evaluates whether an observed difference is unlikely to be an artifact of random sampling chance (typically denoted by p<.05p < .05). It answers: "Does this numerical difference matter to a statistician?"

    • Clinical Significance: Evaluates whether an observed therapeutic effect carries tangible, meaningful, and practical importance for the health, daily functioning, and lived experiences of patients, or meaningfully guides clinical decision-making. It answers: "Does this difference matter to a clinician?"

Systematic Framework for Evaluating Experimental Research

  • Why? (Study Purpose):

    • Delineating the core clinical rationale, theoretical premise, and operational research hypothesis or objective driving the investigation.

  • How? (Methodology):

    • Design: Identifying whether the study employs a true experimental design (RCT) or a quasi-experimental design.

    • Sample: Evaluating recruitment criteria, statistical power analysis, sample size adequacy, and demographic representativeness.

    • Data Collection: Inspecting intervention fidelity, procedural consistency, valid instrumentation, blinding protocols, and strategies mitigating attrition.

    • Ethics Review: Confirming institutional review board (IRB) oversight, participant protection, informed consent procedures, and clinical safety monitoring.

  • What? (Results and Translation):

    • Examining the statistical metrics, mathematical significance levels (pp-values, tt-values), and clinical relevance to determine applicability in real-world nursing practice.

Review of Statistical Terms:

  • Correlation: A statistical measure that describes the extent to which two variables fluctuate together, indicating both the strength and direction of their relationship. Correlation DOES NOT equal causation.

    • Measured by Pearson’s R Correlational Coefficient.

      • Example: The correlation between study time and exam scores often exhibits a positive relationship, suggesting that increased study time is associated with higher scores. The coefficient helps us determine the strength and direction of the linear relationship between 2 numerical variables.

      • Studies where r = ± 0 report help explain whether the linear data trends positively or negatively. If r = 0, then there is no relationship amongst the data.

        • Closer to ± 1 means a stronger linear relationship; closer to 0 means a weaker linear relationship.

  • Causation: A relationship between two variables where a change in one variable directly results in a change in the other, implying a cause and effect connection. It is essential to distinguish causation from correlation, as correlation does not necessarily imply causation.

    • T-Test: A statistical hypothesis test that compares the means of two groups to determine if they are significantly different from each other.

      • It is commonly used in experimental research to assess the impact of interventions.

  • Statistical Significance: A determination of whether the results of an experiment or study are likely to be due to chance or represent a true effect.

    • Statistical significance is often assessed using a p-value, with a common threshold set at p < 0.05, indicating that there is less than a 5% probability that the observed differences are due to chance.

    • If it is significant, then it would likely be found in the general population.