Chapter 10: Single-Variable, Independent-Groups Designs

Overview of Experimental Design

Experimental design is characterized by specific criteria to test hypotheses regarding the causal effects of the independent variable ( $IV$ ).
Core Requirements: - Includes at least two levels of the independent variable ( $IV$ ). - Utilizes random assignment of participants to different conditions. - Employs specific procedures for hypothesis testing. - Includes controls to mitigate major threats to internal validity.

Concepts of Variance

Variance is a fundamental requirement in research; without it, there is nothing to test.
Research designs are specifically structured to control unwanted sources of variance to evaluate the effects of the independent variable accurately.

Forms of Variance

Systematic Between-Groups Variance: - Experimental Variance: The variability in scores due to the manipulation of the independent variable. - Extraneous Variance: The variability accounted for by the effects of extraneous variables (variables other than the $IV$ that might affect the dependent measure). These are potential sources of confounding and must be controlled.
Nonsystematic Within-Groups Variance: - Often referred to as error variance, this is due to chance factors and individual differences among participants.

The F-test (ANOVA) Ratio

Results are analyzed using the F-test, which calculates the ratio of between-groups variation to within-groups variation.
Formula: $F = \frac{\text{Between-groups}}{\text{Within-groups}}$
In terms of mean squares ( $MS$ ): $F = \frac{MS_{\text{between}}}{MS_{\text{within}}}$

Controlling Variance

Maximizing Experimental Variance: Ensure real differences exist between experimental groups using manipulation checks.
Controlling Extraneous Variance: Goal is to have groups as similar as possible at the start so that the only difference is the $IV$ manipulation. Methods include: - Random assignment. - Selecting participants similar on specific variables. - Including extraneous variables (e.g., gender, age) as part of the design. - Matching participants or using within-subjects designs.
Minimizing Error Variance: Reduce variability due to chance or measurement error. Methods include: - Maintaining carefully controlled measurement conditions. - Using reliable measures. - Utilizing random assignment. - Employing within-subjects designs.

Manipulation Checks

A manipulation check is a specific test to determine if the $IV$ manipulation functioned as intended.
Case Study Example: Gender and Anger Expression - Hypothesis: Females, but not males, tend to turn anger inward rather than expressing it externally. - Provocation Manipulation: Researcher intentionally fumbles procedures, forcing participants to repeat parts of the study. - Dependent Measure (Expressed Hostility): Data shows females respond with less hostility than males. - First Manipulation Check (Reported Anger): Suggests females might not have been angered by the provocation (lower reported percentages). - Second Manipulation Check (Physiological Arousal): Indicates the report of less anger by females is real, validated by physical measures.

Nonexperimental Designs

These designs lack the critical controls of experimental research and should be used with caution.
Ex Post Facto Design: - Weakest design. Attempting to determine causes after an event has already happened. - Does not control for confounding variables. - Example: A study finding $28/30$ ( $93\%$ ) of hyperactive children consume food additives, claiming additives cause hyperactivity.
Single-Group, Posttest-Only Design: - No control over confounding variables; relies on an "implicit control group" (assumptions of what would have happened without treatment). - Example: $20$ hyperactive children change to a no-additive diet for $4$ weeks; $8/20$ remain hyperactive.
Single-Group, Pretest-Posttest Design: - Pretest documents change, but cannot rule out factors like history, maturation, or regression to the mean.
Pretest-Posttest, Natural Control-Group Design: - Compares a naturally occurring Group A (pretest $→$ treatment $→$ posttest) against a naturally occurring Group B (pretest $→$ no treatment $→$ posttest). - Reasonably strong but fails to control for "selection" confounding, as participants are not randomly assigned.

Experimental Research Designs

These meet all experimental criteria and provide more powerful hypothesis tests.
Randomized, Posttest-Only, Control-Group Design: - Participants are randomly assigned to Group A (Treatment) or Group B (Control) and then measured. - Random assignment controls for selection. Comparisons between groups control for history and maturation. - Groups are considered "statistically equal" because small differences result only from sampling error.
Randomized, Pretest-Posttest, Control-Group Design: - Adds a pretest to the randomized posttest-only structure. - Quantifies change and verifies initial group equality. - Strength: Excellent control over confounding. - Potential Issue: Instrumentation effects.
Multilevel, Randomized, Between-Subjects Design: - A multi-group extension (Groups $1, 2, \dots, N$ ). - May or may not include a pretest. Controls virtually all confounding sources. - Example: Testing room temperature effects on typing speed using six separate temperature conditions.

Statistical Analysis and ANOVA

Data Type Selection: - Nominal Data: Use Chi-Square ( $\chi^2$ ). - Ordinal Data: Use Mann-Whitney U-test (for two groups). - Interval or Ratio Data: - Two groups: $t$ -test of posttest measures. - More than two groups/complex designs: Analysis of Variance (ANOVA).

Analysis of Variance (ANOVA) Table

ANOVA evaluates differences in group means by comparing variance estimates (mean squares).
The larger the differences between group means, the greater the $F$ -value.
Example ANOVA Summary Table: - Between-groups: $df = 5$ , $SS = 1134.67$ , $MS = 226.93$ , $F = 5.51$ , $p = .006$ - Within-groups: $df = 42$ , $SS = 1731.25$ , $MS = 41.22$ - Total: $df = 47$ , $SS = 2865.92$

Specific Mean Comparisons

A significant $F$ -test indicates at least one group differs significantly from another.
Planned (A Priori) Comparisons: Decided based on theory before data collection to test specific causal relationships.
Post Hoc Tests (Posterior Comparisons): Applied after finding a significant $F$ to see exactly which groups differ. They control for Type I errors. - Examples: Tukey, Newman-Keuls, Sheffe.

Visualizing Data and Error Bars

Graphs like frequency polygons and histograms are used to show group differences.
Error Bars: Most commonly represent the Standard Error of the Mean (the standard deviation of the distribution if every possible sample was taken from the population).
Rule of Thumb: If error bars overlap between conditions, the groups are likely not significantly different.

Specialized Experimental Designs

Solomon’s Four-Group Design: - Combines randomized posttest-only and randomized pretest-posttest designs. - Specifically designed to account for/control possible interaction between the pretest and the manipulation. - Structured as a factorial study crossing Treatment/No Treatment with Pretest/No Pretest.
Correlated-Groups Designs: Within-subjects and matched-subjects designs.
Single-Subject Designs.
Factorial Designs: Explore multiple independent variables simultaneously.

Contingency Tables for Discrete Variables

Used to determine relationships between two discrete variables.
Requires knowing percentages in all cells to determine relationships.
Relationship Profiles: - Relationship present: Percentages in adult behavior (e.g., Abusive vs. Not Abusive) differ significantly based on childhood experience (e.g., Abused $70\%/30\%$ vs. Not Abused $10\%/90\%$ ). - No relationship: Percentages are identical across categories (e.g., both Abused and Not Abused groups show $70\%$ abusive adult behavior).

Ethical Principles

Treatment Studies: Control participants should be offered the effective treatment after the study concludes.
Informed Consent: Essential for all manipulations.
Risk: Random assignment to dangerous conditions is considered a severe ethical violation.