Chapter 13: Alternative Research Designs

Internal Validity

A type of evaluation of your experiment; it asks whether your IV is the only possible explanation of the results shown for your DV.

Confounding

Caused by an uncontrolled extraneous variable that varies systematically with the IV.

Extraneous Variables

Uncontrolled variables that may unintentionally influence the dependent variable (DV) and thus invalidate an experiment.

Cause-And-Effect Relation

Occurs when we know that a particular IV (cause) leads to specific changes in a DV (effect).

What is random assignment?

Distributing participants into treatment groups on a random basis, ensuring equality before the experiment.

How does random assignment help in experiments?

It controls for selection bias.

Does random assignment guarantee equality in small sample sizes?

No, random assignment doesn't guarantee equality in small sample sizes.

Random Selection

A control technique that ensures that each member of the population has an equal chance of being chosen for an experiment.

Selection

A threat to internal validity that can occur if participants are chosen in such a way that the groups are not equal before the experiment; the researcher cannot then be certain that the IV caused any difference observed after the experiment.

History

A threat to internal validity; refers to events that occur between the DV measurements in a repeated-measures design

Maturation

An internal validity threat; refers to changes in participants that occur over time during an experiment; could include actual physical maturation or tiredness, boredom, hunger, and so on.

Testing

A threat to internal validity that occurs because measuring the DV causes a change in the DV.

Statistical Regression

This threat to internal validity occurs when low scorers improve or high scorers fall on a second administration of a test solely as a result of statistical reasons.

Interactions With Selection

This threat to internal validity can occur if there are systematic differences between or among selected treatment groups based on maturation, history, or instrumentation.

Experimental Morality

This threat to internal validity can occur if experimental participants from different groups drop out of the experiment at different rates.

Instrumentation

This threat to internal validity occurs if the equipment or human measuring the DV changes its measuring criterion over time.

Diffusion or Imitation of Treatment

This threat to internal validity can occur if participants in one treatment group become familiar with the treatment being received by another group and copy that treatment.

Case-Study Approach

An observational technique in which a record of observations about a single participant is compiled

Experimental Analysis of Behavior

A research approach popularized by B.F. Skinner, in which a single participant is studied

Baseline

A measurement of a behavior made under normal conditions (i.e., no IV is present; a control condition

A

Refers to the baseline measurement in a single-case experimental designs

B

Refers to the outcome (treatment) measurement in a single-case experimental designs

A-B Design

A single-case experimental design in which you measure the baseline behavior, institute a treatment, and use a posttest. Similar to pretest-posttest designs however it lacks a control group. Causality is difficult to interpret due to potential extraneous variables.

A-B-A Design

A single-case experimental design consisting of baseline measurement, a treatment, a posttest, and a return to the baseline condition. It may not be recommended if the participant is left without a beneficial or necessary treatment in the second baseline

A-B-A-B Design

A single-case experimental design consisting of a baseline, treatment, posttest, return to baseline, repeated treatment, and second posttest. This design gives the best chance of isolating causation. However, when the treatment leads to a behavioral change, reverting to baseline may not be possible, practical, or ethical.

Quasi-Experimental Design

A research design used when the researcher cannot randomly assign experimental participants to the groups but the researcher does manipulate an IV and measure a DV

Nonequivalent Group Design

A quasi-experimental design involving two or more groups that are not randomly assigned and a comparison group (no treatment) is compared to one or more treatment groups

Interrupted Time-Series Design

A quasi-experimental design, involving a single group of participants, that includes repeated pretreatment measures, introduces a treatment (interruption), and repeated post-treatment measures.

What is a Pretest-Posttest Control-Group Design?

A true experimental design involving two randomly assigned groups, with the experimental group receiving the treatment and the control group not. Both groups are measured before and after the treatment.

How does a Pretest-Posttest Control-Group Design control threats?

It controls threats like history, maturation, and testing by affecting both groups equally. Any difference in posttest scores can be attributed to the independent variable.

Solomon Four-Group Design

Adds two additional groups to the pretest-posttest control-group design. Provides stronger control for internal validity and enhances external validity. Complicated for statistical analysis, often requiring factorial design approaches.

Single-Case Experimental Design

Involves manipulating variables and controlling for internal validity, allowing for cause-and-effect conclusions.

Case Study

Observational technique involving intense study of a single individual without manipulating variables.

Reasons for Using Single-Case Experimental Designs

- When a sample of one is the only option
- When variability among individuals on a variable is minimal
- When finding a single instance that contradicts a widely accepted theory
- When research is expensive, time-consuming, or difficult to control

What do repeated measures in single-case experimental designs do and help avoid?

They measure the participant's behavior precisely multiple times under standardized conditions to track stability, detect changes, and avoid confounding variables.

What do baseline measures in single-case experimental designs do? How many observations should you collect to determine a baseline?

They collect data on the behavior before introducing the independent variable (IV) to establish a pattern that provides a control condition to compare changes in behavior. Collect at least three observations to determine a baseline

Arguments Against Statistical Analysis in Single-Case Experimental Designs

- Visual inspection of data is often used to assess changes, with some researchers believing that statistical methods are unnecessary.
- Statistical significance might not always reflect practical or clinical significance.
- Subtle effects revealed by statistical methods may not be replicable or may not have external validity.

Arguments for Statistical Analysis in Single-Case Experimental Designs

- Statistical methods can increase the accuracy of conclusions by detecting effects that visual inspection might miss.
- Helps to identify both Type I and Type II errors.
- Statistical analysis can be beneficial when baselines are unstable or when testing in real-world settings with high variability.

Type I Error (alpha)

Rejecting null hypothesis when it is true (false positive)

Type II Error (beta)

Failing to reject a false null hypothesis (false negative)

Posttest-Only Control-Group Design

Similar to pretest-posttest but without pretests.
Relies on random assignment to ensure group equivalence. Controls internal validity threats effectively, especially in large samples.

Quasi-Experimental Internal Validity Threats

Extraneous variables: Uncontrolled factors that systematically vary with the IV and may unintentionally influence the DV

Maturation: Changes in participants over time could affect the results

Instrumentation: Changes or inconsistencies in measurement tools or procedures over time

Statistical Regression: Extreme scores on a pretest might regress towards the mean, skewing results.

Interaction Between Selection and History: Events affecting different groups differently can confound results.

Interrupted Time-Series Design Internal Validity Threats

History: Events occurring during the study may affect results, especially if the treatment and measurements span a long period.

Maturation: Changes due to aging or development over time could influence results.

Instrumentation: Changes in measurement procedures over time can introduce errors.

The two general methods we use to protect the internal validity of our experiment are ____ and ___

random assignment, experimental design

Why is it essential to use random assignment of our participants to their groups?

It is essential to randomly assign participants to groups to help assure that the groups are equal before the experiment.

Distinguish between random assignment and random selection.

Random selection refers to choosing participants from a population in a nonsystematic manner. Random assignment refers to allocating those randomly selected participants to different groups in a nonsystematic manner.

What is the drawback of using the pretest-posttest control-group design to help with internal validity?

It is problematic to use the pretest-posttest control group design to help with internal validity because this design cannot control all internal validity threats.

A friend tells you she was a participant in a psychology experiment and says, "It was crazy! We took a personality test, watched a film, and then took the same test again?" From this description, you could tell she was in the
a. Control group of a posttest-only control-group design
b. Experimental group of a posttest-only control-group design
c. Control group of a pretest-posttest control-group design
d. Experimental group of a pretest-posttest control-group design

d. Experimental group of a pretest-posttest control-group design

What is the drawback of using the Solomon four-group design as a control for internal validity?

The drawback of using the Solomon four-group design to control for internal validity is that there is no statistical test to analyze all the data from this design.

Why is the posttest-only control-group design a good choice for controlling internal validity?

The posttest-only control group design is a good choice for controlling internal validity because it seems to control the internal validity threats

Why were single-case designs quite popular in psychology's early years but less popular today?

The use of single-case designs has decreased recently because of statistical innovations—an increase in both statistical tests and computerized analysis programs.

How can a single-case experimental design be used to disprove a theory?

A single-case experimental design can be used to disprove a theory because it takes only a single negative instance to invalidate a theory's general applicability.

To come up with a comparison in the single-case design, we first measure behavior before the treatment during the _____ period. To get a stable measurement, we should make at least ____ observations.

baseline, three

In essence, _____ serve(s) as the control condition in the single-case design
a. Baseline measurements
b. Repeated measures
c. Changing one variable at a time
d. Experimental analysis of the behavior

a. Baseline measurements

Summarize two arguments for and two arguments against the use of statistical analysis in single-case designs

For: Visual inspections of data may lead to incorrect conclusions. Statistical analysis gives increased accuracy.

Against: Treatments should produce effects that are visually apparent. Clinical significance is more important than statistical significance.

A-B
a. Leaves the participant in a baseline phase
b. Best single-case design for determining cause-and-effect relations
c. Has many threats to internal validity

c. Has many threats to internal validity

A-B-A
a. Leaves the participant in a baseline phase
b. Best single-case design for determining cause-and-effect relations
c. Has many threats to internal validity

a. Leaves the participant in a baseline phase

A-B-A-B
a. Leaves the participant in a baseline phase
b. Best single-case design for determining cause-and-effect relations
c. Has many threats to internal validity

b. Best single-case design for determining cause-and-effect relations

Why might you be forced to use an A-B single-case design in the real world? Give an example of such a situation.

You might be forced to use an A-B single-case design in the real world because it can be impractical or unethical to reverse a treatment. Many examples are possible. Suppose you work with an individual who suffers from anorexia or bulimia. You institute a treatment and the individual shows marked improvement. However, you fear that removing the treatment could result in a recurrence of the problem.

Differentiate between experimental designs, quasi-experimental designs, and ex post facto designs.

Experimental designs involve manipulation of IVs, control of extraneous variables, and measurement of DVs so that cause-and-effect relationships can be determined. Ex post facto designs involve IVs that have already occurred or that are predetermined, such as sex. Quasi-experimental designs involve manipulating IVs with groups of participants that are predetermined.

Give two reasons why you might choose to use a quasi-experimental design rather than an experimental design

You might choose to use a quasi-experimental design rather than an experimental design in a situation when random assignment is impossible or if you wish to evaluate an ongoing program.

Nonequivalent group design
a. typically has one group of participants
b. has two groups of participants
c. involves pretesting participants
d. does not involve pretesting participants
e. is prone to the internal validity threat of history
f. is prone to several internal validity threats

b. has two groups of participants
c. involves pretesting participants
f. is prone to several internal validity threats

Interrupted time-series design
a. typically has one group of participants
b. has two groups of participants
c. involves pretesting participants
d. does not involve pretesting participants
e. is prone to the internal validity threat of history
f. is prone to several internal validity threats

a. typically has one group of participants

d. does not involve pretesting participants (*Interrupted time-series design observations before the treatment are not true pretests because they often represent already available data. However, it could easily be argued that both designs include pretesting)

e. is prone to the internal validity threat of history

What was the key to Geronimus's (1991) research that allowed her to conclude that the effects of teenage pregnancy are not as negative as typically thought?

The key element in Geronimus's research that allowed a more positive conclusion about teen pregnancy was the realization that finding a true control group would be impossible; hence the need for a strong comparison group.

We summarized two interrupted time-series analyses in the text: one dealing with changing AFDC (A-B) requirements and one dealing with changing motorcycle helmet laws (A-B-A). Why are we more certain about our conclusion in the case of the helmet laws than with the AFDC requirements?

We are more certain about our conclusion in the helmet law case because it simulated an A-B-A design, whereas the AFDC study was only an A-B design. There is more evidence regarding the helmet law example to suggest a relationship than the AFDC example.

If Prohibition (the outlawing of alcoholic beverages in the 1920s) were to be treated as an experiment to determine its effects on alcohol consumption, what design would this represent?
a. nonequivalent group design
b. single-case design
c. interrupted time-series design with control group
d. interrupted time-series design

d. interrupted time-series design