Chapter 13: Quasi-Experimental Designs

PBSI 302

How does a quasi-experiment differ from a true experiment?

researchers do not have full experimental control (there are IVs and DVS but there’s no random assignment)

What are the 2 types of quasi-experiments?

independent-groups and repeated-measures

What is a true experiment?

a study or research design where we measure variables

the experimenter has complete control over assignment of subjects to conditions and presentation of conditions to participants

What is a non-experiment?

experimenter has no control over the presentation of the IVs

can only record what happens in a certain situation (also no random assignment)

What is a nonequivalent control group design?

different participants (groups) at each level of the IV, at least one treatment group and one comparison group, no random assignment

What is the posttest only version of nonequivalent control group design?

only tested after some intervention

What is the pretest/posttest version of nonequivalent control group design?

tested before and after some intervention

What do selection effects do in quasi-experiments?

lack random assignment causes systematic difference between levels of the IVs

What do design confounds do in quasi-experiments?

another variable changes systematically with the IV

What are maturation threats in quasi-experiments?

an observed change in an experimental group unrelated to the IV

What are history threats in quasi-experiments?

some external/historical factor influences most participants

What is regression to the mean in quasi-experiments?

an extreme finding becomes less extreme over time

What is an attrition threat in quasi-experiments?

systematic dropout over time

What is a testing/instrumentation threat in quasi-experiments?

when participants are measured differently

What is observer bias in quasi-experiments?

observer expectations influence interpretation of results

What are demand characteristics in quasi-experiments?

participants guess the study’s goals and behave differently

What are placebo effects in quasi-experiments?

when changes in the DV are observed in control group

What is this an example of?

• financial concerns =potential additional differences between getting/not getting cosmetic surgery

• solution → creating matched groups and using wait-list design

selection effects

What is a wait-list design?

all participants plan to receive treatment but are randomly assigned to do so at different times

What is this an example of?

• other government programs promoting organ donation awareness in opt-out countries?

• solution → intentionally selecting countries where it wasn’t an issue and/or statistical control

design confound

What is this an example of?

• this was ruled out in the in-class cosmetic surgery example because:

  1. participants selected for outlier high/low self-esteem scores and

  2. similar pretest self-esteem levels between surgery and comparison group

regression to the mean

What is this an example of?

• this was ruled out in the in-class cosmetic surgery example because

  • people who dropped out were not systematically different in self-esteem

  • determined using missing values analysis (statistically)

attrition threat

What is this an example of?

• this was ruled out in the in-class cosmetic surgery example because

  • comparison group → both groups would be equally impacted

testing/instrumentation threats

What is this an example of?

• using a series of questionnaires → disguises true purpose

• self-report methods → mitigates observer bias

• however, placebo effect still potentially problematic

observer bias, demand characteristics, and placebo effects

Despite potential internal validity threats, what are the 5 other priorities to balance in quasi-experiments?

real-world opportunities, ethics, external validity, construct validity, and statistical validity

Both quasi-experiments and correlational studies may use independent-groups design. What is this an example of?

• marital satisfaction in relation to how you met?

correlational study

Both quasi-experiments and correlational studies may use independent-groups design. What is this an example of?

• cosmetic surgery (or not) & self-esteem?

quasi-experiment

Neither correlational studies nor quasi-experiments use random assignment. What is this an example of?

• IV is measured (ie., self-report)

correlational study

Neither correlational studies nor quasi-experiments use random assignment. What is this an example of?

• pre-existing groups

quasi-experiment

Neither correlational studies nor quasi-experiments use manipulated variables. However, quasi-experiments use techniques like what?

active selection

What do different degrees of control over alternative explanations do?

attempt to support causal claims

What are participant variables?

age, gender, ethnicity, etc.

Are participant variables measured or manipulated?

measured

When do you use participant variables over quasi-independent variables?

when you want to document similarities/differences associated with social identities

When do you use quasi-independent variables over participant variables?

when you want to focus on interventions and individual differences (ie., cosmetic surgery v. not; introverts v. extroverts)

Does this describe Large-N designs or Small-N designs?

• grouped participants

Large-N designs

Does this describe Large-N designs or Small-N designs?

• grouped averages

Large-N designs

Does this describe Large-N designs or Small-N designs?

• function of large samples

Large-N designs

Does this describe Large-N designs or Small-N designs?

• versatile settings

Large-N designs

Does this describe Large-N designs or Small-N designs?

• individual participants

Small-N designs

Does this describe Large-N designs or Small-N designs?

• individual observations

Small-N designs

Does this describe Large-N designs or Small-N designs?

• function of careful designs

Small-N designs

Does this describe Large-N designs or Small-N designs?

• intervention settings

Small-N designs

Does this describe Large-N designs or Small-N designs?

• evaluating the efficacy of a treatment or intervention

Small-N designs

What are stable-baseline designs?

behavior is observed for extended baseline period before beginning a treatment or intervention

Are stable-baseline designs between-subjects or within-subjects?

within-subjects

What are multiple-baseline designs?

researchers stagger their introduction of an intervention across a variety of individuals, times, or situations to rule out alternative explanations

What are reversal designs?

observe a problem behavior before treatments, then add treatment, than take away (reverse), then add it back

What is the goal of reversal designs?

see if reversal of treatment → opposite effect in DV, and if re-adding treatment → effect in DV comes back

When evaluating the 4 validities in small-N designs, what validity does this describe?

• okay if study is designed well

internal validity

When evaluating the 4 validities in small-N designs, what validity does this describe?

• depends on goals

external validity

When evaluating the 4 validities in small-N designs, what validity does this describe?

• depends on measures and definitions

construct validity

When evaluating the 4 validities in small-N designs, what validity does this describe?

• unique concerns

• focus on effect sizes

statistical validity

The degree to which a quasi-experiment supports a causal statement depends on which two things?

the quality of the design & the pattern of results

When conducting quasi-experiments, researchers are forced to give up what?

full experimental control

Parents of 3 aggressive children were instructed to record the number of violent behaviors each child exhibited during the hour before dinner. Then they were to begin using a positive reinforcement schedule with one of the children. A week later, they were to add the technique with the second child. Finally, a week later, they were to add the schedule for the third child. They were to continually record the aggressive acts each day.

What type of design is this following? y

multiple-baseline design

What describes a quasi-experimental design in which participants are not randomly assigned to groups and are tested only after exposure to the quasi-independent variable?

nonequivalent control group posttest-only design

What kind of design involves repeated measurement of a variable before and after some event?

interrupted time-series design

Dr. Ba is interested in whether joining a fraternity/sorority causes people to become more concerned about their attractiveness. She recruits a group of 55 freshmen (25 men, 30 women) who are planning to go through recruitment on her campus. After they join, she gives them a measure of attractiveness concern (the Body Concern Scale).

If Dr. Ba is interested in a causal relationship between joining a sorority or fraternity and attractiveness concern, why doesn’t she conduct a true experiment?

she is unable to randomly assign participants to join a fraternity/sorority

Dr. Ba is interested in whether joining a fraternity/sorority causes people to become more concerned about their attractiveness. She recruits a group of 55 freshmen (25 men, 30 women) who are planning to go through recruitment on her campus. After they join, she gives them a measure of attractiveness concern (the Body Concern Scale).

What does Dr. Ba need to do to her current design to make it an interrupted-time series design?

take measurements of body concern before & after joining a fraternity/sorority

In addition to measuring the group of participants who joined a fraternity/sorority, Dr. Ba decides to give the same measure to another group of 55 participants who decided not to join a fraternity/sorority. After conducting the study, Dr. Ba finds out that the people who joined a fraternity/sorority all saw a documentary on body image sponsored by the InterGreek Council the night before recruitment began.

Which threat(s) to internal validity are likely present?

history threat and selection threat

If Dr. Ba gave the same measure to another group of 55 participants who did not join the study, this could help address all but which of the following internal validity threats?

experimenter bias