FMP: CHAPTER 7 Experimental Designs

Experimental research design

• seeks to determine if a specific treatment influences an outcome.

• The researcher assesses this by providing specific

treatment to one group and withholding it from another

and then determining how both groups scored on an

outcome.

• Experiments include true experiments, with the

random assignment of subjects to treatment

conditions, and quasi-experiments that use

nonrandomized assignments (Keppel, 1991).

True experiments, random assignment, nonrandomized assignments

• Experiments include ___, with the

___ of subjects to treatment

conditions, and quasi-experiments that use

___ (Keppel, 1991).

BASIC PRINCIPLES OF EXPERIMENTAL METHOD

• To translate the Method of Difference into an

experiment, we need to formulate a specific, testable

question or hypothesis, such as, ‘Does stress impair

memory for events?’

• To answer it, we manipulate one variable (the

independent variable, or IV), and measure the effects

of these manipulations on another variable (the

dependent variable, or DV), while keeping all other

variables as constant as possible.

independent variable or IV

• Manipulated Variable

To answer it, we manipulate one variable ___ and measure the effects of these manipulations

dependent variable or DV

variable in a study or experiment that is observed and measured to determine the effect of changes or manipulations in another variable ___ while keeping all other variables as constant as possible

The Peril of Confounding Variables

• In designing an experiment, the tricky bit is to ensure

that the independent variable is the only thing that

varies systematically between the groups.

• Variables that have unwanted influences on our

experimental results are called confounding

variables.

confounding variables.

• Variables that have unwanted influences on our

experimental results are called ___.

Between-groups design (between-subjects design)

• is an experimental design in which each condition in

  the experiment is performed by a different group of participants.

Within-subjects designs (repeated measures designs)

• is an experimental design in which each participant

takes part in more than one condition of the study.

Matched-pairs design

• is an experimental design in which participants are carefully matched on variables that might be relevant to the research (such as age, socioeconomic status, intelligence, etc.)

Post-test only/control group design

is an experimental design in which participants are allocated randomly to different conditions and given different treatments.

Pre-test/post-test control group design

the subjects are allocated to groups and are measured both before some manipulation and after the manipulation.

Solomon four-group design

is an experimental design in which participants are allocated randomly to different groups, given different treatments (different levels of an independent variable), and then tested in some way, in the expectation that the different treatments will have affected their behavior in some way. 

ADVANCED EXPERIMENTAL DESIGNS

• Longitudinal design

• Cross-sectional design

• Multifactorial design

• Mixed design

Longitudinal design

is a repeated-measures design that is often used in developmental research. A group of participants is tested repeatedly over time (e.g. at different ages).

Cross-sectional design

is a between-groups design that involves using different groups of participants to represent different points in development (e.g. different ages).

Multifactorial design

is an experimental design in which the effects of more than one independent variable are investigated within the same study. ____ permit investigators to study how variables might interact with each other.

Mixed design

is an experimental design that combines within-subjects and between subjects (e.g. the effects of sex and alcohol on memory performance could be investigated with a mixed design.

COMPONENTS OF AN EXPERIMENTAL STUDY METHOD PLAN

• Participants

• Variables

• Instrumentation and Materials

• Experimental Procedures

• Threats to Validity

• The Procedure

• Data Analysis

• Interpreting Results and Writing a Discussion Section

Participants

• Readers need to know about the selection, assignment, and the number of ___ who will take part in the experiment.

• Often investigators aim to recruit a study sample that shares certain characteristics by formally stating specific inclusion and exclusion study criteria when designing their study.

Variables

1. Identify the IVs in the experiment and how they will be manipulated in the study.

2. Include a manipulation check measure that evaluates whether your study successfully manipulated the independent variable(s) of interest. A manipulation check measure is defined as a measure of the intended manipulated variable of interest.

3. Identify the dependent variable or variables (i.e., the outcomes) in the experiment.

4. Identify other ___ to be measured in the study. Three categories of ___ are worth mentioning.

Instrumentation and Materials

The experimental study plan calls for a thorough discussion

about the ___ used - their development, their items,

their scales, and reports of reliability and validity of scores

on past uses.

• 1. Thoroughly discuss the ___ used for the

  manipulated variable(s) of interest.

  2. Often the researcher does not want participants to know

  what variables are being manipulated or the condition

  they have been assigned to (and sometimes what the

  primary outcome measures of interest are).

Experimental Procedures

The specific _____ also need to be identified.

1. Identify the type of experimental design to be used in the proposed study.

2. Identify what is being compared in the experiment.

3. Provide a diagram or a figure to illustrate the specific research design to be used.

Threats to Validity

Experimental researchers need to identify potential threats to the internal validity of their experiments and design them so that these threats will not likely arise or are minimized.

Internal validity threats

are experimental procedures, treatments, or experiences of the participants that threaten the researcher’s ability to draw correct inferences from the data about the population in an experiment. For example, history, maturation, testing, instrumentation, etc.).

External validity threats

arise when experimenters draw incorrect inferences from the sample data to other persons, other settings, and past or future situations. For instance, interaction of selection and treatment, interaction of setting and treatment and interaction of history and treatment.

The Procedure

Step 1. Administer measures of the dependent variable or a

variable closely correlated with the dependent variable to

the research participants.

Step 2. Assign participants to matched pairs based on their

scores on the measures described in Step 1.

Step 3. Randomly assign one member of each pair to the

experimental group and the other member to the control

group.

Step 4. Expose the experimental group to the

experimental treatment and administer no treatment or

alternative treatment to the control group.

Step 5. Administer measures of the dependent variables

to the experimental and control groups.

Step 6. Compare the performance of the experimental

and control groups on the post-test(s) using tests of

statistical significance.

Step 1.

Administer measures of the dependent variable or a

variable closely correlated with the dependent variable to

the research participants.

Step 2.

Assign participants to matched pairs based on their scores on the measures described in Step 1.

Step 3.

Randomly assign one member of each pair to the experimental group and the other member to the control group.

Step 4.

Expose the experimental group to the experimental treatment and administer no treatment or alternative treatment to the control group.

Step 5.

Administer measures of the dependent variables to the experimental and control groups.

Step 6.

Compare the performance of the experimental and control groups on the post-test(s) using tests of statistical significance.

Data Analysis

1. Report the descriptive statistics.

2. Indicate the inferential statistical tests used to examine the hypotheses in the study.

3. For single-subject research designs, use line graphs for baseline and treatment observations for abscissa (horizontal axis) units of time and the ordinate (vertical axis) target behavior.

line graphs

For single-subject research designs, use ___ for baseline and treatment observations for abscissa (horizontal axis) units of time and the ordinate (vertical axis) target behavior.

abscissa

For single-subject research designs, use line graphs for baseline and treatment observations for ____ (horizontal axis) units of time and the ordinate (vertical axis) target behavior.

ordinate

For single-subject research designs, use line graphs for baseline and treatment observations for abscissa (horizontal axis) units of time and the ___ (vertical axis) target behavior.

Interpreting Results and Writing a Discussion Section

• Interpret the findings in light of the hypotheses or research questions and to draft a discussion section. In this interpretation, address whether the hypotheses or questions were supported or whether they were refuted.

• Address whether the results might have been influenced by unique strengths of the approach, or weaknesses (e.g., threats to internal validity), and indicate how the results might be generalized to certain people, settings, and times.

• Finally, indicate the implications of the results, including implications for future research on the topic.