9- The within-subjects experimental research design (chapter 9)-- (part of experimental Designs)

a _____________________ study is the ultimate in equivalent groups because

the group in one treatment condition is absolutely identical to the group in every other condition.

There is only one treatment group and each subject is given all values of the IV.

The comparison is made between scores obtained at different levels of the IV for same subjects.

within-subjects

a within-subjects design is often called a ______________________________ because the research study repeats measurements of the same

individuals under different conditions.

repeated-measures experimental design

2 kinds of within-groups designs:

1) concurrent measures

2) repeated measures (one after the other)

In a concurrent measure :

participants are exposed to all the levels of an independent variable at roughly the same times, and a single attitudinal or behavioral preference is the dependent variable.

In a repeated measure:

- most common approach for within-subjects designs. Every subject receives all values of the IV, and the participants own performance is the basis of comparison.

repeated measures design accomplish 2 things:

1. equating groups (by using same participants)

2. reducing within-group variance (by controlling for individual differences) Controlling for individual differences increases sensitivity and thus the ability to detect a treatment effect.

Error variance is reduced considerably because the participants become their own control (individual differences are eliminated)

A __________________________________________, or repeated-measures experimental

design, compares two or more different treatment conditions (or compares a

treatment and a control) by observing or measuring the same group of individuals in all of the treatment conditions being compared. Thus, a within-subjects design looks for differences between treatment conditions within the same group of participants. To qualify as an experiment, the design must satisfy all other requirements of the experimental research strategy, such as manipulation of an independent variable and control of extraneous variables.

within-subjects experimental design

T-F

In a within-subject, participants have multiples scores

True

What are two major sources of potential confounding for a within-subjects

design. (That affects Internal validity)

1-Confounding from environmental variables.

2-Confounding from time-related variables

When a within-subjects experimental study compares different treatments that are administered at different times, it must be concerned with threats to internal validity from environmental variables and time-related factors that may change systematically from one treatment to another and may influence the participants' scores.

1-Confounding from environmental variables.

Give an example of why environmental variables can be a confound and a threat to internal validity.

one treatment may be evaluated during the morning and another treatment during the afternoon. Any such variable may cause differences in scores from one treatment to another and, therefore, provides an alternative explanation for the differences between treatments.

How do we control environmental variables?

environmental factors such as the room, the experimenter, or the time of day, can be controlled by

(1) randomization, (2) holding them constant, or (3) matching across treatment conditions.

2- confounding from time-related variables

Why time-related variables can be a confound and a threat to internal validity?

A serious concern of within-subjects designs comes from the fact that the design often requires a series of measurements

made over time. During the time between the first measurement and the final measurement, the participants may be influenced by a variety of factors other than the treatments being investigated, and these other factors may affect the participants' scores. If this occurs, then the internal validity of the study is threatened because a change in a participant's score from one treatment to the next could be caused by

an outside factor instead of the different treatments.

2- confounding from time-related variables

identify 5 time-related threats to internal validity.

• History

refers to environmental/outside/external events, Events that occur in participants' lives at home, in school, or at work may affect their performance or behavior in different sections of the research study.

• Maturation

Any systematic changes in participants' physiology or psychology that occur during a research study and affect the participants' scores. The change in performance may not have been caused by the treatments but by maturation. (physical growth, cognitive development, wisdom, boredom, etc.)

• Instrumentation

A.K.A instrumental bias or instrumental decay) refers to changes in a measuring instrument that occur over time.

• Regression to the mean

Statistical regression, or regression toward the mean, refers to the tendency for extreme scores on any measurement to move toward the mean (regress) when the measurement procedure is repeated. Individuals who score extremely high on a measure during the first testing are likely to score lower on the second testing, and, conversely, individuals who score extremely low on a measure during the first testing are likely to score higher on the second testing.

• Repeated Testing (order effect)/(carry effect)

in previous treatment conditions.

Possible effects of the pretest on the posttest (the same measure taken at two different times). Progress on the test may be due partially to experience.

order effects are directly related to experience obtained by participating

Ex: School program to reduce racism among High School students. Before and after Sept. 11

What type of confounding from time-related variables can this be?

History

History is a threat to internal validity because any differences that are observed between treatment conditions may be caused by history instead of the treatments.

Ex: School program to help six-year old poor readers. Pretest = September; posttest = 6 months later.

What type of confounding from time-related variables can this be?

Maturation

Maturation is a threat to internal validity because observed differences between treatment conditions may be caused by maturation instead of the treatments.

Ex: a researcher observing a group of children and recording occurrences of aggressive behavior. In this situation, part of the measurement depends on the subjective interpretation of the observer, whose criteria may change from one time to another. As a result, the same behavior may be judged differently at different times. Notice that the changes in the participants' scores are not caused by the treatment but by a change in the measurement instrument (the researcher).

What type of confounding from time-related variables can this be?

Instrumentation

Instrumentation is a threat to internal validity because any observed differences between treatment conditions may be caused by changes in the measuring instrument instead of the treatments.

When is History, maturation, instrumentation and regression of a particular concern?

concern in research situations when the series of treatments extends over a relatively long time.

But

are not directly connected to experience in a previous treatment.

Ex: Your grade on an exam, for example, is based on a combination of knowledge and luck. Some of the answers you really know; others you guess. The student who gets the highest score on the first exam probably combines knowledge and good luck. On the second exam, this student's knowledge is still there, but luck is likely to change; thus, the student will probably score lower on the second exam.

High scores are more likely to go down & low scores are more likely to go up.

What type of confounding from time-related variables can this be?

Regression to the mean

Regression is a threat to internal validity because changes that occur in participants' scores from one treatment to the next

can be caused by regression instead of the treatments.

Ex: the experience of being tested in one treatment may explain

why the participants' scores are different in the following treatment.

What type of confounding from time-related variables can this be?

• Repeated Testing (order effect)

T-F

you should realize that time-related effects and order effects are only threats for within-subjects experiments that compare different treatments at different times. In studies that administer the different treatments all together, there is no opportunity for these threats to exist.

True

1. How many participants would be needed for a within-subjects experiment comparing four different treatment conditions with a total of 20 scores in each treatment?

a. 20

b. 40

c. 80

d. Cannot answer without more information

a. 20

2. In a within-subjects research study comparing different treatment conditions at different times, what kind of validity is threatened by factors that change over time, such as history and maturation?

a. Internal validity

b. External validity

c. Both internal and external validity

d. Neither internal nor external validity

a. Internal validity

3. In a within-subjects study that extends over a relatively long time, it is possible that there will be systematic changes in the participants' skills or knowledge during the time of the study. If these changes influence the participants' scores, causing scores at the end of the study to be

different from scores at the beginning, then what is the effect is called?

a. History

b. Instrumentation

c. Maturation

d. Regression toward the mean

c. Maturation

What is one of the major problem with a within-subject group design?

The Carryover effects

carryover effects/order effects

effects that one treatment may have on another treatment.

Exposure to one manipulation may produce persistent consequences influencing the participants response to subsequent manipulations.

The carryover effects include what?

practice effects, fatigue, boredom, interference.

What are the 2 possible solutions to deal with the carryover effect?

randomization & counterbalancing

The process of matching treatments with respect to time is called:

counterbalancing

Counterbalancing

different participants undergo the treatment conditions in different orders so that every treatment has some participants who experience the treatment first, some for whom it is second, some third, and so on. As a result, the treatments are matched, or balanced, with respect to time.

This procedure disrupts any systematic relationship between time and the order of treatment conditions, and thereby eliminates potential confounding from time-related threats or order effects.

With two treatments, for example, half of the participants begin in treatment I, and then move to treatment II. The other half begin in treatment II, then receive treatment I. As a result, the two treatments are matched; for both treatments, 50% of the participants experience the treatment first and 50% experience the treatment second.

This is an example of :

Counterbalancing

So that different participants undergo the treatment conditions in different orders so that every treatment has some participants who experience the treatment first, some for whom it is second, some third, and so on. As a result, the treatments are matched, or balanced, with respect to time.

Ex: examining the effect of swearing in response to pain. In one

condition, the participants were told to shout their favorite swear words while experiencing a painful stimulus (ice water) and in the second condition they shouted a neutral word. Half of the participants started with the swearing condition and half started with the neutral word condition. After a brief rest, the two groups switched words. Thus, the two conditions (curse and neutral) were _________________________, with half of the participants swearing first and

half swearing second.

counterbalanced

What is the purpose for counterbalancing?

The purpose of counterbalancing is to eliminate the

potential for confounding by disrupting any systematic relationship between the order of treatments and time-related factors.

______________________________ refers to all possible treatment orders are used equally - equal numbers of participants in each treatment condition.

present the treatments in every possible sequence

Complete counterbalancing

Ex: with four treatment conditions, there are 4

4= 4 x 3 x 2 x 1 = 24 different sequences.

these 24 different sequences are :

The complete counterbalancing

How many different sequences there are in a treatment condition of 5?

calculate the complete counterbalanced.

5 ---> 5 x 4 x 3 x 2 x 1 = 120 different sequences

A simple and unbiased procedure for selecting sequences is to construct a:

Latin square

To create a Latin square for four treatment conditions, start with a 4 3 4 matrix and fill it in with the letters

A, B, C, and D, as follows:

List the letters ABCD in order in the top row of the matrix. To create the next row, simply move the last letter in line to the beginning. This creates DABC for the second row. Continue moving the last letter to the beginning of the line to create

each new row.

1. What is the effect of increasing the time between treatment conditions in a within-subjects experiment?

a. It decreases the threat of time-related history effect.

b. It decreases the threat of the order effect fatigue.

c. It decreases the threat of a time-related maturation effect.

d. None of the other options is an effect of increasing time between treatments.

b. It decreases the threat of the order effect fatigue

2. For a within-subjects study comparing two treatments, A and B, a researcher expects that practice in the first treatment will improve the participants' scores in the second treatment. If the order of treatments is counterbalanced, then what scores will be influenced by the practice?

a. Scores in treatment A but not in treatment B.

b. Scores in treatment B but not in treatment A.

c. Scores in treatment A for half the participants and scores in treatment B for half the participants.

d. Practice will not influence the scores because the treatments are counterbalanced.

c. Scores in treatment A for half the participants and scores in treatment B for half the participants.

3. Which of the following describes a completely counterbalanced within-subjects experiment?

a. Each group receives a different treatment.

b. Each participant receives each treatment in the same order.

c. A series of treatments is presented in every possible sequence.

d. Participants receive a random order of treatment conditions.

c. A series of treatments is presented in every possible sequence.

What are some of the advantages of a within-subject design

- fewer participants required

- greater sensitivity to treatment effect due to elimination of variance caused by individual differences (eliminates all of the problems based on individual differences)

- good when participants are hard to find

- each participant acts as his own control

- very powerful design under suitable conditions.

What are some of the disadvantages of a within-subject design

- not suitable when carryover effects

(fatigue, practice, boredom) are permanent.

each participant often goes through a series of treatment conditions, with each treatment administered at a different time. Whenever the treatments occur at different times,

there is an opportunity for time-related factors, such as fatigue or the weather, to influence the participants' scores. For example, if a participant's performance steadily declines over

a series of treatment conditions, you cannot determine whether the decline is being caused by the different treatments or is simply an indication that the participant is getting tired.

- participant attrition may be a problem.

(some of the individuals who start the research study may be gone before the study is completed.)

When do we know if we have to choose btw a between or within subject research design experiment?

see p.231

_____________________________ design attempts to eliminate the problems associated

with between-subjects experiments (individual differences) and the problems associated with within-subjects experiments (order effects).

matched-subjects

Ex: a 22-year-old white female with an IQ of 118 who was in one group would

be matched with another 22-year-old white female with an IQ of 118 in another group.

This is a example of what design?

matched-subjects design

1. What is measured and removed to reduce the variance in within-subjects design compared to a between-subjects design.

a. The individual differences

b. The carry over effects

c. The progressive error effects

d. The instrumentation effects

a. The individual differences

2. Which of the following is an advantage of the between-subjects design versus the within-subjects design?

a. It generally requires fewer participants.

b. It usually is a more sensitive test (more likely to detect a treatment effect).

c. It eliminates the risk of order effects.

d. It eliminates potential problems that may be caused by individual differences.

c. It eliminates the risk of order effects.

3. For an experiment that compares two treatment conditions with ten scores in each treatment, which design would require fewer subjects?

a. Between-subjects design

b. Within-subjects design

c. Matched-subjects design

d. All would require the same number of subjects.

b. Within-subjects design

1. What is the appropriate hypothesis test for a within-subjects design comparing mean differences for three treatment conditions?

a. An independent-measures t test

b. A repeated-measures t test

c. A repeated-measures analysis of variance.

d. A chi-square test for independence

c. A repeated-measures analysis of variance.

2. Which of the following is an advantage of a two-treatment within-subjects design compared to a multiple-treatment design?

a. There is a reduced risk of participant attrition.

b. There is a reduced risk that time-related factors influence the data.

c. It is easier to counterbalance a design with only two treatments.

d. All of the above are advantages of a two-treatment design.

d. All of the above are advantages of a two-treatment design.