Module 9-13 (terms are not really in order…)

Single-case Designs

Single-case Designs

Can take two forms

1. Case study

2. Single-case experiment

Case study

• An in-depth investigation of a single case of interest

• Involves simple observation rather than manipulation of a variable

Single-case experiment

• A design in which single participants serve as their own control group to study the effects of a manipulation or treatment

• A participant acting as their own control

  • We can also make comparisons of behavior in the same individual over time

• Often used as a pilot study to verify methodology, reliability of data gathering, and direction of results before spending the time and money to do a full-scale study

Single-case experiment Advantages

• Decrease in error variance (variability in scores that is not due to the independent variables)

• Due to less error variance they increase the power to detect a true difference compared to studying multiple participants in an experimental design

• Increases statistical power

Single-case experiment Disadvantage

Inability to generalize the results to others

Variation of the Single-Case Experiment

A group of participants who share a single attribute can be investigated using the same methods as we use to study a single participant

• For example, the population of soldiers who have been diagnosed with PTSD

  • In these cases, the “group” is already formed because we are not randomly assigning the participants to a group

• Could measure the severity of their symptoms before and after a treatment

  • This is not a quasi-experiment, because we still have only one group

    • If we began comparing them to soldiers without PTSD, then we would have a quasi-experiment instead

Uses of Single-case Designs

• When measures need to be added or refined

• Used to assess behavioral interventions for children with behavioral disorders

  • The children come into the study with a specific diagnosed condition, and they cannot be randomly assigned to disorders

• A special type of this design is used in functional behavioral assessment (FBA)

Functional Behavioral Assessment (FBA)

• Used to assess problem behaviors in school children

  • Often in response to government policies

• Usually conducted prior to attempting an intervention

  • As the goal of this is to identify environmental stimuli that might trigger or maintain a problem behavior

  • This information can then guide the development of an intervention that lessens or avoids the problem triggers and reinforcers

• Can be conducted in either non-experimental ways (observation) or in experimental ways by

  • manipulating environmental variables, such as noise, suspected of being related to problem behavior

• Is usually conducted with a particular student instead of a number of students

Types of Single-case Designs

1. Reversal Designs

2. Multiple Baseline Designs (Case Series Design)

Reversal Designs (ABAB design)

• Periods of treatment and no treatment are alternated

  • We take an initial baseline of a targeted behavior

  • Use a treatment to improve the behavior

  • Reverse or remove the treatment to allow the behavior to go back to the original level of the targeted behavior

  • Then reinstate the treatment

• Researchers expect to see that behavioral improvement will follow treatment, and once treatment is withdrawn, the behavior will go back to the baseline or pre-treatment level

  • It is necessary to resume the treatment since we previously found it to be helpful so we end the study with improvement in behavior for ethical reasons

Reversal Designs (ABAB design) Advantages

• Have the opportunity to use systematic observation of behavior as it occurs in everyday life

• Can see the alterations of behavior at each measurement point

  • And we can document any minor variabilities

• Can monitor not only the extent to which the behavior changes,

  • But also the time it takes for the behavior to change during the course of the treatment

• Is brief in terms of initiation of treatment and completion of data analysis

  • We have our baseline measurement, and we can easily and quickly measure the behavior on a daily or other regular basis

• The ability to use each participant as their own control and observe the replication of any treatment effects within the two treatment portions of the study are additional benefits

  • Even when subtle changes in behavior are observed, we can then see whether we should use this same treatment approach for evaluation in a larger scale study

Reversal Designs (ABAB design) Disadvantages

• There is no control condition to determine whether a simple observation of behavior is altering the behavior

• Often have carry-over effects of the original treatment

  • so the behavior does not go back to baseline, but remains at post-treatment levels because the participant is not unlearning the newly learned skills

• Ethical concerns related to withholding a known beneficial treatment, even briefly

  • Once we know that a treatment is successful in changing a “bad” behavior into a “good one,” it may not be ethical to remove the treatment and wait for the condition being treated to re-emerge

Reversal Designs (ABAB design) Pros & Cons

Pros:

1. Observe everyday behavior

2. Brief process

3. Clear process for data collection and analysis

4. Participants serve as own control

5. Replicate treatment effects within a single study

Cons:

1. No control condition

2. Possible carry-over effects

3. Ethical concerns about reversing treatment

Multiple Baseline Designs (Case Series Design)

• A research design in which multiple people are studied with treatment beginning at different times

• The method allows for the determination of the impact of the timing of treatment

• This design controls for the fact that an individual’s behavior may change over time just because we start to pay more careful attention to it

• We are again observing behavioral change over time in individuals

  • However, this time, we stagger the time at which the intervention takes place for each of several participants

Multiple Baseline Designs (Case Series Design) Process and example

1. The baseline testing and observation of the behavior would begin at the same time with each of the three participants, labeled A, B, and C.

2. Baseline behavior would be systematically documented for a period of time.

3. During baseline, the measurement of the behavior in question could be monitored every day for

   1. Five days in Participant A

   2. Ten days in Participant B

   3. Fifteen days in Participant C

4. Participant A would begin the treatment phase of the study at day six.

   1. The other participants would still be in the baseline pre-treatment stage.

5. After more baseline observations, participant B would then be exposed to the treatment followed by participant C.

6. The initiation of treatment after baseline documentation would continue until all participants in the study were exposed to the treatment.

   1. Because we are watching each participant for different amounts of time, we should be able to compare the participants and see if the amount of time being watched influences their outcomes.

7. As with the reversal design, the treatment would be withdrawn from all participants after being administered for the same amount of time.

8. The expectation would be that for each participant, the behavior of interest would not change until the intervention began and the behavior would go back to baseline after treatment withdrawal.

The Cohort quasi-experimental Design

• The key feature of this design is that none of the participants have a feature or experience of interest at the beginning of our measures

  • Can take the form of birth era (the period of time to which anything is assigned, as in Millenials and Gen z) or any group that enters a “system” at a given point in time and is followed over time, such as grade school children, college freshman, nursing students, military enlistees, etc

    • For example, a birth cohort (e.g., Millenials) can be followed over time with multiple measurements to get an understanding of how responses to life events such as graduation from high school or birth of a first child might be different from other cohorts (e.g., Generation Z)

• Two Types:

  • 1. Prospective

  • 2. Retrospective

Birth era

The period of time to which anything is assigned, as in Millenials and Gen z

Prospective Cohort Design

• Cohort is identified first

  • The Framington Heart Study, begun in 1948, has followed initially healthy participants over time to see what factors contribute to the development of cardiovascular disease.

    • The study is now looking at its third generation of participants.

• Might have to wait a long time before an outcome of interest occurs

Retrospective Cohort Design

• Help us understand rare or unusual events

  • We could consider Gulf War veterans as our cohort and investigate the differences between veterans exposed to neurotoxins and those who were not to see the effects of neurotoxin exposure on health measures.

    • Have some challenges, might not know exactly who was and was not exposed and whether exposure differed among the participants

      • We don't know what other health habits, like smoking, might be present in our groups.

The Case-control Quasi-experimental Design

• Behavior is observed first, then measurement of related variables is documented retrospectively

  • Researchers first take note of a behavior (such as adverse psychological responses to COVID-19)

  • Then they measure potential causes for the behavior, such as a pre-existing diagnosis of obsessive-compulsive disorder (OCD)

Time-Series Quasi-experimental Designs

• Involves repeated measures over time

• Common type of design in developmental psychology

• Different from Repeated Measures in that we do not have random assignment to groups

• This along with cross-sectional designs are often used to investigate the changes associated with aging

Interrupted Time-Series Design

The study of a variable over time, with the interruption being a treatment procedure

• A one-group design in which the "interruption" is a treatment

• The time-series refers to the periods before and after the treatment

• This design features:

  • 1. Several pre-tests taken over time,

  • 2. An intervention,

  • 3. Multiple post-tests taken over time

When is Interrupted Time-Series Design Useful?

• This design has been used in place of a true experimental design when the manipulation of behavior is not feasible or not ethical.

• Used when we need to evaluate changes in a population of interest

• Example: The Centers for Disease Control and Prevention (CDC) reports that heart disease is the number one cause of death in adult women

  • Using this design, you could track the rate of heart disease among women in a community before and after a public health campaign providing information about how to prevent heart disease.

  • The campaign serves as the "interruption."

  • If heart disease rates decrease significantly after the campaign, you could argue that it was effective.

Multiple Time-Series Design

A type of time series design featuring the addition of a control group of participants who are not exposed to the treatment, but have the same number of measurements of the same variables.

• Combines aspects of the non-equivalent control group design and the times series design

• Extending the number of measures compared to the non-equivalent group design improves control of some of the threats to internal validity

• These designs differ from the interrupted time-series in that there is now more than one group

  • Each group is evaluated on a dependent measure multiple times before and after an intervention

    • But one group does not have exposure to the intervention

  • The comparisons are made not only within the experimental group before and after the intervention

    • But also between the experimental and the control groups

Program Evaluation (What is program evaluation?: A Brief Introduction on youtube)

• Uses the methods of science to determine the effectiveness of a program

• Has the inclusion of a value judgment in response to the data

• Variables like the "success" of a program should be operationalized at the very beginning

• Factors commonly assessed:

  • 1. The need for the program or service

  • 2. The design of the program or service

  • 3. How the program implements its service delivery

  • 4. The outcomes of the program or service

  • 5. The efficiency of the program or service

• Quantitative, qualitative, or mixed methods can be used

  • The design decision will depend on:

    • the type of the program,

    • the questions the researcher is trying to answer,

    • the data used to determine program success

Best Practices in Open Science: Do You Really Know How Your Groups Differ?

• This means that although we think we know what our independent variable is, we really don't know for sure.

• Careful researchers try to account for these types of confounding variables in their studies, and use specific statistical methods to rule them out as factors

  • However, there is always the possibility that factors we don't know about are still important to the relationships we're proposing.

• We must be extremely cautious and humble regarding the reporting of our findings

• Stating what we do and do not know about our variables and their relationships as completely and clearly as possible is not only honest,

  • But it also helps other researchers attempt to replicate and extend our findings

Multi-Method Analysis

Qualitative methods can be used alone or with Quantitative methods

Address Typical Power Differentials Between Researcher and Participants (Commonalities in Qualitative Methodology)

• Researchers try to match the gender, race, ethnicity, first language, and socio-economic status between the participant and those who represent the “face” of the study (in-depth interviewers, focus group facilitators, etc.)

• Real or perceived hierarchy provides a potential barrier between the researcher and the participants and perhaps increases the risk of social desirability bias

  • a kind of response bias wherein study participants maximize what they believe are "good" responses and minimize what they believe to be "bad" or undesirable responses

Address Diversity Holistically

It is noteworthy that methodologies which overtly acknowledge the potential for bias in the scientific method tend to be more qualitative in nature.

Qualitative Theories

Three approaches to Qualitative work:

1. Grounded Theory

2. Ethnography

3. Phenomenology

Grounded Theory Approach

• Theory “emerges” from the data itself

• A “constant comparative method” wherein new data is matched to existing data throughout the study to look for commonalities and emerging themes

• Rather than citing a theoretical framework and hypothesis in advance of the research

  • This theory approaches the research with general ideas and/or topics of inquiry and collects research until data saturation is reached

• Sampling and analysis are not distinct, but rather iterative and aimed to “go where the questions are”

• Is often the framework of choice for exploratory questions where there is a significant gap between theory and empirical understanding of the topic

• Seeks to show the depth of the phenomena under study

• Seeks to define and differentiate themes from each other

Ethnography

• Research questions determined by in-depth field work

• Systematically studies patterns between people and cultures

• Often bridges the gap between the fields of psychology, sociology, and anthropology

• Is relational and contextual

• It delves deeply into the values, shared understandings, and relationships within a community

• Can enlighten social change by providing a platform for marginalized voices and encouraging social action

• Usually involves immersion into the setting

• Is relatively unstructured as compared to grounded theory

• Heavily relies on participant observation

• Is typically inductive

• Highly relational and iterative

• Provides an opportunity to study a cultural topic in depth

Phenomenology

• Relationship between people and objects mutually influence each other to create theory

• Rooted in philosophy

• Reject data and themes altogether and prefer to collect capta

  • Collection of consciousness and objects that influence each other

• Most often used early in the research process, with other approaches, and as part of the “discovery” phase of research

• Seeks to holistically understand phenomena and remains flexible to meet this lofty goal

• Maximizes the point of view of the researcher

• The understandings of the participant are assumed to be real and valid, and they are analyzed with less interpretation, not more

• Can be an excellent choice for social justice work

Capta

Collection of consciousness and objects that influence each other

Limitations of Qualitative Research

• The widespread erroneous assumption that these methods are less rigorous

• That the research quality is dependent on the skills of the researcher and easily biased by the researcher’s own perspective

• That they are less well understood and that their rigor is more difficult to assess

  • This is avoided by honest and authentic attempts at making data collection and analysis transparent by publishing the codebook

• That it is particularly susceptible to social desirability bias

  • Wherein study participants maximize what they believe are “good” responses and minimize what they believe to be "bad," or undesirable, responses

• The high potential for confirmation bias

  • The researcher is more likely to find/analyze the data in ways that support their hypothesis rather than contradict it

Ex-post-facto

A research study that measures behaviour after the event in question, also known as “after-the-fact”

• For example,

  • when evaluating a new vaccine, we would typically exclude pregnant women from participating in the study because of the potential for damaging the fetus. However, if a woman had already taken the vaccine prior to finding out that she was pregnant, she is now in an ex-post-facto group that we can study ethically

Triangulation

Using more than one method of data collection to provide validity to the research findings.

Dualistic thinking

a way of thinking that is conceptually divided into two opposing thoughts or categories

Correlational Research

• Both a statistical technique and a research method

• Research designed to determine whether an association exists between two variables

• Can only indicate that two variables are related

• “associated,” “related,”

Positive Correlation

As A increases, B increases

Negative correlation

As A increases, B decreases

Zero correlation

There is no relationship between A & B

Independent groups design

• Also known as a between subjects design

• Participants are non-overlapping between groups

• You’re either in the treatment group OR in control group

  • Not in both groups, just one

• Simplest form of this only has 2 groups

Random-Groups design (random assignment)

• Each participant in the sample has an equal chance of being assigned to each of the groups

• Allows researchers to establish causality

• Only assigned to one of the groups (non-overlapping/independent)

• Equates groups on potential confounding (or extraneous) variables

• It is imperative that researchers ensure that their conditions are equivalent

Experimental Condition

• Intervention or treatment

  • Give them drug pill

• The thing you think causes your effect

Control/Baseline Condition (Business-as-usual)

• Absence of treatment

  • Give them placebo pill

• Without the thing you think is the cause

Formal Experimental Designs

• Only difference between groups is what is the variable we hypothesize to be the cause (X) of some outcome (Y)

  • Why X might have caused Y

• Can establish causality

Independent Variables (IV)

• Manipulate

• Levels of this are the Experimental and Control conditions

  • The presence or absence of what you’re testing (ex. drug and placebo)

• Only thing determining the difference between the 2 groups is the experimenter nothing else

• What we think is the cause

• Should be the only thing that differs systematically between your 2 or more groups

Dependent Variables (DV)

• What we Measure

• Outcome

• Can be categorical or quantitative

• What we think is the effect

How Many Participants Do You Need in Your Study? (Independent groups design)

Depends on the effect size and desired power

Effect size

• How much effect does our independent variable actually have on our dependent variable

• How big of an effect you are expecting to find

• If that effect exists, how sure or likely you want to be define that

• If the group difference is very big and obvious, we don't need as many people to be able to detect it

  • As when the group difference is small.

Power

• The probability of detecting an effect if it actually exists

  • The chance of detecting an effect that is there

• A researcher's ability to find an effect size of interest if it actually exists

  • Need to collect data from enough participants to detect an effect if indeed an effect is present

• Usually in terms of percent

• If effect size is believed to be small you want more people

• If effect size is believed to be big, you don’t need as many people

Representative samples

• Have characteristics representative of the broader population

• Important to make sure results generalize to other related groups of individuals

Confounding Variables

• A ‘third variable’ that may differ between groups

• Problematic if they systematically covary with the IV and DV

• Decrease internal validity(the ability to make causal claims)

Confounding Variables: Examples

• Differences in the environment (ex. room temp)

• Differences in the individuals (e.g., differential motivation for mathematics)

• Differences in the researcher (e.g., researcher cues)

1.Single-blind, 2.Double-blind

1. Participants don’t know what condition/group they’re in(they should never know)

2. Participants and Researcher both don’t know what condition/group participant is in (This one is considered Gold Standard)

• can reduce expectancy effects

Matched Groups (9.6)

• Groups are matched on some variable of importance (e.g., age, gender)

• Ensure groups are equal to make causal claims

• Increases internal validity

Internal Validity

• Deals with experimental control

• Extent to which we can be sure the IV is the cause of the DV

External Validity

• Deals with generalizability

• Extent to which we can be sure we can generalize our results to different populations

Threats to Internal Validity

• Confounds

• Experiments

The Golden Rules for Experiments

• Random assignment to condition

  • Every member of a group should have an equal probability of being assigned to any group

• Make everything identical between conditions other than the variable(s) you are intentionally manipulating

Repeated Measures Design

• Participants serve as their own controls

• Participants experience all levels of the independent variable (IV)

• Measure the dependent variable more than once (at each level of the IV)

• Also called within-subjects design

  • Because you’re comparing within a person

Example of a Repeated Measures Design

• Independent variable: Text size

  • Levels: small text (5 point font) and large text (16 point font)

• Dependent variable: Reading speed

  • Read a passage with small font and measure reading speed

  • Read a passage with large font and measure reading speed

Advantages of Repeated Measures Designs

• Fewer participants needed

• More power to detect significant results

• Can measure change over time

  • Such as how seasons affect people’s mood

Disadvantages of Repeated Measures Designs

• Unique situations than can only be experienced once

• Order effects can decrease internal validity

Types of order effects

• Practice effects

• Fatigue effects

• Carryover effects

• Sensitization effects

Practice Effects

• When performance is better in subsequent levels of the independent variable due to experiences in previous levels

  • Could happen in a study evaluating whether learning to play an instrument from an instructor or watching recordings is more effective

Fatigue Effects

• Performance decreases on subsequent levels of the IV

• Participants may get tired or bored since they are repeatedly experiencing the same/similar tasks

Example of Practice Effect

In a study on reading speed, participants read short passages in small font (5 point) or large font (16 point). They read 10 passages in the small size font then the same 10 passages in the large font.

• Faster reading speed during the large font size reading could be example of this

Example of Fatigue Effect

In a study on reading speed, participants read short passages in small font (5 point) or large font (16 point). They read 10 passages in the small size font then the same 10 passages in the large font.

• Slower reading speed during the large font size could be example of this

Carryover Effects

Effect of one level of the IV spills over into the next level

Carryover Effects Examples

• Experiencing one dosage level of a medication could have lasting effects on the mood that carry over into the next dosage

• Experiencing a hot room could have lasting effects on mood that carry over to evaluation in a cold room

• Coffee will taste different before you eat cake compared to after you eat cake

Sensitization Effects

• Participants act differently on the next level of the IV after exposure to one or more other levels of the IV

• Specific type of carryover effect in which an experience in one condition makes participants extra sensitive to the manipulated variable in the next condition

Sensitization Effects Example

Studying the effect of room lighting on reading speed

• After reading in a dark room, reading in a bright room could be harder as eyes adjust

Counterbalancing

• Balance order effects across different conditions to which participants are exposed

• This is done to control order effects

• Two types of this

  • Within-subjects

  • Across-subjects

Within-Subjects Counterbalancing

• All participants experience all possible orders

  • All participants go through each level of the independent variable, just in a different order

Two types of this:

1. Reverse Counterbalancing

2. Block Randomization

Across-Subjects (Between-Subjects) Counterbalancing

Participants only experience one of the orders

• There are two conditions A and B

  • Have half of the participants experience A then B

  • Have other half of participants experience B then A

Two types of this:

1. Complete Counterbalancing

2. Partial counterbalancing

• Each condition should appear in each ordinal position (first, second, third, etc.) equally often

• Each treatment should also precede and be followed by every other condition an equal number of time

Reverse Counterbalancing

• Type of Within-Subject Counterbalancing

2-3 conditions: Each person experiences all conditions in one order, and then in a reversed order

Block randomization

• Type of Within-Subject Counterbalancing

4+ condition: Each person experiences all possible orders of the conditions, in random sequence

Complete Counterbalancing

• Type of Across-Subject Counterbalancing

2-3 Conditions: All orders are used, each participant only experiences one order

Partial counterbalancing

• Type of Across-Subject Counterbalancing

4+ Conditions: in which a subset of possible orders are presented and each participant experiences one subset

Other Ways to Control for Order Effects

• Different materials and stimuli

• Washout Period:

  • A brief interval between levels of the independent variable to prevent order effects

Washout Period

A brief interval between levels of the independent variable to prevent order effects

Matched Groups Design

• Researchers often use this method if they have a limited number of participants, and there is great variability among participants in a variable that is relevant to the dependent variable

• Get pairs of participants who are really similar in a lot of ways (ideally identical twins) and get them to experience the 2 different levels of the IV

• Pairs of participants experience the different levels of the independent variable—just as in a between-subjects design

• Participants are matched on a variable relevant to the dependent variable

• Advantages of between- and within-subjects designs

Basic Experimental Designs

One independent and one dependent variable

Complex Experimental Designs

• Multiple variables

• Factorial designs: multiple independent variables

• Multivariate designs: multiple dependent variables

Factorial Designs

• Multiple independent variables

• When it’s this design, tells you that the predictors are categorical

  • In a study you have a drug X drug Y and a placebo

    • Those are categories

• The reason we would have a study

  with multiple predictors is almost always because we think that, the effect of one of the predictors on the outcome might depend on what's happening with the other predictor

  • Has main and interaction effects

Multivariate Designs

Multiple dependent variables

2 x 2 Between-subjects Factorial Design

• Two independent variables in the same study

• Different participants in the various conditions

• 4 groups

• (# x #) # refer to the number of levels in each factor

Example of a Factorial Design

How does playing video games influence cognitive functioning?

• IV#1: Game type (Adventure vs. RPG)

• IV#2: Time played (3 hrs vs. 6 hrs)

• Confirmatory hypotheses:

  • The researcher specifies what effects they expect to find

• Exploratory hypotheses:

  • The researcher does not specify what results will be found

You think there's something about the effect of one of them might differ, depending on the other one

Confirmatory/Directional Hypotheses

• The researcher specifies what effects they expect to find

• You have a very specific effect that you think you’re gonna find

Exploratory Hypotheses

• The researcher does not specify what results will be found

• I think, there might be interaction between these 2 predictors, But I'm not sure what it's gonna be

An Example of a Factorial Design: Contingency Table

• Factorial designs have main effects and interaction effects

• This shows cell means and marginal means

Main Effects in Factorial Designs

• Compare all differences between levels of one independent variable across levels of the other

• When we just isolate the effect of one of the predictors

  • Just comparing the rpg game vs the adventure game (ignoring time spent playing)

  • Or just comparing playing for 3hrs vs playing for 6hrs (ignoring the type of game that was played)

• The differences in mean scores between the levels of each independent variable across the values of the other independent variable (i.e., the marginal means)

• We state that there are two main effects, a main effect for “game type” and a main effect for “total time”

Interaction Effects in Factorial Designs

• Describing main effects in reference to each other

• Combined Effects of independent variables

• An effect of one IV on the DV depending on the level of the other IV

• Every participant in each cell is exposed to one level of both independent variables. For example, the ten participants in each cell are exposed to one level of the game type variable and one level of the total time variable. You could say that both independent variables come together in that cell. Researchers refer to this as an interaction. The meaning of the prefix “inter” is between or among, so we can say there is an “inter”action between the two variables that takes place in that particular cell. It’s as if you could not describe one variable without the other

Example: Ice Cream Preference

• Ice cream you prefer on its own

  • = Main Effect

• Preference for what ice cream you want, depending on what you're having (with Pie)

  • ________ ________

Other Variations of Factorial Designs (Specifically 2×3)

• More than two levels for the independent variable

• A main effect does not clarify which levels differ

• Post-hoc tests:

  • Allow researchers to compare the means of the levels of the IV

Post-Hoc Tests

Allow researchers to compare the means of the levels of the IV

Higher-Order Factorial Designs

• Have additional main effects and interactions

• A design with three or more independent variables (2×2×2)

  • 2 (three hours versus six hours)

  • x 2 (adventure versus RPG)

  • x 2 (low versus high delay of gratification) design has six conditions

Repeated Measures Complex Designs

• Participants are exposed to all levels of the independent variable

• Increased statistical power

• Be wary of order effects, interference effects, fatigue, and other confounds

• The same group of participants take part in all of the conditions

Mixed Designs

Includes between-subjects and repeated-measures design features

• Example:

  • We randomly assign participants to study in either a re-reading or retrieval practice condition and have all participants take a test in a hot and cold room.

  • This is a 2 (Between factor: re-reading or retrieval practice study method) x 2 (Within factor: hot and cold room temperature) between-within mixed design