5- XP- Threats to Internal Validity

Selection

If assigning subjects to comparison groups results in unequal distribution of these subject-related variables, then there is a possible threat to internal validity

Selection

It so happened that there were an equal number of boys and girls in the classes, so for convenience the boys were assigned to the Control Group and the girls to the Experimental Group. One day at school, the boys were told to go to one room and the girls to another room, where they were exposed to their respective conditions. Two days later, the Generalization Probe was conducted. The mean score for children in the Control Group was 1.2 and the mean score for children in the Experimental Group was 3.4. We conclude that the 20-minute interactive video improved the children's self-protection skills in a potential abduction situation

History

Outside events may influence subjects in the course of the experiment or between repeated measures of the dependent variable

History

Suppose that the dependent variable is measured twice for a group of subjects, once at Time A and later at Time B, and that the independent variable is introduced in the interim. Suppose also that Event A occurs between Time A and Time B. If scores on the dependent measure differ at these two times, the discrepancy may be due to the independent variable or to Event A

History

The name of each child in the classes was written on a separate slip of paper. All the slips were put in a bowl and mixed up thoroughly. Students were assigned to the Experimental Group and to the Control Group alternately as their names were pulled out of the bowl one at a time. One day at school, the children in the Control Group were told to go to one room and the children in the Experimental Group to another room, where they were exposed to their respective conditions. Immediately afterwards, while walking back to their regular classroom, all the children in the Control Group saw a man laughing and joking with their school principal. Two days later, the Generalization Probe was conducted, during which many of the Control Group children recognized the stranger as the man who made their principal laugh. The mean score for children in the Control Group was 1.2 and the mean score for children in the Experimental Group was 3.4. We conclude that the 20-minute interactive video improved the children's self-protection skills in a potential abduction situation.

Maturation

Subjects may change in the course of the experiment or between repeated measures of the dependent variable due to the passage of time per se. Some of these changes are permanent (e.g., biological growth), while others are temporary (e.g., fatigue).

Maturation

During a class early in the school year, the children viewed the 20-minute cartoon (Control condition). Two days later, the Generalization Probe was conducted. The experimenter fell ill soon afterwards, and so it wasn't until a class late in the school year that the children viewed the 20-minute interactive video (Experimental condition). Two days after that, a second Generalization Probe was conducted. The mean score for the children on the first Generalization Probe was 1.2 and their mean score on the second Generalization Probe was 3.4. We conclude that the 20-minute interactive video improved the children's self-protection skills in a potential abduction situation.

Repeated testing

The prior measurement of the dependent variable may affect the results obtained from subsequent measurements

Repeated testing

Due to time constraints, the experiment was run over four consecutive days. On Day 1, children viewed the 20-minute cartoon (Control condition). On Day 2, the Generalization Probe was conducted. On Day 3, the children were exposed to the 20-minute interactive video (Experimental condition). Finally, on Day 4, a second Generalization Probe was conducted. The mean score for children on the first Generalization Probe was 1.2 and their mean score on the second Generalization Probe was 3.4. We conclude that the 20-minute interactive video improved the children's self-protection skills in a potential abduction situation

Instrumentation

Examples include changes in the calibration of a mechanical measuring device as well as the proficiency of a human observer or interviewer

Instrumentation

The name of each child in the classes was written on a separate slip of paper. All the slips were put in a bowl and mixed up thoroughly. Students were assigned to the Experimental Group and to the Control Group alternately as their names were pulled out of the bowl one at a time. One day at school, the children in the Control Group were told to go to one room and children in the Experimental Group to another room, where they were exposed to their respective conditions. Two days later, the Generalization Probe was conducted. For ease of record keeping, all Control Group children were tested first, then all the Experimental Group children. The student teacher scored children's responses to the confederate's lures. In the beginning, he hid indoors and strained to see and hear through an open window; later on, he discovered he could see and hear better by hiding outside and peeking around a corner. The mean score for children in the Control Group was 1.2 and the mean score for children in the Experimental Group was 3.4. We conclude that the 20-minute interactive video improved the children's self-protection skills in a potential abduction situation.

Regression to the mean

Subjects with extreme scores on a first measure of the dependent variable tend to have scores closer to the mean on a second measure.

Regression to the mean

One day at school, the children viewed the 20-minute cartoon (Control condition). Two days later, the Generalization Probe was conducted. Then, in a class the following week, the children viewed the 20-minute interactive video (Experimental condition). The plan was to administer a second Generalization Probe two days after that. However, at this point, the experimenter realized that she had insufficient funding to complete the study and would only be able to retest ten children. She selected the ten poorest performing children on the first Generalization Probe, the mean score of which was 0.1. Their mean score on the second Generalization Probe was 2.5. We conclude that the 20-minute interactive video improved the children's self-protection skills in a potential abduction situation.

Experimental mortality/ Attrition

In the course of an experiment, some subjects may drop out before it is completed.

Experimental mortality/ Attrition

The name of each child in the classes was written on a separate slip of paper. All the slips were put in a bowl and mixed up thoroughly. Students were assigned to the Experimental Group and to the Control Group alternately as their names were pulled out of the bowl one at a time. One day at school, the children in the Control Group were told to go to one room and children in the Experimental Group to another room, where they were exposed to their respective conditions. Some of the children in the Experimental Group appeared bored by the interactive video, became disruptive, and were removed from the room. Two days later, the Generalization Probe was conducted. The mean score for children in the Control Group was 1.2 and the mean score for the remaining children in the Experimental Group was 3.4. We conclude that the 20-minute interactive video improved the children's self-protection skills in a potential abduction situation.

Selection-Maturation Interaction

Subject-related variables and time-related variables may interact.

Selection-Maturation Interaction

It so happened that there were an equal number of boys and girls, so for convenience the boys were assigned to the Control Group and the girls to the Experimental Group. During a class early in the school year, a Generalization Probe was conducted for all children. The experimenter fell ill soon afterwards, and so it wasn't until a class late in the school year that the children in the comparison groups were separated, with the Control Group children viewing the 20-minute cartoon and the Experimental Group children viewing the 20-minute interactive video. Two days after that, a second Generalization Probe was conducted. To see the results, click here (Figure 1). We conclude that the 20-minute interactive video improved the children's self-protection skills in a potential abduction situation.

Experimenter bias

Expectations of an outcome by persons running an experiment may significantly influence that outcome

Experimenter bias

The name of each child in the classes was written on a separate slip of paper. All the slips were put in a bowl and mixed up thoroughly. Students were assigned to the Experimental Group and to the Control Group alternately as their names were pulled out of the bowl one at a time. One day at school, children in the Control Group were told to go to one room and children in the Experimental Group to another room, where they were exposed to their respective conditions. Two days later, the Generalization Probe was conducted. For ease of record keeping, all the Control Group children were tested first, then all the Experimental Group children. Both the student teacher, who recorded how the children responded to the confederate's lures, and the confederate who presented the lures, were heavily involved in the production of the interactive video, and both of them strongly believed in its efficacy. The mean score for children in the Control Group was 1.2 and the mean score for children in the Experimental Group was 3.4. We can conclude that the 20-minute interactive video was effective in changing what the children did in a potential abduction situation

Reactivity

is a threat to internal validity when participants alter their behavior due to their awareness of being observed or studied

Internal validity

In an experiment, the researcher ensures that all participants receive the same instructions and conditions except for the independent variable. This practice is intended to enhance -

Regression to the mean

When extreme scores tend to move closer to the mean upon retesting, potentially masking true effects of an intervention, which threat to internal validity is present

Experiment mortality/ attrition

A researcher conducts a longitudinal study on the impact of cognitive training. However, some participants drop out of the study over time, leaving fewer participants in the follow-up. This dropout affects the final group comparison. Which threat to internal validity is this?

Selection bias

occurs when participants in different groups differ systematically before the experiment begins

Experimenter bias

occurs when experimenter unintentionally communicates their expectations to the participants, influencing the results

History

- effects are more likely to occur in long term studies than in short term studies

Experiment mortality/ Attrition

While random assignment helps to distribute participant characteristics evenly between groups, it does not eliminate the problem of - as it can still introduce bias if the characteristics of those who drop out differ systematically between groups. 

Repeated testing/ testing

- effects are a threat to internal validity when participants' familiarity with the test influences their subsequent performance

Instrumentation

- effects are less likely when using automated measurement tools