RMDA 9 threats to internal validity

internal validity

strength of the connection between the cause and effect relationship

History

1. When some event takes place between your measures that has nothing to do with your experiment or independent variable. Something else is happening to your subjects between your measures

   1. Ex: If there is a long-term treatment study with psychiatric patients measuring their depression levels, but during the study there has been new staff members or a new diet introduced, this can influence the outcome.

Maturation

1. Individual development over time. Participant is growing physically, mentally, emotionally

   1. Ex: Group of 12 year old boys who are gymnasts are tested on how well they can do various gymnastic moves. The scores are average or slightly below average. 2 years later, they are tested again and there is a massive improvement. The gap in time affects the study because the boys have matured physically, aiding in their improvement and capability of gymnastics.

Testing

1. Being tested more than once can change a participant's scores, no matter what. Repeated testing can make participants grow accustomed and do better

   1. Ex: SAT prep claims to familiarize students with content that will be on the SAT. The first time someone may take the SAT, it will be unfamiliar and they may not get a high score. After taking it multiple times, they may get used to the format of the test rather than studying the content. This is a threat because repeatedly taking a test will result in familiarization and will inevitably lead to better test scores. This means it cannot be proved that the SAT prep was the reason for the improvement in test scores.

Instrumentation

1. a survey, an inventory, an IQ test, a person (like a clinical psychologist). Instruments can change and cause differences in the data.

   1. Ex: Different computer screens with stimulus flash on screen and one monitor is 500 milliseconds delayed than the other. A threat occurs because this can affect when a subject reacts to the stimulus compared to others, thus, changing or causing a difference in the data.

Selection

1. Occurs when the participants in one group differ initially from those in another group. Differences in age, reading level, education, etc.

   1. Ex: You are testing a new technique to help you read better. For the experimental group, you used people who came to the reading clinic (these people received the treatment). For the control group, you use students in a psychology course. This is a threat because there could be a difference (in IQ, reading level, achievement, etc.) in the types of people who come to a reading clinic vs. those in a psychology class. Second, those who come to the reading clinic might have lower scores initially and be able to improve more than the psych students. Third, the people who come to the clinic would likely be more motivated and work harder at learning how to read faster, regardless of technique.

Diffusion/Imitation

1. Participants in an experiment may communicate with each other, and by this communication they may reduce the differences between the groups.

   1. Ex: In an experiment, the experimental group may be asked to play violent video games while the control group reads a story with violence, then they are asked questions at the end such as how they felt, if they noticed any violence, and if they have acted violently. If participants in the experimental group spoke about their experiences and the purpose of the experiment with the control group then this could affect the responses of the questions.

Selection-Maturation Interaction

1. the situation in which one group of participants changes along a given dimension faster than another group. One group develops differently or faster than the other group.

   1. Ex: Children whose families have lower-than-average incomes tend to develop cognitive abilities at different rates from children whose families have average incomes. There is a threat here because if this difference is not taken into account, the experimenter could attribute the difference to an outside treatment or unrelated variable.

Mortality

1. When participants drop out or refuse to take part in an experiment.

   1. Ex: You measure whether children are more creative after playing with dull toys vs fun toys. As you run the experiment, some of the children who are given the dull toys may leave the experiment or not return for other sessions. Thus, after a period of weeks, the fun-toy group might be composed of all the children who began the experiment, whereas the dull-toy group is composed of only half of the original children from that group. This is a threat because if you try to analyze these results, you are faced with the possibility that only a certain type of child remains in the dull-toy group. The dull-toy group would then consist of a different population from the fun-toy group, making your conclusions invalid. The mean can be artificially inflated as well.

Statistical Regression

1. Occurs when extreme scores (very high or very low) change over time without any treatment. 

   1. Ex: A person claims they have come out with a "genius vitamin pill". A person takes it and then scores a 1510 on the SAT. They feel lucky, take the pill again, and take the SAT a second time and score a 1400. This is a threat because the researcher may claim that their pill works after the first time, when in fact the participant was likely luckier the first time and ended up regressing towards the mean.