Completely randomized design
When the treatments are assigned to all the experimental units completely by chance.
Confidentiality
A basic principle of data ethics that requires individual data to be kept private.
Confounding
When two variables are associated in such a way that their effects on a response variable cannot be distinguished from each other.
Inference about the population
Using information from a sample to draw conclusions about the larger population. Requires that the individuals taking part in a study be randomly selected from the population of interest.
Informed consent
A basic principle of data ethics. Individuals must be informed in advance about the nature of a study and risk of harm it may bring. Participating individuals must then consent in writing.
Institutional review board
A basic principle of data ethics. All planned studies must be approved in advance and monitored by and institutional review board charged with protecting the safety and well-being of the participants.
Lack of realism
When the treatments, the subjects, or the environment of an experiment are not realistic. Lack of realism can limit researchers’ ability to apply the conclusions of an experiment to the settings of greatest interest.
Matched pair
A common form of blocking for comparing just two treatments. In some matched pairs designs, each subject receives both treatments in a random order. In others, the subjects are matched in pairs as closely as possible, and each subject in a pair is randomly assigned to receive one of the treatments.
Margin of error
A numerical estimate of how far the sample result is likely to be from the truth about the population due to sampling variability.
Nonsampling error
The most serious errors in most careful surveys are nonsampling errors. These have nothing to do with choosing a sample—they are present even in a census. Some common examples of nonsampling errors are nonresponse, response bias, and errors due to question wording.
Statistically significant
An observed effect so large that it would rarely occur by chance.