Statistics: Sampling, Bias, and Experimental Design Principles

Population

In a statistical study, the entire group of individuals we want information about.

Census

Collects data from every individual in the population.

Sample

A subset of individuals in the population from which we collect data.

Sample Survey

A study that collects data from a sample to learn about the population from which the sample was selected.

Convenience Sampling

Selects individuals from the population who are easy to reach.

Bias

The design of a statistical study shows bias if it is very likely to underestimate or very likely to overestimate the value you want to know.

Voluntary Response Sampling

Allows people to choose to be in the sample by responding to a general invitation.

Random Sampling

Involves using a chance process to determine which members of a population are included in the sample.

Simple Random Sample (SRS)

Of size n is chosen in such a way that every group of n individuals in the population has an equal chance to be selected as the sample.

Strata

Groups of individuals in a population who share characteristics thought to be associated with the variables being measured in a study.

Stratified Random Sampling

Selects a sample by choosing an SRS from each stratum and combining the SRSs into one overall sample.

Cluster

A group of individuals in the population that are located near each other.

Cluster Sampling

Selects a sample by randomly choosing clusters and including each member of the selected clusters in the sample.

Systematic Random Sampling

Selects a sample from an ordered arrangement of the population by randomly selecting one of the first k individuals and choosing every kth individual thereafter.

Undercoverage

Occurs when some members of the population are less likely to be chosen or cannot be chosen in a sample.

Nonresponse

Occurs when an individual chosen for the sample can't be contacted or refuses to participate.

Response Bias

Occurs when there is a systematic pattern of inaccurate answers to a survey question.

Observational Study

Observes individuals and measures variables of interest but does not attempt to influence the responses.

Response Variable

Measures an outcome of a study.

Explanatory Variable

May help explain or predict changes in a response variable.

Confounding

Occurs when two variables are associated in such a way that their effects on a response variable cannot be distinguished from each other.

Experiment

Deliberately imposes treatments (conditions) on individuals to measure their responses.

Placebo

A treatment that has no active ingredient, but is otherwise like other treatments.

Treatment

A specific condition applied to the individuals in an experiment. If an experiment has several explanatory variables, a treatment is a combination of specific values of these variables.

Experimental Unit

The object to which a treatment is randomly assigned.

Subjects

When the experimental units are human beings, they are often called subjects.

Factor

An explanatory variable that is manipulated and may cause a change in the response variable.

Levels

The different values of a factor.

Control Group

Used to provide a baseline for comparing the effects of other treatments. May be given an inactive treatment (placebo), an active treatment, or no treatment at all.

The Placebo Effect

Describes the fact that some subjects in an experiment will respond favorably to any treatment, even an inactive treatment.

Double-Blind Experiment

Neither the subjects nor those who interact with them and measure the response variable know which treatment a subject is receiving.

Single-Blind Experiment

Either the subjects or the people who interact with them and measure the response variable don't know which treatment a subject is receiving.

Random Assignment

Means that experimental units are assigned to treatments using a chance process.

Control

Means keeping other variables constant for all experimental units.

Replication

Means giving each treatment to enough experimental units so that a difference in the effects of the treatments can be distinguished from chance variation due to the random assignment.

Four Key Principles of Experimental Design

Comparison, random assignment, control, and replication.

Completely Randomized Design

The experimental units are assigned to the treatments completely at random.

Block

A group of experimental units that are known before the experiment to be similar in some way that is expected to affect the response to the treatments.

Randomized Block Design

The random assignment of experimental units to treatments is carried out separately within each block.

Matched Pairs Design

A common experimental design for comparing two treatments that uses blocks of size 2.

Sampling Variability

Refers to the fact that different random samples of the same size from the same population produce different estimates.

Statistically Significant

When the observed results of a study are too unusual to be explained by chance alone.

Inference About Cause and Effect

A well-designed experiment that randomly assigns experimental units to treatments allows for conclusions about cause and effect.