Experiments, Observational Studies, and Sampling Methods in Statistics

Observational study

A study based on data in which no manipulation of factors has been employed.

Retrospective study

An observational study in which subjects are selected and then their previous conditions behaviors are determined.

Prospective study

An observational study in which subjects are followed to observe future outcomes.

Experiment

manipulates factor levels to create treatments, randomly assigns subjects these treatment levels, and then compares the responses of the subject groups across treatment levels.

Random assignment

To be valid, an experiment must assign experimental units to treatment groups at random.

Factor

A variable whose levels are manipulated by the experimenter.

Response

A variable whose values are compared across different treatments.

Experimental units

Individuals on whom an experiment is performed.

Level

The specific values that the experimenter chooses for a factor.

Treatment

The process, intervention, or other controlled circumstance applied to randomly assigned experimental units.

Control group

The experimental units assigned to a baseline treatment level, typically either the default treatment or a null, placebo treatment.

Blinding

Any individual associated with an experiment who is not aware of how subjects have been allocated to treatment groups is said to be blinded.

Single-blind

When every individual in either the subjects or treatment administrators is blinded.

Double-blind

When everyone in both classes of individuals who can affect the outcome of an experiment is blinded.

Placebo

A treatment that has no therapeutic effect, often used as a control in experiments.

Placebo effect

A phenomenon where patients experience improvements in their condition simply because they believe they are receiving treatment.

Statistically significant

When an observed difference is too large for us to believe that it is likely to have occurred naturally.

Principles of experimental design

Randomize subjects to treatments to even out effects that we cannot control.

Replication

Replicate over as many subjects as possible; results for a single subject are just anecdotes.

Blocking

Block to reduce the effects of identifiable attributes of the subjects that cannot be controlled.

Matching

A method in a retrospective or prospective study where subjects who are similar in ways not under study may be matched and compared on the variables of interest.

Completely randomized design

A design in which all experimental units have an equal chance of receiving any treatment.

Randomized block design

A design where randomization occurs only within blocks.

Confounding

When the levels of one factor are associated with the levels of another factor in such a way that their effects cannot be separated.

Random

if we know the possible values it can have, but not which particular value it takes.

Generating Random numbers

Numbers that are hard to generate, but several Internet sites offer an unlimited supply of equally likely random values.

Simulation

A model of a real-world situation using random-digit outcomes to mimic the uncertainty of a response variable of interest.

Simulation component

A component that uses equally likely random digits to model simple random occurrences whose outcomes may not be equally likely.

Trial

The sequence of several components representing events that we are pretending will take place.

Response variable

Values that record the results of each trial with respect to what we were interested in.

Sample survey

A study that asks questions of a sample drawn from some population in the hope of learning something about the entire population.

Bias

Any systematic failure of a sampling method to represent its population.

Randomization

A method where each individual is given a random chance of selection to defend against bias.

Sample size

The number of individuals in a sample.

Census

A survey that attempts to include every member of a population.

Population parameter

A value that summarizes a characteristic of a population.

sample statistic

A value that summarizes a characteristic of a sample.

Representative

A sample that accurately reflects the characteristics of the population from which it is drawn.

Simple random sample (SRS)

A sample in which each member of the population has an equal chance of being selected.

Sampling frame

A list of individuals from which a sample is drawn.

Sampling variability

The natural variation in statistics from different samples drawn from the same population.

Stratified random sample

A sample obtained by dividing the population into strata and then taking a random sample from each stratum.

Cluster sample

A sample obtained by dividing the population into clusters and then randomly selecting some clusters to include all individuals within those clusters.

Multistage sample

A sample obtained by combining several sampling methods.

Systematic sample

A sample obtained by selecting every nth individual from a list of the population.

Pilot

A small preliminary study conducted to test the feasibility of a larger study.

Voluntary response bias

A bias that occurs when individuals can choose whether to participate in a survey, often leading to unrepresentative samples.

Sample Statistic

Statistics calculated for sampled data that estimate a population parameter.

Representative Sample

A sample that accurately reflects the corresponding population parameters.

Simple Random Sample

A sample in which each set of n elements in the population has an equal chance of selection.

Sampling Error

The natural tendency of randomly drawn samples to differ from one another.

Stratified Sample

A sampling design where the population is divided into subpopulations and random samples are drawn from each stratum.

Cluster Sampling

A sampling design in which entire groups are chosen at random.

Multistage Samples

Sampling schemes that combine several sampling methods.

Pilot Study

A small trial run of a survey to check whether questions are clear.

Convenience Sample

A sample consisting of individuals who are conveniently available.

Undercoverage

A sampling scheme that biases the sample by giving part of the population less representation.

Nonresponse Bias

Bias introduced when a large fraction of those sampled fails to respond.

Response Bias

Anything in a survey design that influences responses.

Wording Bias

A typical response bias arising from the wording of questions that may suggest a favored response.

Population

the entire group of individuals or instances about whom we hope to learn

Sample

A (representative) subset of a population, examined in hope of learning about the population.