Stats: Modeling the World, Ch. 10-12 (sampling, bias, experiments, observational studies)

Observational Study

A study based on data where there was NO manipulation of factors.

retrospective study

Observational: Subjects are selected and their PREVIOUS conditions or behaviors are determined. (need not be based on random sample, usually focus on estimating differences)

Prospective Study

Observational: Subjects are followed to observe FUTURE outcomes. (no treatments are deliberately applied, focus on estimating differences of groups)

Experiment

MANIPULATES factor levels to create treatments, RANDOMLY ASSIGNS subjects to the treatment levels, then compares the groups receiving the different treatments.

Random Assignment

To be valid, an experiment must assign experimental units to treatment groups at random

Factor

A variable whose levels are manipulated by the experimenter. (ex: drugs, location, etc.)

Response

A variable whose values are compared across different treatments. (ex: higher success rate)

Experimental Units

Subjects or Participants (if they are human). Who the experiment is performed on.

Level

Specific values that experimenter chooses for a factor (ex: how much drug the subject receives)

Treatment

Different levels of a single factor, or a combination of levels or two or more factors. Controlled circumstance applied to randomly assigned experimental units.

Principles of Experimental Design

1. CONTROL: aspects of the experiment that we know may have an effect on the response but are not the factors being studied.
2. RANDOMIZE subjects to treatments to even out effects we can't control.
3. REPLICATE over as many subjects as possible.
4. BLOCK to reduce the effects of identifiable attributes that can't be controlled

completely randomized design

all experimental units have an equal chance of receiving any treatment

statistically significant

If an observed difference is too large to believe that it is likely to have occurred naturally.

Blinding

When any individual associated with the experiment is not aware of how subjects have been allocated to treatment groups.

Single Blind vs. Double Blind

Two main classes of individuals: Those who could INFLUENCE THE RESULTS and those who EVALUATE the results.
If one of these groups is "blinded" than it is "Single-Blind". If BOTH groups are blinded than it is "Double-Blind".

Placebo

Any treatment known to have no effect.

placebo effect

The tendency of human subjects (sometime 20%) to show a response even when administered a placebo.

Blocking

When groups of experimental units are similar, it is often a good idea to gather them together into "groups". By doing so, we isolate the variability attributable to the differences between the "groups" so that we can see the differences caused by the treatments more clearly.

Randomized Block Design

With this, the subjects are randomly assigned to treatments only within their assigned "Groups".

Matching

In an observational study, what we do with subjects that are similar in ways not under study. Like blocking, reduces unwanted variation

Confounding

When the levels of once factor are associated with the levels of another factor in such a way that their effects cannot be separated

population

the entire group of individuals or instances about whom we hope to learn

sample

a representative subset of a population, examined in hope of learning about the population

sample survey

a study that asks questions of a sample in the hope of learning something about the entire population

bias

any systematic failure of a sample to represent its population

randomization

the best defense against bias, in which each individual is given a fair, random chance of selection

sample size

the number of individuals in a sample, represented by the letter n

census

a sample that consists of the entire population

simple random sample (SRS)

a sample of size n in which every set of n elements in the population has an equal chance to be the sample chosen AND every element has an equal chance to be selected

stratified random sample

the population is divided into groups of similar elements, called strata, and a random sample is selected from each stratum

cluster sample

entire groups or clusters are chosen at random
each cluster should be heterogeneous (representative of the population), and all the clusters should be similar to each other (like mini-populations)

systematic sample

a sample drawn by selecting individuals systematically ("every nth person is selected")

multistage sample

a sample selected in stages

voluntary response bias

form of bias, subject puts themselves in the sample

convenience sample

form of bias, when it is easy to take a sample but isn't necessarily representative of the population, as a result a bunch of people are left out

undercoverage

form of bias, a result of voluntary/convenience samples where a bunch of people are left out

nonresponse bias

form of bias, where the surveyor attempts to contact the subject and the subject

response bias

form of bias, where you answer incorrectly

wording of the question

form of bias, where the surveyor tries to guide the subject's response using wording

Population parameter

A numerically valued attribute of a model for a population.

Statistic, sample statistic

Statistics are values calculated for sampled data. T