Q10: Uncertainty

Uncertainty

How much an estimate might change if we took a different sample

Bootstrapping

A way to estimate uncertainty by repeatedly taking samples from the original sample with replacement

Bootstrap sample

One new sample made by randomly choosing from the original sample with replacement

Bootstrap distribution

A distribution made from the statistics calculated from many bootstrap samples

Sampling with replacement

Choosing an item and putting it back before choosing again

Sampling without replacement

Choosing an item and not putting it back before choosing again

Statistic

A number that describes a sample such as a mean difference or slope

Mean

The average of a group of numbers

Sampling distribution

A distribution of a statistic made by taking many samples from the population

Bootstrap standard error

The standard deviation of the bootstrap distribution

Standard deviation

How spread out the values in a group of data are

Standard error

How much a sample estimate can vary from sample to sample

Spread

How far apart the values in a distribution are

Confidence interval

A range of values that are considered reasonable based on the data

95% confidence interval

A range made by a method that captures the true value about 95% of the time when repeated many times

Difference in means

The mean of one group minus the mean of another group

Slope

How much the response changes when the explanatory variable goes up by one unit

Zero for a difference

Means the two group means are the same

Zero for a slope

Means there is no linear change in the response as the explanatory variable increases

CI contains zero

Zero is still a reasonable value for the difference or slope

CI does not contain zero

Zero is not a reasonable value for the difference or slope

Sampling bias

A problem where the sample does not fairly represent the population

Representative sample

A sample that is a good picture of the population

Non-independence

When data points are connected or related instead of being separate from each other

Replicate

One bootstrap sample and the statistic calculated from it

Number of replicates

How many bootstrap samples are made

Stability

How much the bootstrap result stays the same when the process is repeated

Small sample size

A sample with too few observations to give a reliable picture of the population

Bootstrap limitation

A problem that can make bootstrapping unreliable

Precise but wrong

A result that is very consistent but still wrong because the original sample was biased or had another problem

infer

A package in R that makes bootstrapping and other statistical calculations easier

specify()

The infer step where you say what variables and relationship you want to study

generate()

The infer step where you create bootstrap samples

calculate()

The infer step where you calculate a statistic for each bootstrap sample

~ in R

The symbol used to show a relationship between a response and an explanatory variable

Response variable

The variable you are trying to understand or predict

Explanatory variable

The variable used to explain or predict the response

Bootstrap process

Start with the sample then resample with replacement calculate the statistic repeat many times and study the results

Why use replacement

Replacement lets bootstrap samples have different combinations of observations so the statistic can change

Bootstrap distribution spread

A wider spread means more uncertainty and a narrower spread means less uncertainty

Can bootstrapping fix bias

No bootstrapping cannot fix a biased sample

Can more replicates fix a bad sample

No more replicates make the bootstrap result more stable but do not fix problems in the original sample

Bootstrap and small samples

Bootstrapping can be unreliable when the original sample is very small

Bootstrap and non-independent data

A simple bootstrap can give misleading results when observations are not independent

Main bootstrap idea

Use the sample as a stand-in for the population and repeatedly resample it to learn about uncertaint