Lecture 7: Random Error and P-values

the full group you are caring about

(example: all Iowa school children)

population

A smaller group taken from the population

(example: 100 randomly selected Iowa school children)

Sample

You almost never measure the ______, so you rely on ______

population, samples

Why do we use samples instead of populations?

Populations are usually too large to measure fully, so we use samples to estimate population values.

in this type of sample, everyone has an equal chance of being selected.

ex: you assign everyone a number, and numbers are selected at random, and variables are then compared

simple random sampling

simple random sampling keeps sampling ____ and avoids ____

fair, bias

Even if a population is fixed and unchanging, a sample can look very different. What can account for this difference

random error

This is an example of what?

You pull many different smaples of students and measure the height. Every sample gives you a different mean height

random error

random error is _________ and the reason we use p-values and confidence intervals and the reason we talk about uncertainty

unavoidable

the people in your sample might not always represent the ________

population

We can only ______ population rates, but can get exact values for samples

estimate

IA mean height: 5’6’

non-IA mean height: 5’10’

the sample shows a difference, but that doesnt mean the population shows a difference

What can be the factor that causes this differecnce

random error, the sample is very small

This is the probability of observing a result as extreme as your sample if the population has no differences

in other words….

how likely it is that what we observed is due to random error

p-value

A ____ p-value means that it is unlikely to be due to random error

low

A ____ p-value means that it is likely to be due to random error

high

This ALWAYS states that there is no difference between the population

ex: mean height IA = mean height non-IA

null hypothesis

values you want to compare are equal in the population

Purpose of the null hypothesis

Serves as the baseline assumption for statistical testing.

You always test whether the sample data guves you evidence against ______

H0 (null hypothesis)

the null hypothesis is statistically significant if p___ 0.05

less than

Does statistical significance mean a big or important difference?

No — it only means the result is unlikely due to random error, not that it is large or meaningful.

p<0.05 explains

the probability of the results are due to random error (if H0 were true) is less than 5%

low p-values show stronger evidence that the difference is real and not just due to random noise

If p<0.05 the association between variable 1 and 2 is ________

significant

Interpretation of a low p-value

It is unlikely that the sample result occurred due to random error if H₀ were true.

Interpretation of a high p-value

The observed difference could easily be explained by random error under H₀.

what 2 things must you state for every p-value

1. state the null hypothesis'

2. interpret the p-value numerically

What would we state is the p value = 0.03 (2 groups relating height)

1. The two groups have equal mean height in the population

2. There is a 3% probability of observing a difference as big as this samples difference due to random error if H0 is true

Relationship between random error and p-values

P-values measure how likely the observed sample difference is due to random error.

This type of sampling is used when a populaiton has groups that want respresented. Its process requires dividing the population into strata, taking SRS within each stratum and then combining samples

stratified random sampling

stratifies random sampling (does/does not) have equal selection probability

does not

why is stratified random sampling used

to ensure each population is represented.

We will do this when there is way fewe of one type of person in a group

low disease prevalence

what is something that we need for stratified random sampling

a registry of individuals with a condition

this type of sampling is used when you sample groups instead of individuals. The process includes randomly selecting a cluster, then measuring all or some people inside those clusters

cluster random sampling

Examples of clusters

schools, neighborhoods, hospitals

cluster random sampling has selection based off _____ in a group

membership

cluster random sampling can have multiple _______

phases/stages

are observations in cluster random sampling independent?

no

cluster random sampling introduces ___

bias

students within a school resemble one another, not random enough

most basic statistics assume ______ of observations, and have ____ probability of selction

independence, equal

does simple random sampling satisfy independent observations and equal probability of selection

yes

do stratifies and cluster samplings satisfy independent observations and equal probability of selection

NO

If you are using sampling other than simple random sampling, you need

more complex statistics

this would need to be explained in the methods section if used in research article

What happens if independence is violated?

Standard statistical methods can give incorrect results.

What does a p-value NOT tell you?

It does not measure effect size, direction, causation, or practical importance.

Why can a small sample produce misleading results?

Small samples have more variability and are more affected by random error.

You always either ____ the null hypothesis or ____ ____ ___ the null hypothesis

reject

fail to reject

Does failing to reject H₀ mean H₀ is true?

No — it only means there is not enough evidence to conclude otherwise.