Lecture 7 - p-values and confidence intervals

Hypothesis testing results

1) Accepting or rejecting the null hypothesis, based on a critical value

2) p-values, based on our statistic

3) Confidence levels, calculated from a distribution of the statistic that is based on the data

p-value

The probability of observing our statistic or one more extreme, if the null hypothesis is true

If the p-value is less than the significance level…

The statistic has such a low probability of being observed if the null hypothesis is trues, that we should reject the null hypothesis

If the p-value is greater than the significance level…

Accept the null hypothesis, because the p-value tells us that the observed statistic is a probable result when the null hypothesis is true

Finding a _____-value is the reverse of finding the _____-value

p, critical

When you find a critical value, you start with…

A probability, and integrate under the probability distribution from one or both tails until you equal your probability

With a p-value, you start with…

The observed statistic, then integrate under the probability distribution from that observed statistic towards the tail

Critical value(s) correspond to…

The cutoff values of the statistic

The p-value is the…

The cumulative probability from your statistic to the tail

Reporting the p-value…

directly informs of the probability of observing what we did if the null hypothesis is true

T/F: p-values convey more information than the classical approach

True

p-values are controlled by…

Effect size

Sample size

Inherent variation

Relationship between effect size and p-values

Larger effect sizes produce smaller p-values

Relationship between sample size and p-values

Large sample sizes produce smaller p-values

Relationship between inherent variation and p-values

Larger amounts of variation in what is being studied leads to larger p-values

When to use confidence interval

When you want to know all the null hypotheses that are inconsistent with the data

T/F: Without a null hypothesis, you cannot generate the distribution needed to calculate a critical value or a p-values

True

Confidence interval

The set of all null hypotheses that are consistent with the data

Confidence intervals are constructed from a…

Distribution of the statistic that is based on the data

You would (accept/reject) any null hypothesis that lies within the confidence interval, including the end points, the confidence limits

Accept

You would (accept/reject) any null hypothesis that lies beyond the confidence limits

Reject

Confidence interval provides…

Uncertainty

T/F: Confidence intervals should not be your standard practice

False - confidence intervals SHOULD be your standard practice

Large confidence interval means…

Large uncertainty in our estimate of the parameter

Small confidence interval means…

A high degree of certainty

Confidence values are always reported with…

The value of confidence

As confidence (increases/decreases), and significance (increases/decreases), so does that of the confidence interval

Increases, decreases