ISS W2 P-values and hypotheses

True or False: the P-value is the probability that the null hypothesis is true?

False.

This description is frequently assumed to be the practical interpretation of a P-value. It is not strictly correct to describe the P-value as the probability that the null hypothesis is true, but for most people (except under exam conditions) that will not matter much when working to this assumption.

True or False: the P-value is the probability that the alternative hypothesis is true?

False: the P-value is not the probability that the alternative hypothesis is true.

True or False: the P-value is the probability that the null hypothesis is false?

False: the P-value is not the probability that the null hypothesis is false.

True or False: the P-value is the probability that the alternative hypothesis is false?

False: the P-value is the not probability that the alternative hypothesis is false.

True or False: the P-value is the probability of ending up with the given result (or more extreme results) of your sample/test if the null hypothesis were true?

True: P values are computed assuming the null hypothesis to be true. So, the P-value is suggesting the likelihood that any observed difference between groups in your sample/test is due to chance.

What is a null hypothesis?

A hypothesis that there is no difference between the things being tested i.e. that there is no effect in the population.

What is the alternative hypothesis?

A hypothesis that claims that there is an effect in the population.

The closer a P-value is to 0, the _____ confidence we have that the things being tested are _________.

• More

• Different

If the null hypothesis describes that there is no difference between the things being tested, and that the outcome of tests show that it would be easy for the results between the things being tested to be affected by little random things, what do we do with the null hypothesis?

We fail to reject the null hypothesis.

If the null hypothesis describes that there is no difference between the things being tested, and that the outcome of tests show that it would be hard for the results between the things being tested to be affected by little random things, what do we do with the null hypothesis?

We reject the null hypothesis.

What does the P-value of 0.05 mean with regard to “false positives”?

It means that if there is no difference between the things being tested, and if we did this exact same experiment a bunch of times, then only 5% of those experiments would result in the wrong decision (or “false positives”).

What does the P-value of 0.05 mean with regard to the null hypothesis?

The P-value helps us to decide if we should reject the null hypothesis: a P-value below 0.05 means we could reject the null hypothesis, and say that there is a statistically significant difference between the things being tested.

True or false: a small P-value implies that the effect size (or difference) between two things being tested is large?

False: while a P value can inform the reader whether an effect exists, the P value will not reveal the size of the effect.

Standard statistics start from a theoretical probability distribution and give answers to questions such as: “If this study were based on _______ ______ data, how ______ would these actual results be?”

• Completely random

• Unusual

Statistical testing doesn’t quite do what clinicians would wish for; we are interested to know how ____ to the truth a study result is likely to be.

• Near

What is a Confidence Interval?

A confidence interval is the mean of your estimate plus and minus the variation in that estimate; a Confidence Interval represents the uncertainty that the sample mean estimates the population mean of the thing you are investigating.

The wider the confidence interval, the more _____ the estimate.

• Uncertain

The width of a 95% confidence interval for the mean depends on both the _______ of observed [thing you are investigating] in the sample and the ______ of observations made.

• Variability

• Number

For normally distributed data, what does the “mean ± 2xSD contain?

Approximately 95% of all the observations of the [thing you are investigating].

What does the standard deviation (SD) indicate around the mean?

The spread of the distribution of the sample observations.

The ______ the value of the standard deviation (SD), the ____ the spread.

• Larger

• Wider

The larger the value of standard deviation, the wider the spread, how does this affect the variation in data?

This means more variation in the data.