Lecture 6 - Hypothesis Testing

What kind of questions can we answer?

Is there a difference in the proportion of a variable between groups?

Is one treatment/intervention better than another?

Are two variables linked?

Does one variable have an effect on another? How strong of an effect?

The first 4 steps of hypothesis testing are the most crucial part of a research problem. What are they?

1. State the question

2. Define the population

3. Identify the outcome variable and statistical test

4. State the null hypothesis

What are characteristics of the question we are asking?

The question should decide the data collection and no the other way around

What is the clinical/practical importance and are we asking a question that exists in practice?

What population will it be applied in?

Can we get the data necessary to answer it?

What is H0?

The null hypothesis

Statistical tests start with the assumption that observations are the result of random chance (variation)

States that there is no difference and no relationship

What is HA?

The alternative hypothesis

Includes everything outside of the null

There is a difference and relationship

Should include the question the study is designed to answer

Can you prove the alternative hypothesis?

No, you can only reject the null

If our p-value is less than the pre-defined value...

Reject the null, maybe something is there

If our p-value is more than the pre-defined value...

Fail to reject the null, no proof anything is there

What is the p-value?

The probability that you would see your data under the null hypothesis

Probability we use to decide if something we observed is real or just happened by chance

What is the significance level (alpha)?

The level of confidence we want to have for rejecting the null

The % of time we're willing to declare an effect even though it isn't there

Most common is 0.05

What does a significance level of 0.05 mean?

We would expect data as weird as this to happen 5% of the time, assuming the null is correct

If we repeat the study 100 times, we would reject the null inappropriately 5 times

What is a type I error?

Rejecting the null hypothesis when it is true

Finding a relationship that isn't there

Alpha = 0.05

What is a type II error?

Failing to reject the null when it isn't true

Fail to find a relationship that is there

Minimize with power calculations (tell us our probability of this error) or sample size determination (larger sample, more precision, lower type II error)

Beta = 0.2

If we reject the null and the null is false...

Correctly rejected null

Probability = 1-beta

If we reject the null but the null is true...

Type 1 error

Probability = alpha

If we fail to reject the null but the null is false...

Type II error

Probability = beta

If we fail to reject the null and the null is true...

Correctly failed to reject the null

Probability = 1-alpha

What is a one tailed test?

Directional, accounts for a difference in only one direction

Reject the null if the result falls in only one of the tails

All uncertainty is in that tail, so more lenient

What is a two tailed test?

Non directional

Accounts for a difference in either direction

Reject null if the result falls in either tails

Uncertainty is divided between tails, so more strict

What is an example of the difference between one and two tailed tests?

One: spaying a dog will not make it more aggressive

Two: spaying a dog will not make it more or less aggressive