Hypothesis Testing

Null Hypothesis

A statement about the relationship that ought to exist between X and Y if the researcher’s theory is false

example: Fisher believes that Bristol cannot tell the difference between tea made with milk first and milk last

Null Hypothesis v Hypothesis Example

• H: Racial resentment reduces support for affirmative action policies designed to help ethnic mminorities

• H0: There is no relationship between racial resentment support for affirmative action
  policies designed to help ethnic minorities

Alternative Hypothesis (H_a)

Rejecting the null as we have sufficient confidence that the null hypothesis (H₀) is unlikely to be true given the data

Statistical Inference

The decision whether or not to reject the null with the data given

Inference Question

How unlikely is it that the results we find in the data could have emerged purely due to chance, assuming the null is true?

Why test hypothesis against the null?

In hypothesis-testing (and statistical inference more generally), the role of the null hypothesis is to serve as a baseline against which we can test the likelihood of finding what we found in the data

→ The less likely the result is under the null, the more likely it is that the null is false

Statistical significance & the mythical 5% threshold

There is a 5% threshold to determine whether or not to reject the null hypothesis

• If the probability of a Type I error (rejecting the null when it is true) is 5% or less, the result is said to be statistically significant, and the null is rejected.

• If the probability of a Type I error is greater than 5%, the result is not statistically significant and the null is preserved
  

P-value

The estimated probability of making a Type I error given the data is known as a p-value

P-Values, Statistical Significance and Hypothesis Testing

All hypothesis-testing references the null hypothesis (H0), not the researcher’s (Ha)

• If our significance threshold (α) is .05 and we find that the relationship between X and Y in our data is associated with a p-value <.05, then we reject the null

• If our significance threshold (α) is .05 and we find that the relationship between X and Y in our data is associated with a p-value >.05, then we fail to reject the null hypothesis.

Statistical v Substantive Significance

Statistical significance focuses on the probability of an observed effect occurring by chance, while substantive significance focuses on the magnitude or practical relevance of the effect.