Applied Statistics - Chapter 6: Hypothesis testing

Hypothesis testing

A statistical method that uses sample data to evaluate a hypothesis about a population

Hypothesis testing (con’t)

Is one of the most commonly used inferential procedures

Conceptual hypothesis

A general statement about the relationship between the independent and the dependent variables

e.g. The average math ability score for the population of high school students is 47; I think the students at my high school have better math ability

Statistical hypothesis

A statement designed to make inferences about the population that can be shown to be supported or not supported

Statistical hypothesis (con’t)

Is just the conceptual hypothesis written mathematically

e.g. umh > 47

Null hypothesis (H₀)

States that a population parameter is equal to a specific value or a range of values

Must include an equal sign

Alternative Hypothesis (H₁ or H_A)

States that the population parameter isn’t equal to the specific value or range of values

Alternative Hypothesis (H₁ or H_A) [con’t]

Is the complement of the Null Hypothesis

The thing about null and alternative hypotheses

They must be mutually exclusive (have no outcomes in common) and exhaustive (include all possible outcomes).

Directional hypothesis

When the researcher explicitly states the direction of the relationship

Directional hypothesis (con’t)

If the null and alternative hypotheses both involve a range of values, the hypotheses are directional

Directional hypothesis (III)

The alternative hypothesis will always include a < or > sign

Non-directional hypothesis

When the researcher predicts that there will be a difference, but is unable to predict the exact relationship

Non-directional hypothesis (con’t)

If the null and alternative hypotheses involve a single specific value, the hypotheses are non-directional.

Non-directional hypothesis (III)

The alternative hypothesis will always include a ≠ sign

Testing the null

• If there’s enough evidence that the null is incorrect, we reject the null

• If there’s not enough evidence to demonstrate that the statement is false and reject the null, we fail to reject the null

Testing the alternative hypothesis

If we have enough evidence to reject the null hypothesis, we can say that the alternative hypothesis is likely a more accurate statement

The thing about testing the alternative hypothesis

Just because we reject the null doesn’t mean that we accept the alternative as true (the alternative is NEVER accepted

Alpha (α)

The level of significance set by the experimenter beforehand

Alpha (α) [con’t]

We have sufficient evidence to reject the null hypothesis when the probability associated with our outcome (p-value) is less than alpha.

Alpha (α) [III]

Is always set by the researcher

e.g. Set at .05, but can be set lower, such as .025, .01, .001, etc

Alpha (α) [IV]

It’s the proportion of times out of 100 that an event will occur simply by chance, and our statistically significant result is due to chance

What to do when p < or equal to α

Reject the null

What to do when p > α

Fail to reject the null

What does rejecting the null mean?

The results are statistically significant, as they aren’t likely to be due to chance or random events and are worthy of further analysis.

(Nothing’s still proven!)

One-tailed probability

Is used with directional hypotheses

One-tailed probability (con’t)

Is the probability that an outcome will occur at one end of the sampling distribution

Usage of one-tailed probability

Used to answer questions such as “What’s the probability of getting a score greater than or equal to a certain value?”

Two-tailed probability

Is used with non-directional hypotheses

Two-tailed probability (con’t)

Is the probability that an outcome will occur at either extreme of a sampling distribution.

Usage of two-tailed probability

Used to answer questions such as “What’s the probability of getting a score that differs from a certain value?”

The observed value

A value that’s the result of the statistical test performed is determined by the data.

The critical value

A value determined by alpha, which is set by the experimenter beforehand

When do we reject the null hypothesis?

When the:

• Observed value is more extreme than the critical value

• The p-value associated with the observed value is smaller than or equal to alpha

When do we fail reject the null hypothesis?

When the:

• Observed value is less extreme than the critical value

• P-value associated with the observed value is larger than alpha

Type 1 error (false positive)

Rejecting the null when it’s actually true; happens 5% of the time

Type 1 error (false positive) [con’t]

Its implication is that a researcher concludes that there is an effect when there is not an effect

Type 2 error (false negative)

Failing to reject the null when it’s false

Type 2 error (false negative) [con’t]

Its implication is that a the researcher misses an effect

The power of a statistical test

The probability of rejecting the null hypothesis when the null hypothesis is false

Factors that affect power

• The actual obtained difference (e.g. M - u)

• Variability

• The sample size (n)

Factors that affect power (con’t)

• The significance level (alpha)

• Whether the test is one-tail or two-tail