Hypothesis testing

Hypothesis Testing

It is also called significance testing

• is the key to our scientific inquiry.

statistical hypotheses.

• Involves the statement of a null hypothesis, an alternative

hypothesis, and the selection of a level of significance.

• Tests a claim about a parameter using evidence (data in a

sample)

Statistical Hypotheses

• Statements of circumstances in the population that the statistical process will examine and

decide the likely truth or validity

• Statistical hypotheses are discussed in terms of the population, not the sample, yet tested on

samples

• Based on the mathematical concept of probability

• Null Hypothesis

• Alternative Hypothesis

Statistical Hypotheses

If the hypothesis is stated in terms of population

parameters (such as mean and variance), the

hypothesis is called statistical hypothesis.

Null Hypothesis

The case when the two groups are equal; population means are the same

• Null Hypothesis = H0

• This is the hypothesis actually being tested

• H0 is assumed to be true

• The null hypothesis (H0) is a claim of “no difference in the population”

• The null hypothesis (denoted by H0) is a statement that the value of a population parameter (such as proportion, mean, or standard deviation) is equal to some claimed value.

• We test the null hypothesis directly.

• Either reject H0 or fail to reject H0

Null Hypothesis (H0)

• Symbol: =

• Verbal equivalents:

  • equal to

  • the same as

  • not changed from

  • is

Alternative Hypothesis (Ha​ )

- Two tailed test

Symbol: ≠
Verbal equivalents:

• not equal

• different from

• changed from

• not the same as

Alternative Hypothesis (Ha​ ) One tailed test

• right tailed

1. Symbol: >
   Verbal equivalents:

   • greater than

   • above

   • higher than

   • longer than

   • bigger than

   • increased

   • at least

Alternative Hypothesis (Ha​ ) One tailed test

• left tailed

1. Symbol: <
   Verbal equivalents:

   • less than

   • below

   • lower than

   • smaller than

   • shorter than

   • decreased or reduced from

   • at most

Null Hypothesis (H0):

• Role: Initial claim / status quo

• Meaning:

  • No significant difference

  • No changes

  • Nothing happened

  • No relationship between two parameters

  • Independent variable has no effect on dependent variable

Alternative Hypothesis (Ha​)

• Role: Contrary to H0

• Meaning:

  • Significant difference exists

  • There is an effect, change, or relationship

  • Independent variable does affect dependent variable

research hypotheses

• The H0

and H1 must be mutually exclusive

• The H0

and H1 must be exhaustive; that is, no

other possibilities can exist

• The H1 contains our ___

Evaluation of the Null

• In order to gain support for our research

hypothesis, we must reject the Null Hypothesis

• Thereby concluding that the alternative

hypothesis (likely) reflects what is going on in

the population.

• You can never “prove” the Alternative

Hypothesis!

Type I Error

•Rejection of the null hypothesis that is actually true

•Same as a “false positive”

•The alpha value gives us the probability of a Type I error.

Controlling Type I Error

•Alpha is the maximum probability of having a Type I error.

– E.g., 95% CI, chance of having a Type I is 5%

– Therefore, a 5% chance of rejecting H0

when H0

is true

– That is, 1 out of 20 hypotheses tested will result in Type I error

•We can control Type I error by setting a different α level.

Controlling Type I Error

•Particularly important to change α level to be more conservative if

calculating several statistical tests and comparisons.

•We have a 5% chance of getting a significant result just by chance.

So, if running 10 comparisons, should set a more conservative α level

to control for Type I error

– Bonferroni correction: .05/10 = .005 α

Type II Error

•We do not reject a null hypothesis that is false.

•Like a false negative

•E.g., thought the drug had no effect, when it

actually did

•The probability of a Type II error is given by the

Greek letter beta (β). This number is related to the

power or sensitivity of the hypothesis test, denoted

by 1 – β

Controlling Type II Error: Power

•Power: The power of a test sometimes, less formally, refers to the probability of rejecting the null when it is not correct.

•Power = P(reject H0 |H1

is true) = P(accept H1 |H1 is true)

•As the power increases, the chances of a Type II

error (false negative; β) decreases.

•Power = 1-β

•Power increases with sample size

Significance Level

The probability that the test statistic will reject the null

hypothesis when the null hypothesis is true

• Significance is a property of the distribution of a test statistic,

not of any particular draw of the statistic

• Determines the Region of Rejection

• Generally 5% or 1%

• (denoted by α) is the probability that the test statistic will fall in the critical region when the null hypothesis is actually true. Common choices for α are 0.05,

0.01, and 0.10.

Alpha Level

• The value of alpha (α) is associated with the

confidence level of our test; significance level.

• For results with a 90% level of confidence, the

value of α is 1 - 0.90 = 0.10.

• For results with a 95% level of confidence, the

value of alpha is 1 - 0.95 = 0.05.

• Typically set at 5% (.05) or 1% (.01)

critical region (or rejection region)

is the set of all values of the test statistic that cause us to reject

the null hypothesis.

Traditional method

1. Reject H0

2. Fail to reject H0

1. ___if the test statistic falls

within the critical region.

2. ___if the test statistic does not fall within the critical region.

P-value (probability value)

is the probability of getting a value of the test

statistic that is at least as extreme as the one

representing the sample data, assuming that

the null hypothesis is true. The null

hypothesis is rejected if the ___ is very

small, such as 0.05 or less.

P-value

• Reject H0 if the P-value ≤ α (where α is the significance level, such as

0.05).

• Accept H0 if the P-value > α.

confidence interval

estimate of a population parameter contains the likely

values of that parameter, reject a claim that the

population parameter has a value that is not

included