BIOL 315 Module 2: Hypothesis Testing, Statistical Inference, Experimental Design

What are the four procedural steps of hypothesis testing?

1. State H₀ and H_A

2. Compute test statistic to compare sample to a null model (null distribution) that represents H₀

3. Determine the P-value

4. Draw appropriate conclusions

What does the P-value represent?

P is the probability of observing a test statistic as extreme as, or more extreme than, the one observed assuming H₀ is true

What is a null model?

A sampling distribution that shows the probability of observing a specific value assuming that the null hypothesis is true

What is the significance level (α)?

• A probability used as a criterion for rejecting the null hypothesis

• The most common α value in statistics is 0.05

• If P value > α, you fail to reject the null hypothesis

• If P value ≤ α, you reject the null hypothesis

What is a two-tailed test?

• A statistical test where deviation in either direction would reject the null hypothesis (e.g. H₀: p = 0.5, H_A: p ≠ 0.5)

• Normally, α is divided into α/2 on one side and α/2 on the other

What is a Type I error (α)?

• rejecting a true null hypothesis (false positive)

• determined by the significance level α (i.e. if α = 0.05, probability of making a Type I error is 5%

What is a Type II error (β)?

Failing to reject a false null hypothesis (false negative)

What is power?

The probability that a random sample will lead to rejection of a false null hypothesis

• Power = 1 - β

• The smaller the β (Type II error), the greater the power of the test

• Generally, the minimum recommended power = 0.80

What things does power depend on?

1. How different the truth is from the null hypothesis

   • This is called the “effect size.” Detecting a big difference is easier than detecting a small one

2. Type I error rate

   • If Type I error rate decreases, Type II error rate increases, and vice versa (there is a trade-off)

3. Sample size

   • Larger sample size reduces sampling error in the estimate

4. Variability in the population

   • A population with lower variability for the test statistic has a higher power

What is d, and what can you do if you suspect it is small?

• d is the difference between two group means in terms of standard deviation (0.2 is small, 0.5 is medium, 0.8 is large)

• If we suspect a small, but biologically meaningful, difference between two populations or treatments. we can:

  • Increase sample size (note: don’t keep collecting more samples if you don’t like your initial P value)

  • Increasing significance level (α) (note: increasing your α increases the likelihood of falsely rejecting a true null hypothesis)

What are the three ways to reduce bias in an experimental design?

1. Have a control group

   • A control group is made up of units that are treated the same as the experimental units, and are under the same conditions, except that they do not receive the treatment

   • Should be treated simultaneously or at randomized times

   • Provides a baseline for comparison

2. Use randomization

   • Random assignment of the treatment to the experimental unit

   • Minimizes the effects of any confounding variables

3. Use blinding

   • Concealing information about whether a participant is in the control or treatment (single group), and sometimes concealing it from researchers as well (double blind)

   • Reduces bias by preventing modification of behaviour based on the treatment

What are the three main ways to decrease sampling error in an experimental design?

1. Use replication

   • Apply treatment to multiple, independent experimental subjects or units

2. Ensure balance

   • Have an equal number of units in the treatment and the control

3. Use blocking

   • Divide experimental units into groups with known confounding variables (e.g. location, belonging to the same family)

   • Treatments are randomly assigned within blocks

   • Differences between treatments are evaluated only within blocks

What is pseudoreplication?

• When a control and treatment are not independently or randomly allocated because they are confounded with a chamber

• e.g. 2 chambers with 4 plants each with the same treatment in each chamber → it seems like there are 8 replicate units, but there are only 2 truly independent units