Errors

  • Errors
    • Type 1: rejected the null hypothesis that is usually true. sample is usually unrepresentative
    • Type 2: fail to reject a false null hypothesis. sample size usually too small
  • A result is statistically significant if the result is sufficient to reject the null hypothesis
  • One-tailed / two-tailed
    • most hypothesis tests are two-tailed: we test for an increase or decrease
    • if the effect is positive, our criterion is lower to reject the null
  • Effect Size
    • if we have a large effect, we can observe a significant result in a small sample
    • we can also find small effects with large samples
    • cohen’s d measures the size of an effect
    • measures differences between distributions
  • statistical power: probability that the test will reject the null hypothesis (treatment has an effect)
    • depends on size of treatment effect + size of sample
  • the larger the effect, the greater the power of the test