WK10: Statistical inference: population proportion: Relationship between CI and test of significance

Relationship Between Confidence Intervals and Tests of Significance

Introduction

This video discusses the relationship between conclusions drawn from statistical inference using confidence intervals and tests of significance. The principles apply to Z and T procedures for:

  • Single population means
  • Differences between two population means
  • Population proportions

Two-Sided Procedures

  • Both confidence intervals and two-sided tests of significance are two-sided procedures.
  • If the level of confidence and the level of significance are complementary (sum to 100%), the conclusions about the parameter of interest will coincide.

Confidence Intervals and Claimed Parameter Values

  • Claimed Value Inside the Confidence Interval:
    • Conclusion (Confidence Interval): No change or no difference.
    • Statistical Decision (Two-Sided Test): Fail to reject the null hypothesis.
    • Conclusion (Two-Sided Test): No change or no difference.
  • Claimed Value Outside the Confidence Interval:
    • Conclusion (Confidence Interval): There is a change or difference.
    • Statistical Decision (Two-Sided Test): Reject the null hypothesis.
    • Conclusion (Two-Sided Test): There is a change or difference.

Example

  • Known population height: 171 cm.
  • Question: Has the mean height changed?
  • 95% Confidence Interval (based on sample data): 171.17 cm to 174.61 cm.
  • Since 171 is not within the interval, we are 95% confident there is a difference.

Test of Significance

  • Level of significance: 5% (0.05).
  • Test statistic: 2.2.
  • P-value range (two-sided test): 0.02 to 0.04.
  • Since the P-value < 0.05, we reject the null hypothesis.
  • Conclusion: The data provide statistically significant evidence supporting a difference.

One-Sided Tests

  • Conclusions from confidence intervals and one-sided tests don't necessarily coincide.

Example Scenario

  • Test of significance: Level of significance = 5%.
  • P-value (one-sided test): 0.025 to 0.05.
  • Statistical decision: Reject the null hypothesis; claim a change/difference.
  • If the test were two-sided, the P-value range would be 0.05 to 0.1.
  • In this case, we would fail to reject the null hypothesis; claim no change/difference.

Reconciliation of One-Sided Test and Confidence Interval

  • To have conclusions coincide with a one-sided test at a 5% significance level, we need to account for 5% under both tails of the distribution.
  • This means comparing a 5% level of significance in a one-sided test with a 90% confidence interval.