Statistical Significance and Confidence Intervals

  • Inference by Eye

    • Use 95% Confidence Intervals (CIs) to assess statistical significance.

  • Inferring Statistical Significance for a Single Mean

    • A mean is significantly different if it lies outside the 95% CI.
    • CIs that do not include zero indicate significance.

  • Inference by Eye for Two Means

    • Between-Subjects Design:

    • If 95% CIs for independent means overlap by no more than 25% of their total length, the difference is statistically significant.

    • Common significance levels:

      • p ≈ .05
      • p ≈ .01

    • Within-Subjects Design:

    • If CIs overlap at all, more analysis is needed to assess significance.

    • Overlap means difference may not be significant, but no overlap indicates significance.

  • 95% CIs of Condition Means

    • To find the difference:
    • Calculate each participant’s difference score between levels of the independent variable (IV).
    • Construct a 95% CI on the mean of these difference scores.
    • If zero is outside this CI, the difference is significant.

  • Paired-Samples t-test

    • If CIs overlap in a within-subjects design, conduct a paired-samples t-test for further analysis.
    • Example: Significant difference reported with p = .014.