Statistical Significance

Statistical Significance

  • Definition of Statistical Significance

    • Refers to whether a difference between two group mean scores or a correlation coefficient in a study is likely meaningful or merely due to random chance.

  • Example of Chance Outcomes

    • Consider a coin flip:

    • Flipping a fair coin 10 times and getting heads 7 times may seem statistically significant.

    • Expected outcome from chance for 10 flips is 5 heads, indicating potential luck rather than a true bias in the coin.

    • However, if the results were 70 heads out of 100 flips (or 700 out of 1,000), this outcome would lead to a reasonable assumption that the coin is not fair.

  • Role of Inferential Statistics

    • Inferential statistical methods help researchers calculate the probability that observed results are due to chance alone.

    • Determines significance of results derived from experiments or correlational studies.

  • Understanding p Values

    • The p statistic (short for probability) quantifies the level of significance in hypothesis testing.

    • When comparing two means:

    • The p value represents the probability that a difference as large as (or larger than) that observed would occur by chance if there were no real difference in the larger population.

    • This population refers to the entirety of scores from repeated experiments involving all possible subjects.

  • Implication for Correlational Studies

    • For correlations, p reflects the probability that a correlation coefficient as large as (or larger than) the observed one could arise by chance, given that the two variables are truly uncorrelated in the larger population.

  • Convention for Statistical Significance

    • Results are generally deemed statistically significant if the p value is less than 0.05 (5%).

    • Stating results are statistically significant implies that the likelihood of them occurring purely by chance is acceptably low (commonly less than 5%).

  • Consistency Across Studies

    • All experimental and correlational results presented in the textbook are statistically significant at the .05 level or greater.

  • Components of a Test of Statistical Significance

    • The specific formulas to calculate p values vary depending on the type of research conducted. Further exploration of these formulas is expected in subsequent content.