PS1007 #3Inferential Statistics, Hypothesis Testing, and Distributional Analysis

Foundations of Inferential Statistics and Sampling Theory

  • Core Purpose of Inferential Statistics:

    • Inferential statistics provides mathematical procedures to determine whether patterns, differences, or associations observed in a sample can be generalized to an entire population.

    • It tests whether sample findings represent genuine population effects or are merely artifacts of random sampling variability.

  • Population versus Sample:

    • Population: The entire collection of individuals, events, or measurements of interest (e.g., all adults in the United States, all UK psychology undergraduates, or all breast cancer patients in the UK). Population parameters (such as the true population mean, μ\mu, or population standard deviation, σ\sigma) are typically fixed but unknown.

    • Sample: A smaller, representative subset selected from the target population (e.g., 150150 adults randomly selected in the US, or 5050 psychology students). Sample statistics (such as the sample mean, xˉ\bar{x}, or sample standard deviation, ss) are computed directly from observed data to estimate population parameters.

  • Sampling Error and Random Variation:

    • Sampling error is the natural discrepancy between a sample statistic and the true population parameter resulting from taking a sample rather than testing the entire population.

    • Common synonymous terms used across scientific literature include:

    • Noise

    • Chance

    • Luck

    • Randomness

    • Random variation

    • Random error

    • Sampling error

    • Random fluctuation

    • Random sampling variability

    • The Fundamental Sampling Challenge: When drawing an empirical sample, it is impossible to know directly whether that specific sample closely resembles the population or deviates dramatically from it.

  • Foundational Sampling Theorems:

    • The Law of Large Numbers (LLN): States that as the sample size (nn) increases (n→∞n \rightarrow \infty), the sample mean (xˉ\bar{x}) converges closer and closer to the true population mean (μ\mu). Larger samples consistently yield more accurate and reliable estimates of population parameters.

    • The Glivenko–Cantelli Theorem (Fundamental Theorem of Statistics): Asserts that as the sample size increases (n→∞n \rightarrow \infty), the empirical sample distribution function converges uniformly to the true population cumulative distribution function, regardless of its shape.

    • The Central Limit Theorem (CLT):

    • Asserts that as sample size (nn) increases, the sampling distribution of the sample mean approaches a normal distribution, regardless of the shape of the underlying population distribution (whether uniform, exponential, skewed, or bimodal).

    • Simulation Demonstration: Taking 100100 repeated samples of increasing sizes (n=3n = 3, n=10n = 10, n=20n = 20, n=40n = 40, and n=80n = 80) from an extremely positively skewed population of 10,00010,000 scores demonstrates that the resulting distribution of sample means transforms from highly skewed into a bell-shaped, normal distribution.

    • Empirical Rule of Thumb: A sample size of at least 3030 (n≥30n \ge 30) is traditionally considered safe for invoking the CLT, though distributions with moderate variation often approximate normality at sample sizes as small as n=20n = 20.

    • The Mathematical Caveat: The CLT requires that the underlying distribution has a defined mean and variance. Pathological distributions that lack a defined mean (such as the Cauchy distribution) are exceptions, though they are virtually never encountered in empirical applied sciences.

  • Standard Deviation (SD\text{SD}) versus Standard Error of the Mean (SEM\text{SEM}):

    • Standard Deviation (SD\text{SD} or ss):

    • Quantifies the degree of dispersion or variability of individual raw scores around the sample mean.

    • Bakery Analogy: If a sample of 55 bakery muffins yields an average weight of 56.4 g56.4\,\text{g} with an SD\text{SD} of 7.5 g7.5\,\text{g}, this indicates that an individual muffin randomly selected from the batch will typically deviate from the sample mean by approximately 7.5 g7.5\,\text{g}.

    • Standard Error of the Mean (SEM\text{SEM} or SE\text{SE}):

    • Measures the precision or accuracy of the sample mean as an estimate of the true population mean.

    • Reflects the standard deviation of the sampling distribution of sample means across infinite hypothetical resamples of size nn.

    • Formula for SEM:       SEM=sn\text{SEM} = \frac{s}{\sqrt{n}}       Where ss is the sample standard deviation and nn is the sample size.

    • Worked Bakery Calculation: With s=7.5 gs = 7.5\,\text{g} and n=5n = 5:       SEM=7.55=7.52.236≈3.36 g\text{SEM} = \frac{7.5}{\sqrt{5}} = \frac{7.5}{2.236} \approx 3.36\,\text{g}       This indicates that if batches of 55 muffins are repeatedly sampled, the calculated sample means will typically vary by approximately 3.36 g3.36\,\text{g} from the true population mean.

    • Relationship with Sample Size: As nn increases, SEM\text{SEM} decreases in proportion to n\sqrt{n}. Small samples produce unstable, "shaky" means with high SEM\text{SEM}, whereas large samples yield stable estimates with low SEM\text{SEM}.

    • Standard errors can also be calculated for other sample statistics (such as the median, standard deviation, or qq-th quantile), though the term universally refers to the mean unless stated otherwise.

The Framework of Null Hypothesis Significance Testing (NHST)

  • Core Conceptual Structure of NHST:

    • NHST is the dominant inferential paradigm across psychology, medicine, and the broader social sciences.

    • It evaluates whether an empirical observation (e.g., an observed mean difference between two experimental groups) is sufficiently extreme to justify rejecting a default model of "pure chance."

Summary diagram of the components comprising Null Hypothesis Significance Testing
  • Formulation of Hypotheses:

    • Null Hypothesis (H0H_0):

    • The default, conservative baseline claim stating that there is no effect of an intervention, no difference between groups, or no association between variables in the population.

    • Expressed mathematically as a precise, single point parameter: Δ=0\Delta = 0, μ1−μ2=0\mu_1 - \mu_2 = 0, or μ1=μ2\mu_1 = \mu_2.

    • Examples across research domains:

      • Differences between groups: There is no difference in depression scores between a psychotherapy group and a control group.

      • Relationships between variables: There is no association between daily physical activity and telomere length.

      • Interventions: There is no difference in recovery time following myocardial infarction between patients receiving a new medication and those receiving standard care.

      • Predictive outcomes: The presence of a father during the first 55 years of life has no association with violent crime conviction rates by age 3030.

    • Alternative Hypothesis (H1H_1 or HaH_a):

    • The operational research hypothesis proposing that a real effect, difference, or association exists in the population.

    • Non-Directional (Two-Tailed): Postulates an effect in either direction (μ1≠μ2\mu_1 \ne \mu_2; e.g., "There is a difference in depression scores between the therapy group and the control group"). Used when there is no firm theoretical or empirical justification to predict polarity.

    • Directional (One-Tailed): Postulates an effect in a specific direction (μ1<μ2\mu_1 < \mu_2 or μ1>μ2\mu_1 > \mu_2; e.g., "The therapy group will have lower depression scores than the control group"). Requires rigorous a priori justification and cannot be changed post-hoc once data are collected.

  • Why Science Tests the Null Rather than the Alternative:

    • Scientific Parsimony and Legal Conservatism: Science defaults to the simplest model (no effect) unless empirical evidence forces a more complex explanation. This directly mirrors the legal doctrine where a defendant is presumed innocent until proven guilty beyond reasonable doubt.

    • Mathematical Tractability: The null hypothesis specifies an exact numerical value (e.g., difference =0= 0), allowing the exact construction of a probability distribution (the sampling distribution under H0H_0). In contrast, the alternative hypothesis encompasses an infinite range of non-zero values, making it impossible to construct a single sampling distribution without assuming an arbitrary effect size.

    • Error Control: Testing H0H_0 allows researchers to directly set and limit the probability of false positive conclusions (α\alpha).

  • The Four-Step NHST Procedure:

    • Step 1: Formulate clear null (H0H_0) and alternative (H1H_1) hypotheses prior to data collection.

    • Step 2: Collect empirical sample data and compute an appropriate test statistic (e.g., zz, tt, FF, or χ2\chi^2).

    • Step 3: Derive the pp-value associated with the observed test statistic under the sampling distribution assuming H0H_0 is true.

    • Step 4: Compare the pp-value against the a priori significance threshold (α\alpha):

    • If p≤αp \le \alpha: Reject H0H_0; the result is declared statistically significant.

    • If p>αp > \alpha: Fail to reject H0H_0; the result is declared statistically non-significant.

  • Understanding the pp-Value:

    • Verbatim Definition: The probability of obtaining a test statistic or sample result at least as extreme as the one observed, assuming that the null hypothesis is completely true.

    • Range: Mathematically bounded between 00 and 11 (0≤p≤10 \le p \le 1).

    • p=1.00p = 1.00: Observed sample result is completely identical to the null hypothesis expectation.

    • p=0.00p = 0.00: Observed result would be mathematically impossible if H0H_0 were true.

    • Smaller pp-values indicate increasing incompatibility between the collected sample data and the null hypothesis.

    • Significance Level (α\alpha):

    • The pre-set threshold determining the tolerable risk of committing a Type I error.

    • Conventional standard is α=0.05\alpha = 0.05 (5%5\%, or a 1-in-201\text{-in-}20 chance).

    • Stricter thresholds (e.g., α=0.01\alpha = 0.01 or α=0.00001\alpha = 0.00001) are adopted in high-stakes fields like clinical pharmacology or particle physics to reduce false positives to 1 in 100,0001\text{ in }100,000.

    • More lenient thresholds (e.g., α=0.20\alpha = 0.20) might be tolerated in non-critical scenarios (such as predicting ice cream truck arrival times), accepting a 2-in-102\text{-in-}10 false-positive risk.

  • Critical Interpretive Caveats:

    • Never "Accept" the Null Hypothesis: A non-significant result (p>0.05p > 0.05) only indicates insufficient evidence to reject H0H_0. It does not demonstrate that the true difference is zero. Retaining H0H_0 may stem from low statistical power, excessive measurement noise, small sample size, or poor design.

    • Rejecting H0H_0 Does Not "Prove" H1H_1: A statistically significant result indicates that the data are improbable under H0H_0, not that the research theory is conclusively confirmed.

    • Statistical Significance versus Effect Size:

    • The pp-value conveys nothing about the magnitude or real-world importance of an effect.

    • Empirical Illustration:

      • Experiment 1 (Small Sample): A 6-point6\text{-point} difference between Drug A and Drug B yields a non-significant p=0.24p = 0.24.

      • Experiment 2 (Large Sample): A tiny 1-point1\text{-point} difference between Drug A and Drug B yields a statistically significant p=0.04p = 0.04.

    • With a sufficiently large sample, trivial and clinically meaningless differences will reliably achieve statistical significance (p<0.05p < 0.05).

Sampling Distributions and Hypothesis Testing in Practice

  • The Nature of Sampling Distributions:

    • A sampling distribution is the theoretical probability distribution of a specific statistic (e.g., mean, difference between means, median, tt, FF) that would emerge from drawing an infinite number of random samples of size nn from a specified population under H0H_0.

    • It maps the area under the curve directly to probability, establishing which sample outcomes are probable under chance and which are extreme.

  • The Standard Normal Distribution and Empirical Probabilities:

    • A standard normal distribution is symmetric, unimodal, and bell-shaped, characterized by specific fixed areas under the curve:

    • ±1.00 SD\pm 1.00\,\text{SD} contains 68.0%68.0\% (or 68.26%68.26\%) of all observations (34.1%34.1\% on each side of the mean).

    • ±2.00 SD\pm 2.00\,\text{SD} contains 95.5%95.5\% (or 95.44%95.44\%) of observations (13.6%13.6\% between 1 SD1\,\text{SD} and 2 SD2\,\text{SD}).

    • ±3.00 SD\pm 3.00\,\text{SD} contains 99.7%99.7\% (or 99.74%99.74\%) of observations (2.1%2.1\% between 2 SD2\,\text{SD} and 3 SD3\,\text{SD}, with 0.1%0.1\% in each tail beyond 3 SD3\,\text{SD}).

    • Exactly ±1.96 SD\pm 1.96\,\text{SD} encapsulates 95.0%95.0\% of the total area under the curve, leaving exactly 2.5%2.5\% in each tail.

Standard normal probability density distribution showing percentage areas across standard deviations
  • Worked Empirical Case Study: Height Differences by Pet Preference:

    • Research Question: Are individuals who prefer dogs as pets taller than individuals who prefer cats as pets?

    • Hypotheses:

    • H0:μDog=μCatH_0: \mu_{\text{Dog}} = \mu_{\text{Cat}} (no height difference, μdiff=0\mu_{\text{diff}} = 0).

    • H1:μDog≠μCatH_1: \mu_{\text{Dog}} \ne \mu_{\text{Cat}} (two-tailed non-directional difference).

    • Sample Data (N=100N = 100, 5050 per group):

    • Dog owners (nDog=50n_{\text{Dog}} = 50): Mean height xˉ=167.67 cm\bar{x} = 167.67\,\text{cm}, standard deviation s=2.71 cms = 2.71\,\text{cm}.

    • Cat owners (nCat=50n_{\text{Cat}} = 50): Mean height xˉ=164.57 cm\bar{x} = 164.57\,\text{cm}, standard deviation s=1.82 cms = 1.82\,\text{cm}.

    • Observed difference between sample means:       Δxˉ=167.67−164.57=3.10 cm\Delta \bar{x} = 167.67 - 164.57 = 3.10\,\text{cm}

  • Standard Error of the Mean Difference (SEMD):

    • The standard deviation of the sampling distribution of the difference between two independent sample means.

    • Formula:     SEMD=sA2nA+sB2nB\text{SEMD} = \sqrt{\frac{s_A^2}{n_A} + \frac{s_B^2}{n_B}}     Where sAs_A and sBs_B are the sample standard deviations, and nAn_A and nBn_B are the sample sizes.

    • Calculation with Actual Study Data:     SEMD=2.71250+1.82250=7.344150+3.312450=0.1469+0.0662=0.2131≈0.46 cm\text{SEMD} = \sqrt{\frac{2.71^2}{50} + \frac{1.82^2}{50}} = \sqrt{\frac{7.3441}{50} + \frac{3.3124}{50}} = \sqrt{0.1469 + 0.0662} = \sqrt{0.2131} \approx 0.46\,\text{cm}

  • Mathematical Determinants of SEMD Spread:

    • Effect of Sample Size (nn) holding SD constant (sDog=2.71s_{\text{Dog}} = 2.71, sCat=1.82s_{\text{Cat}} = 1.82):

    • Small sample (n=25n = 25 per group):       SEMD=2.71225+1.82225=0.2938+0.1325=0.4263≈0.65 cm\text{SEMD} = \sqrt{\frac{2.71^2}{25} + \frac{1.82^2}{25}} = \sqrt{0.2938 + 0.1325} = \sqrt{0.4263} \approx 0.65\,\text{cm}

    • Baseline sample (n=50n = 50 per group): SEMD≈0.46 cm\text{SEMD} \approx 0.46\,\text{cm}.

    • Large sample (n=100n = 100 per group):       SEMD=2.712100+1.822100=0.0734+0.0331=0.1065≈0.33 cm\text{SEMD} = \sqrt{\frac{2.71^2}{100} + \frac{1.82^2}{100}} = \sqrt{0.0734 + 0.0331} = \sqrt{0.1065} \approx 0.33\,\text{cm}

    • Principle: As sample size increases, standard error decreases, producing a narrower, taller sampling distribution.

    • Effect of Underlying Sample Variance (ss) holding sample size constant (n=50n = 50):

    • Lower variance (sDog=1.36s_{\text{Dog}} = 1.36, sCat=0.91s_{\text{Cat}} = 0.91):       SEMD=1.36250+0.91250=0.0370+0.0166=0.0536≈0.23 cm\text{SEMD} = \sqrt{\frac{1.36^2}{50} + \frac{0.91^2}{50}} = \sqrt{0.0370 + 0.0166} = \sqrt{0.0536} \approx 0.23\,\text{cm}

    • Baseline variance (sDog=2.71s_{\text{Dog}} = 2.71, sCat=1.82s_{\text{Cat}} = 1.82): SEMD≈0.46 cm\text{SEMD} \approx 0.46\,\text{cm}.

    • Higher variance (sDog=5.42s_{\text{Dog}} = 5.42, sCat=3.64s_{\text{Cat}} = 3.64):       SEMD=5.42250+3.64250=0.5875+0.2650=0.8525≈0.92 cm\text{SEMD} = \sqrt{\frac{5.42^2}{50} + \frac{3.64^2}{50}} = \sqrt{0.5875 + 0.2650} = \sqrt{0.8525} \approx 0.92\,\text{cm}

    • Principle: As underlying population/sample standard deviation increases, standard error increases, widening the sampling distribution.

Sampling distributions illustrating narrow, medium, and wide spread under different levels of variance
  • Evaluating Significance against Critical Thresholds:

    • Under H0H_0, the sampling distribution of the mean difference is centered at 0 cm0\,\text{cm}.

    • The 95%95\% acceptance region is bounded by ±1.96×SEMD\pm 1.96 \times \text{SEMD}.

    • For SEMD=0.46 cm\text{SEMD} = 0.46\,\text{cm}, the critical thresholds are:     ±1.96×0.46 cm=±0.90 cm\pm 1.96 \times 0.46\,\text{cm} = \pm 0.90\,\text{cm}

    • Because the observed sample difference of 3.10 cm3.10\,\text{cm} lies far beyond +0.90 cm+0.90\,\text{cm} (exceeding z=+6.7z = +6.7), it is exceedingly unlikely to have occurred through random sampling error alone. H0H_0 is rejected (p<0.05p < 0.05).

    • Thresholds Across Alpha Levels (Two-Tailed):

    • α=0.05\alpha = 0.05: Critical z=±1.96z = \pm 1.96 (95%95\% area under the curve; 2.5%2.5\% in each tail).

    • α=0.01\alpha = 0.01: Critical z=±2.58z = \pm 2.58 (99%99\% area under the curve; 0.5%0.5\% in each tail).

    • α=0.001\alpha = 0.001: Critical z=±3.29z = \pm 3.29 (99.9%99.9\% area under the curve; 0.05%0.05\% in each tail).

Decision Errors and Statistical Power

  • The Four Potential Outcomes in Decision Theory:

    • Crossing the true state of reality (H0H_0 is true vs. H0H_0 is false) with the statistical decision (reject H0H_0 vs. fail to reject H0H_0) creates a 2×22 \times 2 decision matrix:

Decision matrix showing Type I error, Type II error, True Negative, and Power
  • True Negative (Correct Decision): Null hypothesis is true in reality, and the test fails to reject H0H_0. Probability =1−α= 1 - \alpha.

  • False Positive (Type I Error, α\alpha):

    • The null hypothesis is true in reality, but the test incorrectly rejects H0H_0.

    • Concluding there is an effect or difference when none actually exists.

    • Equal to the pre-set significance level (α\alpha; typically 0.050.05).

    • Examples: Convicting an innocent person in court; concluding a completely ineffective drug lowers blood pressure because a random sample showed a lower mean purely by chance.

  • False Negative (Type II Error, β\beta):

    • The null hypothesis is false in reality (a real difference exists), but the test fails to reject H0H_0.

    • Concluding there is no effect when an effect actually exists.

    • Probability designated as β\beta.

    • Examples: Acquitting a guilty criminal due to insufficient evidence; concluding an effective medication does not work because sampling noise obscured the effect.

  • True Positive (Statistical Power, 1−β1 - \beta):

    • The null hypothesis is false in reality, and the test correctly rejects H0H_0.

    • The sensitivity or ability of a test to detect a genuine effect of a given magnitude.

  • Mathematical Row Probabilities:

    • When H0H_0 is true: (1−α)+α=1.0(1 - \alpha) + \alpha = 1.0

    • When H0H_0 is false: β+(1−β)=1.0\beta + (1 - \beta) = 1.0

    • Probability Theory and Chance Extremes:

  • The Infinite Monkey Theorem: An infinite number of monkeys randomly striking typewriter keys for an infinite duration will eventually produce every finite text, including the complete works of William Shakespeare, word-for-word, purely by random permutation.

  • Dice Rolling Example: Rolling 1010 consecutive sixes on a fair six-sided die has an exact probability of:     (16)10=0.000000016538\left(\frac{1}{6}\right)^{10} = 0.000000016538     This is roughly a 1 in 60.5 million1\text{ in }60.5\text{ million} chance. Although extraordinarily rare, it is entirely possible under pure randomness. NHST accounts for this by accepting an explicit 5%5\% error rate (α=0.05\alpha = 0.05) of rare sampling fluctuations.

    • Interrelationships Governing Statistical Power (1−β1 - \beta):

  • In a continuous sampling model comparing a null distribution (centered at Δ=0\Delta = 0) against an alternative distribution (centered at the true effect size Δ\Delta):

    • The Type I error (α\alpha) defines the critical cutoff line on the null curve.

    • The area under the alternative curve that falls to the left of this cutoff is the Type II error (β\beta).

    • The remaining area under the alternative curve to the right of the cutoff is Statistical Power (1−β1 - \beta).

  • Three Factors Dictating Power:

    • Sample Size (nn): Increasing nn reduces standard error, making both the null and alternative distributions skinnier and narrower. This reduces the distribution overlap, shrinking β\beta and substantially increasing power (1−β1 - \beta).

    • Sample/Population Variability (ss): Increasing underlying noise or standard deviation widens and flattens both curves, increasing their overlap and diminishing statistical power.

    • Effect Size (Δ\Delta): A larger true difference pushes the alternative distribution further away from the null distribution along the horizontal axis, minimizing overlap and increasing power.

    • One-Tailed versus Two-Tailed Significance Tests:

  • Two-Tailed Test (Non-Directional):

    • Evaluates the possibility of an effect in either direction (μ1≠μ2\mu_1 \ne \mu_2).

    • The critical alpha region (α=0.05\alpha = 0.05) is evenly divided into two symmetric tails (α/2=0.025\alpha / 2 = 0.025 in each tail).

    • Critical ±z=±1.96\pm z = \pm 1.96.

    • For an SEMD of 1.33 cm1.33\,\text{cm}, the critical difference threshold is ±2.61 cm\pm 2.61\,\text{cm}.

Sampling distribution illustrating a two-tailed rejection region at alpha 0.05
  • One-Tailed Test (Directional):

    • Evaluates an effect in only one specific, predicted direction (μ1>μ2\mu_1 > \mu_2 or μ1<μ2\mu_1 < \mu_2).

    • The entire 5%5\% alpha region is placed in one single tail (α=0.05\alpha = 0.05).

    • Critical z=+1.645z = +1.645 (or −1.645-1.645).

    • For an SEMD of 1.33 cm1.33\,\text{cm}, the critical difference threshold is +2.19 cm+2.19\,\text{cm}.

Sampling distribution illustrating a one-tailed rejection region at alpha 0.05
  • Methodological Comparison:

    • A two-tailed test at α=0.05\alpha = 0.05 has the exact same overall Type I error rate as a one-tailed test at α=0.05\alpha = 0.05, but has less power to detect an effect in the targeted direction because the critical threshold is set further out (1.961.96 vs. 1.6451.645).

    • Directional hypotheses must be fixed before data collection. Changing from a two-tailed to a one-tailed test post-hoc after seeing sample results is methodologically invalid because it doubles the effective Type I error rate.

Distributional Characteristics and Assessing Normality

  • Anatomy of the Normal Distribution:

    • A perfectly normal distribution exhibits three distinct mathematical properties:

    • Modality: Unimodal (possessing a single peak where mean, median, and mode coincide exactly).

    • Skewness: Perfectly symmetric with zero skew (Skewness=0\text{Skewness} = 0).

    • Kurtosis: Mesokurtic (Kurtosis=0\text{Kurtosis} = 0 in excess kurtosis notation, or 33 in raw moment notation).

  • Distributional Departures from Normality:

    • Modality Variations: Distributions can be bimodal (two distinct peaks, such as eruption intervals for the Old Faithful geyser) or uniform (completely flat, where every value across a interval has equal probability).

    • Skewness (Asymmetry):

    • Positive (Right) Skew: The right tail is elongated with excess scores clustering at the lower end (e.g., student statistics anxiety scores, household wealth). Mean exceeds median (xˉ>Mdn\bar{x} > \text{Mdn}).

    • Negative (Left) Skew: The left tail is elongated with excess scores clustering at the higher end (e.g., high student screen time, easy exam scores). Mean is less than median (xˉ<Mdn\bar{x} < \text{Mdn}).

Visual representation of positive skew, zero skew, and negative skew distributions
  • Kurtosis (Peakedness and Tail-Fatness):

    • Leptokurtic (>0> 0 excess kurtosis, or >3> 3 raw): Sharply pointed central peak with heavy, fat tails containing excess outliers relative to a normal distribution.

    • Platykurtic (<0< 0 excess kurtosis, or <3< 3 raw): Broad, flattened central peak with light, thin tails and fewer outliers.

    • Mesokurtic (=0= 0 excess kurtosis, or =3= 3 raw): Standard bell-shaped normal curve.

Visual representation comparing platykurtic, mesokurtic, and leptokurtic distributions
  • Three-Pronged Approach to Assessing Normality:

    • Method 1: Descriptive Statistics (Skewness and Kurtosis):

    • Rules of Thumb for Absolute Values:

      • Value <1.0< 1.0: Mild skewness/kurtosis (close to normal).

      • Value between 1.01.0 and 2.02.0: Moderate skewness/kurtosis.

      • Value >2.0> 2.0: Extreme deviation from normality.

    • Z-Score Significance Transformation:       To evaluate whether observed skewness or kurtosis is statistically significantly different from zero, divide the statistic by its respective standard error:       Zskewness=SkewnessSEskewnessZ_{\text{skewness}} = \frac{\text{Skewness}}{\text{SE}_{\text{skewness}}}       Zkurtosis=KurtosisSEkurtosisZ_{\text{kurtosis}} = \frac{\text{Kurtosis}}{\text{SE}_{\text{kurtosis}}}

      • If the calculated ZZ-score falls outside the range of −1.96-1.96 to +1.96+1.96, the deviation from normality is statistically significant at α=0.05\alpha = 0.05.

    • Method 2: Formal Statistical Tests of Normality:

    • Hypotheses Tested:

      • H0H_0: The sample data are drawn from a normally distributed population.

      • H1H_1: The sample data are not drawn from a normally distributed population.

    • Decision Rule:

      • If p>0.05p > 0.05: Fail to reject H0H_0; normality can be safely assumed.

      • If p≤0.05p \le 0.05: Reject H0H_0; data depart significantly from a normal distribution.

    • Available Tests and Sample Size Sensitivity:

      • Shapiro-Wilk Test: Recommended as the most powerful test for small to moderate samples (N<50N < 50). Not suitable for very large datasets (N>5000N > 5000).

      • Kolmogorov-Smirnov Test (with Lilliefors significance correction): Suited for larger samples (N≥50N \ge 50), but lacks statistical power in small samples.

      • The Large-Sample Fallacy: Both tests are overly sensitive when sample sizes exceed 100100 (N>100N > 100), frequently flagging trivial, clinically inconsequential deviations as statistically significant (p<0.05p < 0.05), producing false positives. Methodologists (e.g., Howell, 2010, Statistical Methods for Psychology) caution against relying solely on formal tests without visual plots.

    • Method 3: Visual Diagnostic Inspection:

    • Quantile-Quantile (Q-Q) Plots: Plots observed sample quantiles against quantiles expected from a theoretical normal distribution. If the sample is normal, the observed points adhere tightly to the diagonal reference line. Systematic curved departures reveal skewness, while S-shaped deviations reveal kurtosis.

    • Box Plots: Displays median, interquartile range (25th25\text{th} to 75th75\text{th} percentiles), minimum, maximum, and directly flags isolated individual outliers.

  • SPSS Worked Diagnostic Example:

    • Evaluating two continuous variables (VAR00001 and VAR00002), each with N=20N = 20:

SPSS statistical output table showing Kolmogorov-Smirnov and Shapiro-Wilk tests of normality
  • Descriptive Analysis for VAR00001:

    • Mean =−0.2865= -0.2865 (SE=0.23454\text{SE} = 0.23454); Median =−0.2900= -0.2900; Variance =1.100= 1.100; SD=1.04891\text{SD} = 1.04891.

    • Skewness =−0.486= -0.486 (SE=0.512\text{SE} = 0.512):     Zskewness=−0.4860.512=−0.95Z_{\text{skewness}} = \frac{-0.486}{0.512} = -0.95     Because −0.95-0.95 falls within ±1.96\pm 1.96, skewness is mild and non-significant.

    • Kurtosis =1.494= 1.494 (SE=0.992\text{SE} = 0.992):     Zkurtosis=1.4940.992=+1.51Z_{\text{kurtosis}} = \frac{1.494}{0.992} = +1.51     Because +1.51+1.51 falls within ±1.96\pm 1.96, kurtosis is non-significant.

    • Formal Tests: Shapiro-Wilk statistic =0.955= 0.955, df=20df = 20, p=0.457p = 0.457; Kolmogorov-Smirnov statistic =0.116= 0.116, df=20df = 20, p=0.200p = 0.200. Because p>0.05p > 0.05, normality is retained.

    • Visual Diagnostics: The Q-Q plot points closely follow the diagonal line. However, the box plot flags case number 1818 (score=−3.03\text{score} = -3.03) as an isolated low outlier.

Normal Q-Q Plot for VAR00001 displaying data points scattered along the diagonal reference lineBox plot for VAR00001 identifying outlier case 18 below the lower whisker
  • Descriptive Analysis for VAR00002:

    • Mean =33235.30= 33235.30 (SE=6725.582\text{SE} = 6725.582); Median =21624.79= 21624.79; Variance =904669162.67= 904669162.67; SD=30077.719\text{SD} = 30077.719.

    • Substantial discrepancy between mean (33235.3033235.30) and median (21624.7921624.79) indicates pronounced asymmetry.

    • Skewness =0.855= 0.855 (SE=0.512\text{SE} = 0.512); Kurtosis =−0.277= -0.277 (SE=0.992\text{SE} = 0.992).

    • Formal Tests: Shapiro-Wilk statistic =0.876= 0.876, df=20df = 20, p=0.015p = 0.015; Kolmogorov-Smirnov statistic =0.234= 0.234, df=20df = 20, p=0.005p = 0.005. Because p≤0.05p \le 0.05, the null hypothesis of normality is rejected.

    • Visual Diagnostics: The Q-Q plot shows systematic non-linear bowing away from the diagonal line, confirming non-normality.

Normal Q-Q Plot for VAR00002 showing non-linear departures from the expected normal line
  • SPSS Step-by-Step Procedure for Normality Testing:

    • Step 1: Navigate through the top toolbar via Analyze →\rightarrow Descriptive Statistics →\rightarrow Explore....

    • Step 2: Transfer target continuous variables into the Dependent List box. (If comparing groups, place the categorical grouping variable into the Factor List to check distributions separately per group).

    • Step 3: Click the Plots... button on the right panel.

    • Step 4: Under the Descriptive menu, check Histogram. Under Boxplots, choose Factor levels together.

    • Step 5: Check the box labeled Normality plots with tests.

    • Step 6: Click Continue, then click OK to generate full descriptive tables, Kolmogorov-Smirnov, Shapiro-Wilk, Q-Q plots, and box plots.