Probability, Sampling Distributions, and Inference Essentials

Randomness, Probability, and the Law of Large Numbers

• Random Variable: An outcome variable whose individual results are uncertain, but displays a regular, predictable pattern over a large number of repetitions.

• Chance Behavior: Unpredictable in the short run, but predictable in the long run.

• Law of Large Numbers: As the number of independent observations $n$ sampled increases, the sample statistic gets closer and closer to the true population parameter.

• Coin Flip Example: The proportion of heads fluctuates wildly for a small number of tosses but stabilizes at $0.5$ as the number of tosses reaches $5000$.

Normal Probability in a Population

• Normal Distribution: A bell-shaped density curve represented as $N(\text{mean}, \text{SD})$ or $N(\frac{x - \text{mean}}{\text{SD}})$, defined by population mean $\text{mean}$ and standard deviation $\text{SD}$.

• Empirical Rule ($68\text{-}95\text{-}99.7$ Rule):

  • $68\%$ of observations fall within $\text{mean} \pm 1\text{SD}$.

  • $95\%$ of observations fall within $\text{mean} \pm 2\text{SD}$.

  • $99.7\%$ of observations fall within $\text{mean} \pm 3\text{SD}$.

• Standardized Score (z-score): Measures the position of an individual value relative to the whole population:   $z = \frac{x - \text{mean}}{\text{SD}}$

• Density Curve Area: Total area under a density curve is exactly $1$ (corresponding to $100\%$ total probability).

• Standard Normal Curve (Table A): Provides the area under the curve to the left of a calculated $z$\text{-score}.

Population Parameters vs. Sample Statistics

• Parameter: Fixed, unknown numerical value associated with a population.

  • Population Mean: $\text{mean}$

  • Population Standard Deviation: $\text{SD}$

  • Population Proportion: $p$

• Statistic: Variable numerical value computed from a sample to estimate a population parameter.

  • Sample Mean: $\bar{x}$

  • Sample Standard Deviation: $s$

  • Sample Proportion: $\hat{p}$

• Example: In a survey of $1600$ Australian adults where $24\%$ watch a specific television program, the statistic is $\hat{p} = 0.24$ ($24\%$), the sample size is $n = 1600$, and the parameter $p$ is the proportion of all Australian adults watching the program.

Sampling Distributions and the Central Limit Theorem

• Sampling Distribution: The frequency distribution of all possible values of a statistic obtained from repeated random samples of the same size $n$ from the same population.

• Properties of Sampling Distribution of Sample Means ($\bar{x}$):

  • Centre: Equal to the population mean ($\text{mean}_{\bar{x}} = \text{mean}$). Unbiased estimator.

  • Spread (Standard Error): Decreases as sample size increases:     $\text{SD}(\bar{x}) = \frac{\text{SD}}{\sqrt{n}}$

• Effect of Sample Size: Larger samples produce less spread in sample means, yielding tighter and more precise estimates of population values.

• Central Limit Theorem (CLT):

  • If individual population observations follow $N(\text{mean}, \text{SD})$, the sample mean $\bar{x}$ follows $N\left(\text{mean}, \frac{\text{SD}}{\sqrt{n}}\right)$ for any sample size $n$.

  • If the population distribution is skewed or non-Normal, the sampling distribution of $\bar{x}$ becomes approximately Normal as sample size $n$ increases (typically $n > 30$).

Standardized Statistic for Sample Means

• Formula: Evaluates how far a sample mean $\bar{x}$ deviates from a hypothesized population mean $\text{mean}$ in standard error units:   $z = \frac{\bar{x} - \text{mean}}{\frac{\text{SD}}{\sqrt{n}}}$

• Standard Normal Distribution: When conditions are met, this test statistic approximately follows $N(0, 1)$.

Statistical Inference Overview

• Statistical Inference: Process of making decisions or drawing conclusions about a population based on a random sample.

• Confidence Intervals: Uses sample data to estimate an interval likely containing the true population parameter.

• Hypothesis/Significance Testing: Tests sample evidence against a hypothesized population parameter value.

Worked Example: Pizza Weights

• Population Data: Pizza weight $X \sim N(450\,g, 28\,g)$.

• Individual Pizza Weight Probability ($X < 430\,g$):

  • Standard score calculation: $z = \frac{430 - 450}{28} = -0.714 \approx -0.71$

  • Cumulative probability lookup (Table A): $P(X < 430\,g) = 0.2389$ (or $23.89\%$).

• Sampling Distribution for Sample Size $n = 4$:

  • Center: $\text{mean}_{\bar{x}} = 450\,g$

  • Standard Error: $\text{SD}(\bar{x}) = \frac{28}{\sqrt{4}} = \frac{28}{2} = 14\,g$

  • Distribution: $\bar{x} \sim N(450\,g, 14\,g)$

• Sample Average Weight Probability ($\bar{x} < 430\,g$):

  • Standard score calculation: $z = \frac{430 - 450}{14} = -1.43$

  • Cumulative probability lookup (Table A): $P(\bar{x} < 430\,g) = 0.0764$ (or $7.64\%$).