Continuous Probability Distributions - Comprehensive Study Guide

Targeted Continuous Probability Distributions and Sampling Distributions

• Targeted Continuous Probability Distributions:

  • Normal (Gaussian) Distribution
  • Standard Normal
  • Exponential Distribution
  • Gamma Distribution
  • Beta Distribution
  • Pareto Distribution

• Sampling Distributions:

  • $F$ distribution
  • $t$ distribution
  • Chi-square distribution

Normal (Gaussian) Distribution

• Definition and Characteristics:

  • The Normal distribution, also known as the Gaussian distribution, is a probability distribution that is symmetric about the mean.
  • In this distribution, the mean, median, and mode are all equal.
  • The distribution indicates that data near the mean occur more frequently than data far from the mean.
  • The normal curve is bell-shaped.
  • Notation: A random variable $X$ in a normal distribution is denoted as $X \sim N(\mu, \sigma^2)$.

• Parameters:

  • Mean ($\mu$): A location parameter that defines the central tendency and the location of the peak. Changing $\mu$ shifts the curve left or right on the horizontal axis.
  • Standard Deviation ($\sigma$) or Variance ($\sigma^2$): A scale parameter that defines the width and variability. Larger standard deviations produce a more spread-out distribution, while smaller ones tighten the distribution.

• Naturally Occurring Examples:

  • Height
  • IQ scores
  • Blood pressure
  • Measurement error

• Probability Density Function (pdf):

  • $f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2}(\frac{x-\mu}{\sigma})^2}$

• Symmetric Property and Empirical Rules:

  • The distribution cannot model skewed data.
  • Empirical Rule (68.3-95.4-99.7):
    • $P(\mu - \sigma < X < \mu + \sigma) = 0.6826$ (approximately $68.3\%$)
    • $P(\mu - 2\sigma < X < \mu + 2\sigma) = 0.9544$ (approximately $95.4\%$)
    • $P(\mu - 3\sigma < X < \mu + 3\sigma) = 0.9973$ (approximately $99.7\%$

Moment Generating Function (MGF) and Moments for Normal Distribution

• Derivation of MGF:

  • $M_x(t) = \frac{1}{\sigma\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{tx} e^{-\frac{1}{2\sigma^2}(x-\mu)^2} dx$
  • After completing the square in the exponent and evaluating the integral:
  • $M_x(t) = e^{\mu t + \frac{1}{2}\sigma^2 t^2}$

• Expectation and Variance:

  • Expected Value: $E(X) = \mu$
  • Variance: $Var(X) = \sigma^2$

Solving Normal Distribution Problems

• Standardization Note: Solving normal probability distributions via calculus for every set of $\mu$ and $\sigma$ is tedious. Instead, we typically work with the Standardized Normal Distribution.

• Example Case Study (Mile Run):

  • Scenario: The time for a runner to run a mile is a normal random variable with $\mu = 4\,\text{minutes}, 1\,\text{second}$ (or $241\,\text{seconds}$) and $\sigma = 2\,\text{seconds}$.
  • Probability Calculation Setup:
    • a) Less than 4 minutes: $P(X \le 240) = \int_{-\infty}^{240} f(x) dx$
    • b) More than 3 minutes, 55 seconds: $P(X \ge 235) = \int_{235}^{\infty} f(x) dx$
    • c) Between 2 and 5 minutes: $F(300) - F(120)$

Exponential Distribution

• Definition:

  • A right-skewed distribution with a minimum at $x=0$.
  • The random variable can only take on positive values.
  • It is used to model inter-arrival times or distances for a Poisson process with mean $\lambda > 0$.

• Probability Density Function (pdf):

  • $f(x) = \lambda e^{-\lambda x}$, for $x > 0$

• MGF, Mean, and Variance:

  • $M_x(t) = (1 - \frac{t}{\lambda})^{-1}$ or $\frac{\lambda}{\lambda - t}$
  • $E(X) = \frac{1}{\lambda}$
  • $Var(X) = \frac{1}{\lambda^2}$

• Cumulative Distribution Function (cdf):

  • $F(x) = 1 - e^{-\lambda a}$ (where $a$ is the upper limit value).

• Memory-less Property:

  • The exponential distribution is the only memory-less continuous distribution.
  • The conditional probability of an exponential random variable exceeding the value $s + t$ given it has exceeded $t$ is the same as the variable originally exceeding $s$.
  • Interpretation: The past has no bearing on future behaviors; every instant is like beginning a new random period.

Gamma Distribution

• Definition:

  • A family of right-skewed distributions modeling positive values only.
  • Used for biological and economic characteristics.
  • If $X$ is the waiting time until the $\alpha$-th event occurs in a Poisson process with rate $\lambda$, then $X$ follows a Gamma distribution.

• Probability Density Function (pdf):

  • $f(x; \alpha, \lambda) = \frac{\lambda^{\alpha}}{\Gamma(\alpha)} x^{\alpha - 1} e^{-\lambda x}$, for $x > 0, \alpha > 0, \lambda > 0$

• Real-Life Perspective Example:

  • To achieve success (e.g., having 10 children), one must wait. If achieving the goal of 1 child takes 9 months (rate $\lambda$), achieving 10 children ($\alpha = 10$) over a period (e.g., 0 to 40 years) is modeled via Gamma.

• Properties of the Gamma Function $\Gamma(\alpha)$:

  • $\Gamma(1) = 1$
  • $\Gamma(\alpha) = (\alpha - 1)\Gamma(\alpha - 1)$
  • $\Gamma(n) = (n - 1)!$ for integers
  • $\Gamma(1/2) = \sqrt{\pi}$

• Moments and MGF:

  • $M_x(t) = (1 - \frac{t}{\lambda})^{-\alpha}$
  • $E(X) = \frac{\alpha}{\lambda}$
  • $Var(X) = \frac{\alpha}{\lambda^2}$

Beta Distribution (First Kind)

• Definition:

  • A family of continuous probability distributions defined on a finite interval (usually $[0, 1]$).
  • It models uncertainty of quantities such as proportions and percentages.
  • Parameters: $\alpha$ and $\beta$ (both are shape parameters).

• Probability Density Function (pdf):

  • $f(x) = \frac{1}{B(\alpha, \beta)} x^{\alpha - 1}(1 - x)^{\beta - 1}$ for $0 \le x \le 1$
  • $B(\alpha, \beta) = \frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha + \beta)}$

• Interpretation of $\alpha$ and $\beta$:

  • Think of $\alpha - 1$ as the number of successes and $\beta - 1$ as the number of failures.
  • If success probability is predicted at $90\%$, set $\alpha = 90$ and $\beta = 10$.
  • Increasing $\alpha$ shifts the distribution right; increasing $\beta$ shifts it left.
  • If both increase, the distribution narrows (higher certainty).

• Curve Shapes:

  • U-shaped (asymptotic ends)
  • Bell-shaped (when $\alpha, \beta > 1$)
  • Symmetric (when $\alpha = \beta$)
  • Skewed right (when $\alpha < \beta$) or Skewed left (when $\alpha > \beta$)

• Expectation and Variance:

  • $E(X) = \frac{\alpha}{\alpha + \beta}$
  • $Var(X) = \frac{\alpha\beta}{(\alpha + \beta)^2(\alpha + \beta + 1)}$

Pareto Continuous Probability Distribution

• Definition:

  • Named after Vilfredo Pareto.
  • A power-law distribution used for modeling income, wealth, city populations, and geophysical phenomena.
  • Characterized by a trend where a large portion of wealth is held by a small fraction of the population.

• The Pareto Principle (80-20 Rule):

  • Also called the "Matthew principle."
  • States that $80\%$ of effects come from $20\%$ of causes (e.g., $80\%$ of wealth is held by $20\%$ of the population; $20\%$ of workers create $80\%$ of results).

• Probability Density Function (pdf):

  • $f(x) = \frac{\alpha x_m^{\alpha}}{x^{\alpha + 1}}$ for $x \ge x_m$
  • $x_m$ is the minimum possible value of $X$.
  • $\alpha$ is the shape parameter.

• Cumulative Distribution Function (cdf):

  • $F(x) = 1 - (\frac{x_m}{x})^{\alpha}$

• Moments (Capture using Higher Raw Moments):

  • Finite moments exist if $\alpha > r$.
  • Expected Value (Mean): $E(X) = \frac{\alpha x_m}{\alpha - 1}$ for $\alpha > 1$
  • Variance: $Var(X) = \frac{\alpha x_m^2}{(\alpha - 1)^2(\alpha - 2)}$ for $\alpha > 2$

• Example Calculation:

  • In a population where income follows Pareto with $\alpha = 3$ and $x_m = 1000$.
  • Proportion between $2000$ and $4000$:
  • $P(2000 < X < 4000) = F(4000) - F(2000)$
  • $F(4000) = 1 - (\frac{1000}{4000})^3 = 1 - (1/4)^3 = 1 - 1/64 = 63/64$
  • $F(2000) = 1 - (\frac{1000}{2000})^3 = 1 - (1/2)^3 = 1 - 1/8 = 7/8$
  • $63/64 - 56/64 = 7/64$

Questions & Discussion

• Discussion Question 1: Symmetry is a vital property of the Normal distribution (equality of mean, median, and mode). Proof of the equality of Mean, Median, and Mode on a symmetrical normal distribution function is required.
• Classwork: Differentiate between the Beta density function and the Binomial density function.
• Group Assignment:
  • 1. Derive the density functions of the following sampling distributions: a) Chi-square, b) Student-t distribution, c) F-distribution.
  • 2. Explain the uses of Chi-square distribution, Student-t distribution, and F-distribution in statistics.