Probability and Probability Distributions Study Notes

Fundamentals of Probability

• Probability: A numerical measure of the likelihood that an uncertain event will occur, satisfying $0 \le P(E) \le 1$.
• Types of Probability:
  • Classical Probability: Applied when outcomes in a sample space are equally likely:     $P(E) = \frac{\text{number of outcomes in event } E}{\text{total number of outcomes in sample space}}$
  • Empirical Probability (Relative Frequency): Calculated from observed frequencies:     $P(E) = \frac{\text{frequency of } E}{\text{total frequency}}$
  • Subjective Probability: Derived from educated guesses, personal estimates, or inexact information.
• Key Terminology:
  • Probability Experiment: A chance process leading to well-defined results.
  • Outcome: The result of a single trial.
  • Sample Space: The set of all possible outcomes of a probability experiment.

• Event: A collection of outcomes from the sample space. A simple event consists of a single outcome; a compound event consists of two or more outcomes.
• Mutually Exclusive Events: Events that share no outcomes and cannot occur simultaneously ($P(A \cap B) = 0$).
  • Core Rules of Probability:
• $0 \le P(E) \le 1$ for any event $E$
• $P(E) = 0$ for an impossible event
• $P(E) = 1$ for an event that is certain to occur
• The sum of probabilities for all outcomes in a sample space equals $1$
• Complement Rule: $P(\bar{E}) = 1 - P(E)$
• Monotonicity: If event $A$ implies event $B$ ($A \subseteq B$), then $P(A) \le P(B)$
• Law of Large Numbers: As the number of experiment trials increases, the empirical probability converges to the theoretical classical probability

Probability Distributions

• Probability Distribution: A theoretical function detailing all possible outcomes for a random variable and their corresponding probabilities, summing or integrating to $1$.
• Discrete Probability Distributions: Applicable to countable outcomes.
  • Probability Mass Function (PMF): Defines individual outcome probabilities $P(X = x)$, where $\sum P(x) = 1$.
  • Cumulative Distribution Function (CDF): Defines accumulated probability $P(X \le x)$.
• Continuous Probability Distributions: Applicable to an infinite continuum of values.
  • Probability Density Function (PDF): A continuous curve $f(x)$ where the total area under the curve equals $1$.
  • The probability of a continuous random variable taking an exact single value is zero ($P(X = x) = 0$); probability is evaluated across continuous intervals $P(a \le X \le b)$.

Expected Value and Variance

• Expected Value ($E(X)$ or $\mu$): The long-run average or weighted mean of a random variable.
  • Discrete Case:$E(X) = \sum x_i P(x_i)$
  • Continuous Case:$E(X) = \int x f(x)\,dx$
• Variance ($\text{Var}(X)$ or $\sigma^2$): The expected squared deviation of a random variable from its mean.
  • Discrete Case:$\text{Var}(X) = \sum (x_i - \mu)^2 P(x_i)$
  • Continuous Case:$\text{Var}(X) = \int (x - \mu)^2 f(x)\,dx$
• Standard Deviation ($\text{SD}(X)$ or $\sigma$): The square root of variance ($\sigma = \sqrt{\text{Var}(X)}$).

Binomial Distribution

• Definition: A discrete probability distribution modeling the number of successes in $n$ independent trials.
• Required Conditions:
  • Fixed number of trials $n$
  • Binary trial outcomes (success or failure)
  • Constant probability of success $p$ (and failure $1 - p$)
  • Independent trials
• Probability Mass Function:$P(X = x) = \binom{n}{x} p^x (1-p)^{n-x}$
• Distribution Parameters:
  • Mean / Expected Value: $E(X) = n p$
  • Variance: $\text{Var}(X) = n p (1-p)$
  • Standard Deviation: $\text{SD}(X) = \sqrt{n p (1-p)}$
• Sample Proportion ($\hat{p} = \frac{X}{n}$):
  • $E(\hat{p}) = p$
  • $\text{Var}(\hat{p}) = \frac{p(1-p)}{n}$

Normal Distribution

• Definition: A continuous, bell-shaped theoretical probability distribution that is perfectly symmetric about its mean $\mu$.
• Notation: $X \sim N(\mu, \sigma^2)$

• Empirical Rule (Imperial Rule):
  • Approximately $68\%$ (specifically 68.3\%$) of data falls within 1\sigma$of$\mu$($P(-1 < z < 1) = 0.683).\n - Approximately 95\%$(specifically$95.4\%$) of data falls within $2\sigma$ of $\mu$ ($P(-2 < z < 2) = 0.954$).
  • Approximately $99.7\%$ of data falls within $3\sigma$ of $\mu$ ($P(-3 < z < 3) = 0.997$).
• Standard Normal Distribution ($Z$):
  • Standardized normal distribution denoted as $Z \sim N(0, 1)$, with mean $\mu = 0$ and standard deviation $\sigma = 1$.
  • Standardization Formula (z-score):$Z = \frac{X - \mu}{\sigma}$
  • Cumulative Standard Normal Function: $\Phi(z) = P(Z < z)$.
  • Symmetry Property: $\Phi(-z) = 1 - \Phi(z)$.