Binomial Distributions, Moment Generating Functions, and Statistical Moments

Objective: To derive the coefficient $A_k$ in the Probability Mass Function (PMF) of the Binomial Distribution by leveraging the property that the sum of all probabilities must equal one.
Starting Formula (PMF): $\sum_{k=0}^{n} A_k p^k (1-p)^{n-k} = 1$
Algebraic Refinement:
- Separate the constant terms not depending on $k$ : move $(1-p)^{n}$ to the right-hand side.
- Introduce a substitution variable $t = \frac{p}{1-p}$ .
- The expression simplifies to: $\sum_{k=0}^{n} A_k t^k = (1+t)^n$
- Note: $1+t = 1 + \frac{p}{1-p} = \frac{1-p+p}{1-p} = \frac{1}{1-p}$ .
The Derivative Technique:
- To find specific coefficients of a polynomial, one can take successive derivatives and evaluate at zero.
- For the left-hand side, taking the $k$ -th derivative with respect to $t$ and setting $t=0$ results in $k! A_k$ . All lower-order terms vanish due to the derivative, and all higher-order terms vanish because they still contain factors of $t$ .
- For the right-hand side, taking the $k$ -th derivative of $(1+t)^n$ yields:
  1. First derivative: $n(1+t)^{n-1}$
  2. Second derivative: $n(n-1)(1+t)^{n-2}$
  3. $k$ -th derivative: $n(n-1)...(n-k+1)(1+t)^{n-k}$
- Setting $t=0$ gives the product $n(n-1)...(n-k+1)$ , which is equivalent to $\frac{n!}{(n-k)!}$ .
Final Result: $k! A_k = \frac{n!}{(n-k)!}$ $A_k = \frac{n!}{k!(n-k)!} = \binom{n}{k}$

Definition: The Moment Generating Function (MGF) is defined as the expectation of an exponential function of the random variable: $M_X(t) = E[e^{tX}]$
Utility: The MGF is a highly convenient tool for finding the moments of a distribution, such as the mean ( $E[X]$ ) and second moment ( $E[X^2]$ ).
Generating Moments:
- To find the $k$ -th moment, take the $k$ -th derivative of the MGF with respect to $t$ , then evaluate at $t=0$ .
- Derivative logic: $\frac{d^k}{dt^k} E[e^{tX}] = E[X^k e^{tX}]$ . Substituting $t=0$ makes the exponential term $1$ , leaving only $E[X^k]$ .
Statistical Summaries:
- The First Moment ( $k=1$ ) is the mean ( $E[X]$ ).
- The Second Moment ( $k=2$ ) is $E[X^2]$ .
- The Third Moment relates to Kurtosis (the shape and "tailedness" of the distribution).
Characteristic Functions:
- Defined as $\Phi_X(t) = E[e^{itX}]$ .
- Unlike MGFs, which only exist if the expectation is finite in an open interval around zero, the characteristic function always exists.
- The characteristic function is instrumental in proving the Central Limit Theorem.

Bernoulli MGF:
- For a Bernoulli random variable $X$ where $P(X=1) = p$ and $P(X=0) = 1-p$ .
- Applying the Law of the Unconscious Statistician (LOTUS): $E[e^{tX}] = e^{t(0)}(1-p) + e^{t(1)}(p) = 1-p + p e^t$
Binomial MGF (Sum of IID Bernoullis):
- Let $Y$ be the sum of $n$ Independent and Identically Distributed (IID) Bernoulli random variables ( $Y = \sum_{i=1}^{n} X_i$ ).
- Because the variables are independent, the MGF of the sum is the product of the individual MGFs: $E[e^{tY}] = E[e^{t(X_1 + X_2 + ... + X_n)}] = E[e^{tX_1} e^{tX_2} ... e^{tX_n}]$ $E[e^{tY}] = E[e^{tX_1}] E[e^{tX_2}] ... E[e^{tX_n}]$
- Since they are identically distributed, this becomes the individual MGF raised to the power of $n$ : $M_Y(t) = (1-p + p e^t)^n$
Calculating the Mean of a Binomial using MGF:
- Take the first derivative of the MGF: $\frac{d}{dt} (1-p + p e^t)^n = n(1-p + p e^t)^{n-1} \times (p e^t)$ .
- Evaluate at $t=0$ : $n(1-p+p)^{n-1} \times p(1) = n(1)p = np$ .
Calculating the Variance of a Binomial using MGF:
- Variance is found via $Var(X) = E[X^2] - (E[X])^2$ .
- Find $E[X^2]$ by taking the second derivative of the MGF and substituting $t=0$ .

Student Confusion: The TA noted that students are struggling during discussion sessions, indicating they may not be fully absorbing the lecture material.
Quiz Warnings: While the first quiz was easy, the professor warned that subsequent quizzes will be significantly more difficult.
Why t=0?: A student asked why $t$ is substituted with zero. The professor explained it is to eliminate the function of $t$ and isolate the constant factor/coefficient (the moment) we are interested in.
LOTUS Explanation: A student asked for clarification on the Bernoulli sum derivation. The professor explained that because of IID conditions, the expectation of a product of independent variables equals the product of their expectations: $E[XY] = E[X]E[Y]$
Visualization Concerns: There were brief technical issues with the screen mirroring being dark or not moving, but the professor confirmed it was working for the majority of the class.