Exponential and Standard Normal Distributions with Moment Generating Functions

Exponential Distribution Review

Probability Density Function (PDF) and Cumulative Distribution Function (CDF)

• PDF for Exponential Distribution: If a random variable $X$ is exponentially distributed with rate parameter $\lambda$, its probability density function is given by:$f(x) = \lambda e^{-\lambda x}, \text{ for } x \ge 0$

• Integration and CDF: To find the cumulative distribution function (CDF), we integrate the PDF. For example, computing the integral results in the constant being divided by $-\lambda$. The integral of the PDF from $0$ to $x$ (or $a$ to $b$) is used to find probabilities.

  • $F(x) = P(X \le x) = \int<em>0^x \lambda e^{-\lambda t} dt = [-e^{-\lambda t}]</em>0^x = -e^{-\lambda x} - (-e^0) = 1 - e^{-\lambda x}$

Tail Probability

• The probability of $X$ being greater than a certain value $x$ is called the tail probability:P(X > x) = 1 - P(X \le x) = 1 - (1 - e^{-\lambda x}) = e^{-\lambda x}

  • Visual Representation: This function starts at $\lambda$ (at $x=0$) and then decays exponentially as $x$ increases.

• Application in Hypothesis Testing: Tail probabilities are crucial in hypothesis testing. For instance, when testing a new drug, the null hypothesis ($H_0$) posits that the drug has no effect. Researchers then ask: "What is the probability of observing a test statistic at least as extreme as the one collected, given that the null hypothesis is true?" If this tail probability (often called the p-value) is very small, it suggests that the observed results are unlikely under the null hypothesis, leading to its rejection. This allows for the conclusion that the drug likely has an effect and can potentially be approved for wider use.

Real-World Relevance: Poisson Process Connection

• The exponential distribution is closely related to the Poisson process. While a Poisson process models the number of events in a fixed interval, the exponential distribution models the waiting time between successive events in a Poisson process.

  • Example: A uniform distribution might describe the exact time of a call, whereas an exponential distribution describes how long one has to wait for a call.

Example: Waiting Time for Calls

• Scenario: A call center receives an average of $\lambda = 10$ calls per hour.

• Problem: Let $X$ be the waiting time between successive calls in minutes.

• Rescaling: If the rate is $10$ calls/hour, then in minutes, the rate is $\frac{10}{60} = \frac{1}{6}$ calls/minute. So, the new $\lambda = 1/6$.

• Question: What is the probability that one has to wait for more than $20$ minutes for a call?

  • P(X > 20) = e^{-(1/6) \times 20} = e^{-20/6} = e^{-10/3}

Moment Generating Functions (MGF)

• Definition: For a random variable $X$, the moment generating function (MGF) is defined as the expected value of $e^{tX}$. It is a transform that provides a powerful tool for analyzing distributions.

  • $M<em>X(t) = E[e^{tX}] = \int</em>{-\infty}^{\infty} e^{tx} f(x) dx$

• Purpose: The primary use of MGFs is to easily compute the mean (first moment) and variance (related to the second moment) of a random variable, especially for distributions where direct integration is cumbersome.

• Connection to Laplace Transforms: The MGF is essentially a type of Laplace transform. Laplace transforms are fundamental in engineering and signal processing for converting time-domain information (which can be very complex) into amplitude and frequency information (which is often more tractable).

• Historical Context: This mathematical framework originates from giants like Laplace and Fourier, who studied phenomena like heat propagation (Fourier's work on the heat equation) and wave dynamics, with applications extending to modern fields like signal processing and camera rework.

MGF for Exponential Distribution

• For an exponential distribution with rate $\lambda$, the MGF is derived as:

  • $M_X(t) = \frac{\lambda}{\lambda - t}$

• **Computing Mean ($E[X]$):

  • The first derivative of the MGF evaluated at $t=0$ gives the mean.

  • $M_X'(t) = \frac{\lambda}{(\lambda - t)^2}$

  • $E[X] = M_X'(0) = \frac{\lambda}{(\lambda - 0)^2} = \frac{\lambda}{\lambda^2} = \frac{1}{\lambda}$

• **Computing Variance ($Var(X)]$):

  • The second derivative of the MGF evaluated at $t=0$ gives $E[X^2]$.

  • $M_X''(t) = \frac{d}{dt} \left( \lambda (\lambda - t)^{-2} \right) = \lambda (-2)(\lambda - t)^{-3}(-1) = \frac{2\lambda}{(\lambda - t)^3}$

  • $E[X^2] = M_X''(0) = \frac{2\lambda}{(\lambda - 0)^3} = \frac{2\lambda}{\lambda^3} = \frac{2}{\lambda^2}$

  • $Var(X) = E[X^2] - (E[X])^2 = \frac{2}{\lambda^2} - \left(\frac{1}{\lambda}\right)^2 = \frac{2}{\lambda^2} - \frac{1}{\lambda^2} = \frac{1}{\lambda^2}$

Standard Normal Distribution

• Definition: The standard normal distribution is a specific normal distribution with a mean of $\mu = 0$ and a standard deviation of $\sigma = 1$ (and thus a variance of $\sigma^2 = 1$).

• Notation: It is typically denoted by $Z$ (for Z-score).

• PDF: The probability density function for the standard normal distribution is:

  • $f(z) = \frac{1}{\sqrt{2\pi}} e^{-z^2/2}$

• Bell Curve: The graph of the standard normal distribution is the familiar bell curve, symmetrical around its mean of $0$. The inflection points of the curve are at $\pm \sigma$ (i.e., $\pm 1$ for the standard normal).

• Normalization Constant: The integral of the PDF over all real numbers (from $-\infty$ to $\infty$) must equal $1$ for it to be a valid density function. Proving this (that $\int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}} e^{-z^2/2} dz = 1$) requires multivariable calculus (Calc III) and is not trivial.

• Difficulty of Direct Calculation: Computing the mean and variance of the standard normal distribution using direct integration involves complex integrals that often require techniques like u-substitution and integration by parts repeatedly, making it a