STAT 112: Introduction to Probability Distributions 2 - Cumulative Distribution Functions and Moments

Cumulative Distribution Function Definition and Properties

The cumulative distribution function, frequently referred to as the distribution function and abbreviated as CDF, represents the probability that a random variable $X$ takes on a value less than or equal to a specific real number $x$. Mathematically, it is defined as $F(x) = P(X \le x)$, where $x$ is any real number within the range $(-\infty < x < \infty)$.

The distribution function $F(x)$ is characterized by several fundamental properties that dictate its behavior across the real number line. First, it is a nondecreasing function, meaning that for any two values where $x \le y$, the corresponding function values must satisfy $F(x) \le F(y)$. Second, the limits of the function at the extremes are fixed: the limit as $x$ approaches negative infinity is zero ($\lim_{x \to -\infty} F(x) = 0$), and the limit as $x$ approaches positive infinity is one ($\lim_{x \to \infty} F(x) = 1$). Finally, $F(x)$ is continuous from the right. This is expressed as $F(x + h)_{h \to 0^+} = F(x)$ for all values of $x$.

Distribution Functions for Discrete Random Variables

For a discrete random variable $X$, the cumulative distribution function is obtained by summing the probabilities of all outcomes up to and including the value $x$. This is expressed as $F(x) = P(X \le x) = \sum_{u \le x} P(X = u) = \sum_{u \le x} p(u)$, where the summation encompasses all values $u$ that $X$ can take such that $u \le x$. If the random variable $X$ assumes a finite set of values $x_1, x_2, \dots, x_n$, the CDF follows a piecewise defined structure:

$F(x) = \begin{cases} 0, & -\infty < x < x_1 \\ p(x_1), & x_1 \le x < x_2 \\ p(x_1) + p(x_2), & x_2 \le x < x_3 \\ p(x_1) + p(x_2) + p(x_3), & x_3 \le x < x_4 \\ \dots, & \dots \\ p(x_1) + p(x_2) + p(x_3) + \dots + p(x_n) = 1, & x_n \le x < \infty \end{cases}$

Characteristics and Examples of Discrete Distribution Functions

Example 1 illustrates these principles through a random variable $X$ representing the number of tails observed when tossing two fair coins. The probability mass function (PMF) is given as $P(X = 0) = 0.25$, $P(X = 1) = 0.5$, and $P(X = 2) = 0.25$. The resulting distribution function is defined as:

$F(x) = \begin{cases} 0, & -\infty < x < 0 \\ 0.25, & 0 \le x < 1 \\ 0.75, & 1 \le x < 2 \\ 1, & 2 \le x < \infty \end{cases}$

Observations regarding discrete CDFs reveal specific attributes. The graph of such a function is known as a staircase or step function. The magnitude of the "jumps" occurring at specific points (e.g., at $0, 1, 2$) corresponds exactly to the probabilities of those outcomes ($0.25, 0.5, 0.25$ respectively). This allows one to derive the probability function directly from the distribution function. Furthermore, the function is monotonically increasing, moving from left to right while taking values between $0$ and $1$. The right-continuity is observed as the value of the function at an integer point is taken from the higher step; for instance, the value at $1$ is $0.75$ rather than $0.25$.

Continuous Distribution Functions and Probability Density

A nondiscrete random variable $X$ is considered absolutely continuous, or simply continuous, if its distribution function can be represented as the integral of a probability density function (pdf) $f(t)$. Specifically, $F(x) = P(X \le x) = \int_{-\infty}^x f(t)dt$. For a function to serve as a valid density function, it must satisfy two conditions: $f(x) \ge 0$ for all values of $x$, and the total area under the curve must equal one, calculated as $\int_{-\infty}^{\infty} f(x)dx = 1$.

In the continuous case, the probability that $X$ takes on any specific point value is zero. However, the probability that the random variable falls within a given interval between $a$ and $b$ is defined by the definite integral of the density function over that interval: $P(a < X < b) = \int_a^b f(x)dx = F(b) - F(a)$.

Mathematical Operations and Relationship between PDF and CDF

The fundamental relationship between the probability density function (pdf) and the cumulative distribution function (cdf) is rooted in calculus. The CDF is found by integrating the pdf: $F(x) = \int_{-\infty}^x f(t)dt$. Conversely, by the Fundamental Theorem of Calculus, the pdf is the derivative of the cdf: $f(x) = \frac{d}{dx} [F(x)]$.

Example 2 demonstrates the calculation of constants and probabilities. Given $f(x) = cx^2$ for $0 < x < 3$ and $0$ otherwise, the constant $c$ is found by solving $\int_0^3 cx^2dx = 1$. Integrating yields $[\frac{cx^3}{3}]_0^3 = 9c = 1$, resulting in $c = \frac{1}{9}$. Using this density function, the probability $P(1 < X < 2)$ is calculated as $\int_1^2 \frac{x^2}{9}dx = [\frac{x^3}{27}]_1^2 = \frac{8}{27} - \frac{1}{27} = \frac{7}{27}$.

Another example involves a person waiting for an elevator for a maximum of $2$ minutes, with a pdf defined piecewise: $f(x) = x$ for $0 \le x < 1$, and $f(x) = 2 - x$ for $1 \le x < 2$. The resulting CDF is derived by integrating over each interval:

$F(x) = \begin{cases} 0, & x < 0 \\ \frac{x^2}{2}, & 0 \le x < 1 \\ 2x - \frac{x^2}{2} - 1, & 1 \le x < 2 \\ 1, & x \ge 2 \end{cases}$

Probability Computation Scenarios

Computation of constants in probability mass functions (pmf) is essential. For instance, given $P(X = x) = \frac{c\lambda^x}{x!}$ for $x = 0, 1, 2, \dots$, the constant $c$ is determined by the requirement that the sum of probabilities equals one. Since $\sum_{x = 0}^{\infty} \frac{\lambda^x}{x!} = e^{\lambda}$, it follows that $ce^{\lambda} = 1$, hence $c = e^{-\lambda}$. In this distribution, $P(X = 0) = e^{-\lambda}$ and $P(X \ge 2) = 1 - P(X \le 1) = 1 - [p(0) + p(1)] = 1 - e^{-\lambda} - \lambda e^{-\lambda}$.

In a real-world application involving electronic components, the failure time (in hours) follows the density function $f(x) = \frac{1}{500} e^{-x/500}$ for $x > 0$. To find the probability a component lasts more than $2000$ hours, we calculate $P(X > 2000) = \int_{2000}^{\infty} \frac{1}{500} e^{-x/500}dx = [-e^{-x/500}]_{2000}^{\infty} = e^{-4}$. The probability of failure between $500$ and $2000$ hours is $P(500 < X < 2000) = [-e^{-x/500}]_{500}^{2000} = e^{-1} - e^{-4}$. The probability of failure before $500$ hours is $P(X < 500) = [-e^{-x/500}]_0^{500} = 1 - e^{-1}$.

Mathematical Expectation

Mathematical expectation, also known as the expected value or mean, is a central concept representing the average value of a random variable over many repetitions. For a discrete random variable $X$ with values $x_1, \dots, x_n$, it is defined as $E(X) = \sum_{i=1}^n x_i P(X = x_i)$. In the specific case where all probabilities are equal to $\frac{1}{n}$, the expectation simplifies to the arithmetic mean: $E(X) = \frac{x_1 + x_2 + \dots + x_n}{n}$. If the variable has infinite values, the series must converge absolutely for the expectation to exist.

For continuous random variables, the expectation is defined via integration as $E(X) = \int_{-\infty}^{\infty} xf(x)dx$, provided absolute convergence is met. The expectation is denoted by $\mu_x$ or $\mu$ and acts as a measure of central tendency.

Example 3 describes a die game where a player wins $\text{GHs } 20$ for a $2$, $\text{GHs } 40$ for a $4$, and loses $\text{GHs } 30$ for a $6$, with no gain/loss for other faces. The expected sum is calculated as $E(X) = 0(\frac{1}{6}) + 20(\frac{1}{6}) + 0(\frac{1}{6}) + 40(\frac{1}{6}) + 0(\frac{1}{6}) - 30(\frac{1}{6}) = 5$. For a continuous density $f(x) = \frac{1}{2}x$ on $0 < x < 2$, the expectation is $E(X) = \int_0^2 x(\frac{1}{2}x)dx = \int_0^2 \frac{x^2}{2}dx = [\frac{x^3}{6}]_0^2 = \frac{4}{3}$.

Properties of Expectation

Mathematical expectation follows several algebraic rules. If $c$ is a constant, then $E(cX) = cE(X)$. If $b$ is also a constant, then $E(cX + b) = cE(X) + b$. Linearity holds for any two random variables, such that $E(X + Y) = E(X) + E(Y)$. Furthermore, if $X$ and $Y$ are independent random variables, the expectation of their product equals the product of their expectations: $E(XY) = E(X)E(Y)$.

Variance and Standard Deviation

Variance measures the dispersion or "scatter" of random variable values around the mean $\mu$. It is defined as a non-negative number $Var(X) = E[(X - \mu)^2]$. The standard deviation, denoted by $\sigma_x$ or $\sigma$, is the positive square root of the variance: $\sigma = \sqrt{Var(X)}$. If values cluster near the mean, the variance is small; if they are widely distributed, the variance is large.

For a discrete variable, $Var(X) = \sum_{i=1}^n (x_i - \mu)^2 p(x_i)$. If the probabilities are equal, this becomes the standard variance formula for a set of numbers. For continuous variables with density $f(x)$, variance is calculated as $Var(X) = \int_{-\infty}^{\infty} (x - \mu)^2 f(x)dx$, provided the integral converges.

In Example 5, using the density $f(x) = \frac{1}{2}x$ for $0 < x < 2$ with a previously calculated mean $\mu = \frac{4}{3}$, the variance is computed: $Var(X) = \int_0^2 (x - \frac{4}{3})^2 (\frac{1}{2}x)dx = \int_0^2 (x^2 - \frac{8}{3}x + \frac{16}{9})(\frac{1}{2}x)dx = [\frac{x^4}{8} - \frac{4x^3}{9} + \frac{4x^2}{9}]_0^2 = 2 - \frac{32}{9} + \frac{16}{9} = \frac{2}{9}$. The standard deviation is $\sigma = \frac{\sqrt{2}}{3}$.

Theorems of Variance

There are four key theorems governing variance operations. Theorem 1 provides an alternative calculation method: $Var(X) = \sigma^2 = E(X^2) - \mu^2$. Theorem 2 states that for constants $c$ and $b$, $Var(cX + b) = c^2 Var(X)$. Theorem 3 defines properties for independent variables: $Var(X + Y) = Var(X) + Var(Y)$ and $Var(X - Y) = Var(X) + Var(Y)$. Theorem 4 addresses non-independent variables by incorporating covariance: $Var(X \pm Y) = Var(X) + Var(Y) \pm 2Cov(X, Y)$.

Solved Practical Applications

In a lottery scenario with $10,000$ tickets, prizes include $200$ at $\$5$, $20$ at $\$25$, and $5$ at $\$100$. The probabilities for various winnings are $P(X = 5) = 0.02$, $P(X = 25) = 0.002$, $P(X = 100) = 0.0005$, and $P(X = 0) = 0.9775$. The expected value is $E(X) = 5(0.02) + 25(0.002) + 100(0.0005) + 0 = 0.2$. Thus, the fair price for a ticket is $20 \text{ cents}$.

When tossing a pair of fair dice to find the expectation of the sum of points, let $X$ and $Y$ represent the points on each die. Since $E(X) = E(Y) = 1(\frac{1}{6}) + 2(\frac{1}{6}) + \dots + 6(\frac{1}{6}) = \frac{7}{2}$, the total expectation is $E(X + Y) = \frac{7}{2} + \frac{7}{2} = 7$.

Finally, for a continuous variable with $f(x) = 2e^{-2x}$ on $0 < x < \infty$, we use the special gamma function formula $\int_0^{\infty} x^{r-1} e^{-\beta x} dx = \frac{(r-1)!}{\beta^r}$ to compute moments. Here, $E(X) = 2 \int_0^{\infty} x e^{-2x} dx = 2(\frac{1}{4}) = \frac{1}{2}$. Similarly, $E(X^2) = 2 \int_0^{\infty} x^2 e^{-2x} dx = 2(\frac{2!}{2^3}) = \frac{1}{2}$. The variance is then $Var(X) = E(X^2) - [E(X)]^2 = \frac{1}{2} - \frac{1}{4} = \frac{1}{4}$.