Random Variables and Probability Distributions

Vocabulary flashcards covering discrete and continuous random variables, common distributions (Binomial, Poisson, Uniform, Exponential, Normal), expectation, variance, standard normal standardization, percentiles, joint distributions, independence, covariance, and correlation.

Random variable

A variable whose possible values depend on the outcome of a given random experiment. It is called discrete if it can take on a finite or countable set of values, and continuous if it can take on an uncountable set of values.

Distribution function

The cumulative function of a random variable $X$, defined as $F_X(x) = P\{X \le x\}$ for all $x \in \mathbb{R}$.

Probability mass function

For a discrete random variable $X$, the function defined as $p_X(x) = P\{X = x\}$ for all $x \in \mathbb{R}$. Also referred to as the probability function or discrete density.

Expected value of a discrete random variable

For a discrete random variable $X$ that can take $n$ values $x_1, \dots, x_n$ with probability function $p_X(\cdot)$, it is defined as $\mathbb{E}[X] = \sum_{i=1}^n x_i p_X(x_i)$.

Linearity of expected value

The property stating that $\mathbb{E}[aX + b] = a\mathbb{E}[X] + b$ for all $a, b \in \mathbb{R}$, and $\mathbb{E}\left[\sum_{i=1}^n X_i\right] = \sum_{i=1}^n \mathbb{E}[X_i]$ for random variables $X_1, X_2, \dots, X_n$.

Variance

A measure of dispersion for a random variable $X$ with expected value $\mathbb{E}[X] = \mu$, defined as $\text{Var}(X) = \mathbb{E}[(X - \mu)^2] = \mathbb{E}[X^2] - \mu^2$.

Standard deviation

The square root of the variance of a random variable $X$, denoted and defined as $\sigma(X) = \sqrt{\text{Var}(X)}$.

Variance of a linear transformation

The proposition stating that for all $a, b \in \mathbb{R}$ and for any random variable $X$, $\text{Var}(aX + b) = a^2 \text{Var}(X)$.

Binomial random variable

A random variable $X \sim \text{Bin}(n, p)$ representing the total number of successes in $n \in \mathbb{N}$ independent experiments with success probability $p \in (0, 1)$. Its PMF is $p_X(k) = \binom{n}{k}p^k(1-p)^{n-k}$ for $k \in \{0, 1, \dots, n\}$, with $\mathbb{E}[X] = np$ and $\text{Var}(X) = np(1-p)$.

Poisson random variable

A random variable $X \sim \text{Pois}(\lambda)$ representing the number of times an event occurs in a time interval $\Delta t$ with mean rate $\lambda$. Its PMF is $p_X(k) = \frac{e^{-\lambda}\lambda^k}{k!}$ for $k \in \mathbb{N}$, with $\mathbb{E}[X] = \text{Var}(X) = \lambda$.

Probability density function

For a continuous random variable $X$, a function $f_X : (-\infty, +\infty) \rightarrow \mathbb{R}_{\ge 0}$ such that $P\{X \in B\} = \int_B f_X(x)\mathrm{d}x$ for all $B \subseteq \mathbb{R}$.

Expected value of a continuous random variable

For a continuous random variable $X$ with probability density function $f_X$, defined as $\mathbb{E}[X] = \int_{-\infty}^{+\infty} x f_X(x)\mathrm{d}x$.

Uniform distribution

A continuous random variable $X \sim \mathcal{U}(a, b)$ over interval $[a, b]$ with density $f_X(x) = \frac{1}{b-a}$ for $x \in [a, b]$ and $0$ otherwise. It has $\mathbb{E}[X] = \frac{a+b}{2}$, $\text{Var}(X) = \frac{(b-a)^2}{12}$, and $F_X(x) = \frac{x-a}{b-a}$.

Exponential distribution

A continuous random variable $X \sim \text{Exp}(\lambda)$ with density $f_X(x) = \lambda e^{-\lambda x}$ for $x \ge 0$ and $0$ for $x < 0$. It has $\mathbb{E}[X] = \frac{1}{\lambda}$, $\text{Var}(X) = \frac{1}{\lambda^2}$, and $F_X(x) = 1 - e^{-\lambda x}$.

Normal random variable

A continuous random variable $X \sim \mathcal{N}(\mu, \sigma^2)$ with density function $f_X(x) = \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$ for $x \in \mathbb{R}$, with expected value $\mathbb{E}[X] = \mu$ and variance $\text{Var}(X) = \sigma^2$.

Standard normal random variable

The normal random variable $Z = \frac{X - \mu}{\sigma} \sim \mathcal{N}(0, 1)$ obtained through standardization of $X \sim \mathcal{N}(\mu, \sigma^2)$. Its CDF is denoted $\Phi(x)$ and satisfies $\Phi(-x) = 1 - \Phi(x)$.

Percentile of a normal distribution

For $X \sim \mathcal{N}(\mu, \sigma^2)$ and $\alpha \in (0, 1)$, the quantity $x_\alpha$ satisfying $P\{X > x_\alpha\} = \alpha$, called the $(1 - \alpha) \cdot 100^{\text{th}}$ percentile of $X$.

Joint cumulative distribution function

For two random variables $X$ and $Y$, the function $F : \mathbb{R} \times \mathbb{R} \rightarrow (0, 1)$ defined by $F(x, y) = P\{X \le x, Y \le y\}$.

Joint probability mass function

For two discrete random variables $X$ and $Y$, defined as $p(x, y) = P\{X = x, Y = y\}$. The marginal distribution for $X$ is $p_X(x) = \sum_y p(x, y)$.

Joint probability density function

For two continuous random variables $X$ and $Y$, a function $f : \mathbb{R} \times \mathbb{R} \rightarrow \mathbb{R}_{\ge 0}$ such that $P\{X \in A, Y \in B\} = \int_A \int_B f(x, y)\mathrm{d}x\mathrm{d}y$. The marginal density for $X$ is $f_X(x) = \int_{-\infty}^{+\infty} f(x, y)\mathrm{d}y$.

Independent random variables

Two random variables $X$ and $Y$ such that $F(x, y) = F_X(x)F_Y(y)$ for all $x, y \in \mathbb{R}$. In the discrete case $p(x, y) = p_X(x)p_Y(y)$, and in the continuous case $f(x, y) = f_X(x)f_Y(y)$.

Covariance

A quantity measuring the joint variability of two random variables $X$ and $Y$, defined as $\text{Cov}(X, Y) = \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])] = \mathbb{E}[XY] - \mathbb{E}[X]\mathbb{E}[Y]$.

Correlation

A measure of the linear relationship between random variables $X$ and $Y$, defined as $\text{Corr}(X, Y) = \frac{\text{Cov}(X, Y)}{\sqrt{\text{Var}(X)\text{Var}(Y)}} \in (-1, 1)$.