Random Variables

Definition: A random variable is a variable whose value is a numerical outcome of a random phenomenon.

Discrete Random Variables

Examples:
- The outcome of throwing a die.
- The number of heads when tossing 'n' coins.
- The number of customers arriving at a bookstore.
- The position of participants in a running race.

Continuous Random Variables

Examples:
- Heights of randomly selected persons (age > 16).
- Weights of randomly selected persons (age > 16).
- The time one has to wait in a queue.
- The service time in a public service system.
- The lifetime distribution of electric bulbs.

Probability Distribution

Discrete Example: A random variable X with probability distribution:
- X: -2, -1, 0, 1, 2, 3
- f(X): 0.1, K, 0.2, 2K, 0.3, 3K
Continuous Example: X is a continuous random variable with PDF:
f(x) = \begin{cases} kx, & 0 \leq x \leq 2 \ 2k, & 2 \leq x \leq 4 \ 6k - kx, & 4 \leq x \leq 6 \ 0, & \text{elsewhere} \end{cases}

Probability Mass Function (PMF)

Sketching: If the probability distribution of X is given:
- X: 1, 2, 3, 4
- f(X): 0.4, 0.3, 0.2, 0.1

Independence

An assembly consists of three mechanical components with meeting specifications probabilities: 0.95, 0.98, and 0.99 respectively. Assuming independence, determine the probability mass function of the number of components meeting specifications.

Probability Density Function (PDF)

Let X denote the current in a thin copper wire in milliamperes, with a range of [0, 20mA] and a probability density function f(x) = 0.05 for 0 \leq x \leq 20.
- Sketch the PDF
- Probability that current is less than 10 amperes.
- Probability that current is greater than 5 amperes.
- Probability that current is between 10 and 15 amperes.
- P(X=12) = 0 (for continuous RVs)

Important Note on Continuous Random Variables

For continuous random variables, P(X = x) = 0 for every x.
The probability is obtained by integrating the PDF.
A PDF can be greater than 1.
A PDF can be unbounded.
- Example: If f(x) = (2/3)x^{-1/3} for 0 < x < 1, then \int f(x) dx = 1 even though f is not bounded.

Cumulative Distribution Function (CDF)

Definition: Cumulative Distribution Function (CDF) for discrete and continuous random variables.

PMF and CMF

If the random variable X takes values 1, 2, 3, and 4, such that 2P(X=1) = 3P(X=2) = P(X=3) = 5P(X=4), find the PMF and cumulative distribution function (CMF) for X. Also, sketch the PMF and CMF of X.

CDF for Continuous Random Variables

If f_X(x) is the PDF of continuous RV X, then CDF of X is defined.
Examples:
1. f(x) = 0.05, 0 \leq x \leq 20
2. f(x) = \begin{cases} \frac{2}{9}x, & 0 \leq x \leq 3 \ 0, & \text{elsewhere} \end{cases}

Properties of CDF for Continuous Random Variables

Let F be the CDF for a random variable X. Then:
- P(x < X \leq y) = F(y) - F(x)
- P(X > x) = 1 - F(x)
- P(a < X \leq b) = P(a \leq X < b) = F(b) - F(a)
- P(a < x <b) = P(a \leq X \leq b)

Cumulative Density Function - Problems

Given CDF for a random variable X, F(x) = \begin{cases} 0, & x < 0 \ 1 - e^{-x}, & x \geq 0 \end{cases} , find:
- The probability mass function
- P(1<x<3)
- P(1 \le x<5)
- P(2 < x \le 5)
- P(3 \le x \le 5)
- P(x \le 5)
- P(x \ge 3)
Given the PDF of X, f(x) = \begin{cases} \frac{2}{9}x, & 0 \leq x \leq 3 \ 0, & \text{elsewhere} \end{cases} , find the cumulative density function (CDF) for X and evaluate the following probabilities using CDF.
- P(x \ge 2)
- P(x \le 1.5)
- P(0.5 < x \le 2.5)
- P(1 \le x \le 3)
- P(x \le 5)

Mean and Variance of Discrete and Continuous Random Variables

Definition

The expected value, or mean, or first moment of X is defined to be
E(X) = \begin{cases} \sum x f(x) & \text{if X is discrete} \ \int x f(x) dx & \text{if X is continuous} \end{cases}
If X is a random variable with pmf/pdf f(x) and if g(x) is a function of X. Then, E[g(x)] = \int g(x) f(x) dx

Discrete Random Variables

Find the mean and variance of the number of heads obtained while tossing three coins.
Let X equal the number of bits in error in the next four bits transmitted. The possible values of X are {0, 1, 2, 3, 4} and corresponding probabilities are P(X=0)=0.6561; P(X=1)=0.2916; P(X=2)=0.0486; P(X=3)=0.0036 and P(X=4)=0.0001. Find the mean, variance, and standard deviation of X.

Continuous Random Variables

Find the mean and variance of X if the probability density function of X is f(x) = 0.125x, 0 < x < 4.
Given the pdf of X is f(x) = ax^2, 0 \leq x \leq 1.
- Find ‘a’.
- Find the value of ‘k’ if P(X \leq k) = P(X > k).
- Find the mean and standard deviation of X.

Rules of Expectation as an Operator

Let X and Y be two Random variables
- E(cX) = cE(X), c \in R
- E(X + Y) = E(X) + E(Y)
- E(c) = c, c \in R
- Example: E(5) = 5
If X is a random variable with pmf/pdf and if g(x) is a function of X. Then, E[g(x)] = \int g(x) f(x) dx

Central Moments and Row Moments

nth central moment: \mu_n = E[(X - E(X))^n]
nth row moment: m_n = E[X^n]
Central moments are used.
- m_1 = \mu = \text{Average(Mean) of X}
- \mu1 = E[(X - E(X))^1] = E(X) - E(m1) = m1 - m1 = 0
- Skewness = \frac{\mu3}{ (\mu2)^{3/2} }
  - If skewness > 0 ⇒ skewed towards the left.
- \mu_4 = E[(X - E(X))^4]
- Kurtosis = \frac{\mu4}{(\mu2)^2}
  - If kurtosis = 3 ⇒ Normal(\mu=0, \sigma=1)
  - Logistic(\alpha=0, \beta=0.55153) kurtosis = 4.2

Uniform Row Moments

Uniform ∼ (0,1)
- E(X) = \int_0^1 x f(x) dx = \frac{1}{2}
  - Generate (Draw) N Random values xi, i=1:N from X. Find average \frac{1}{N} \sum{i=1}^N x_i. This will tend to \frac{1}{2} when N \rightarrow \infty.
- E(X^2) = \int_0^1 x^2 f(x) dx = \frac{1}{3}
  - Generate(Draw) N Random values xi, i=1:N from X. Compute xi^2 \forall i. Find average: \frac{1}{N} \sum{i=1}^N xi^2 . This will tend to \frac{1}{3} when N \rightarrow \infty.

Variance - Second Central Moment

\mu2 = E[(X - m1)^2] = E(X^2 - 2m1X + m1^2); m_1 = E(X)
- = E(X^2) - E(2m1X) + E(m1^2)
- = E(X^2) - 2m1E(X) + m1^2
- = E(X^2) - 2m1^2 + m1^2
- = E(X^2) - m_1^2
- = E(X^2) - (E(X))^2
Var(X) = E[(X - m_1)^2] = E(X^2) - (E(X))^2

Variance of Z = X + Y

Var(X) = E[(X - \mux)^2] = E(X^2) - (E(X))^2; \mux = E(X)
Let X and Y be two RVs, Let Z = X + Y; Var(Z) = ?.
E(Z) = E(X + Y) = E(X) + E(Y) = \mux + \muy
Var(Z) = E[(Z - E(Z))^2]
Var(Z) = Var(X + Y) = E[(X + Y - (\mux + \muy))^2]
- = E[((X - \mux) + (Y - \muy))^2]
- = E[(X - \mux)^2 + (Y - \muy)^2 + 2(X - \mux)(Y - \muy)]
- = E[(X - \mux)^2] + E[(Y - \muy)^2] + 2 \times E[(X - \mux)(Y - \muy)]
- = Var(X) + Var(Y) + 2Cov(X, Y)
If X and Y are independent, Cov(X,Y) = 0
Therefore, Var(X + Y) = Var(X) + Var(Y)
Definition: Let X and Y be random variables with means \mux and \muy and standard deviations \sigmax and \sigmay. Define the covariance between X and Y by
- Cov(X, Y) = E[(X - \mux)(Y - \muy)]

Variance of Z = X - Y

Let X and Y be two RVs,
Var(X - Y) = E[(X - \mux - Y - \muy)^2]
- = E[(X - \mux)^2] + E[(Y - \muy)^2] - 2 \times E[(X - \mux)(Y - \muy)]
- = Var(X) + Var(Y) - 2Cov(X, Y)
If X and Y are independent
Var(X - Y) = Var(X) + Var(Y)

Rules of Variance as an Operator

Let X and Y be two independent RVs, a, b \in R
If X_i are random variables, then

Variance and Co-variance of Multivariate Distribution

X = \begin{bmatrix} X1 \ X2 \ X3 \ X4 \end{bmatrix}; E(X) = \begin{bmatrix} E(X1) \ E(X2) \ E(X3) \ E(X4) \end{bmatrix} = \begin{bmatrix} \mu1 \ \mu2 \ \mu3 \ \mu4 \end{bmatrix} = \mu
Cov(X) = E[(X - \mu)(X - \mu)']
= E \begin{bmatrix} (X1 - \mu1)(X1 - \mu1) & (X1 - \mu1)(X2 - \mu2) & (X1 - \mu1)(X3 - \mu3) & (X1 - \mu1)(X4 - \mu4) \ (X2 - \mu2)(X1 - \mu1) & (X2 - \mu2)(X2 - \mu2) & (X2 - \mu2)(X3 - \mu3) & (X2 - \mu2)(X4 - \mu4) \ (X3 - \mu3)(X1 - \mu1) & (X3 - \mu3)(X2 - \mu2) & (X3 - \mu3)(X3 - \mu3) & (X3 - \mu3)(X4 - \mu4) \ (X4 - \mu4)(X1 - \mu1) & (X4 - \mu4)(X2 - \mu2) & (X4 - \mu4)(X3 - \mu3) & (X4 - \mu4)(X4 - \mu4) \end{bmatrix}
= \begin{bmatrix} \sigma{11}^2 & \sigma{12}^2 & \sigma{13}^2 & \sigma{14}^2 \ \sigma{21}^2 & \sigma{22}^2 & \sigma{23}^2 & \sigma{24}^2 \ \sigma{31}^2 & \sigma{32}^2 & \sigma{33}^2 & \sigma{34}^2 \ \sigma{41}^2 & \sigma{42}^2 & \sigma{43}^2 & \sigma{44}^2 \end{bmatrix}
Variance Co-Variance Matrix
It is a symmetric Positive definite matrix.

Exercises

Find the first 4 row moments of the variable with pmf, f(x) = 0.2, x = -2, -1, 0, 1, 2.
Find the first two row moments of a continuous random variable with probability density function, f(x) = 2e^{-2x}, x \geq 0.
Find the first 4 central moments of the discrete random variable, the cumulative mass function for which is given by, F(x) = \begin{cases} 0, & x < 1 \ 0.5, & 1 \leq x < 3 \ 1, & x \geq 3 \end{cases} .
Given the mean of two independent random variables X and Y are 12, -50 and the standard deviation of X and Y are respectively 2 and 5, then find:
1. Average and Variance of 3X - 5Y
2. Mean and Standard deviation of 5X + 2Y + 100.
Find the mean and variance of X.
Let X be a random variable with the following probability distribution: Find E(X) and E(X^2) and then, using these values, evaluate E[(2X + 1)^2].

Moments and Moment Generating Function of X (MGF)

The moments uniquely characterize a distribution
If all moments of two distributions are the same, then the distributions are also the same.
The functional form of the moment generating function gives a clue regarding the resulting distribution.
Definition: The moment−generating function (MGF) of the random variable is the function MX(t) of a real parameter t defined by MX(t) = E[e^{tx}], for all t \in R

Why E(e^{tx}) is called a Moment Generating Function of X (MGF)

The k−th row moment mk of a random variable with the moment−generating function MX(t) is given by
- mk = \frac{d^k}{dt^k} MX(t)|_{t=0}
- MX(t) = \sum{k=0}^{\infty} \frac{E[X^k] t^k}{k!}

Moments and Moment Generating Function of X (MGF) - Examples

Find the MGF of X, the outcome while throwing a die, and hence find the mean and variance.
If the random variable X has the MGF M_X(t) = (\frac{3}{3-t}). Find the mean and variance of X.
If the density function of a continuous random variable X is given by: f(x) = \frac{1}{2} e^{-|x|}, -\infty < x < \infty. Find the moment generating function of X, the mean of X and the variance of X.
If the rth row moment of a continuous random variable X about the origin is r!, find the MGF of X.

Chebyshev's Theorem

The probability that any random variable X will assume a value within k standard deviations of the mean is at least 1 - \frac{1}{k^2}. That is,
- P(\mu - k\sigma < X < \mu + k\sigma) \ge 1 - \frac{1}{k^2}.
Example: A random variable X has a mean \mu = 8, a variance \sigma^2 = 9, and an unknown probability distribution. Find
- P(-4 < X < 20)
- P(|X - 8| \ge 6).

Chebyshev’s Theorem – Exercise

A random variable X has a mean \mu = 10 and a variance \sigma^2 = 4. Using Chebyshev’s theorem, find:
- (a) P(|X − 10| \ge 3);
- (b) P(|X − 10| < 3);
- (c) P(5 < X < 15);
- (d) the value of the constant c such that, P(|X − 10| \ge c) \le 0.04.
Compute P(μ − 2σ < X < μ + 2σ), where X has the density function given below, and compare with the result given in Chebyshev’s theorem:
f(x) = \begin{cases} 6x(1 − x), & 0 < x < 1 \ 0, & \text{elsewhere} \end{cases}

Random Variables

Random Variables

Discrete Random Variables

Continuous Random Variables

Probability Distribution

Probability Mass Function (PMF)

Independence

Probability Density Function (PDF)

Important Note on Continuous Random Variables

Cumulative Distribution Function (CDF)

PMF and CMF

CDF for Continuous Random Variables

Properties of CDF for Continuous Random Variables

Cumulative Density Function - Problems

Mean and Variance of Discrete and Continuous Random Variables

Definition

Discrete Random Variables

Continuous Random Variables

Rules of Expectation as an Operator

Central Moments and Row Moments

Uniform Row Moments

Variance - Second Central Moment

Variance of Z = X + Y

Variance of Z = X - Y

Rules of Variance as an Operator

Variance and Co-variance of Multivariate Distribution

Exercises

Moments and Moment Generating Function of X (MGF)

Why E(e^{tx}) is called a Moment Generating Function of X (MGF)

Moments and Moment Generating Function of X (MGF) - Examples

Chebyshev's Theorem

Chebyshev’s Theorem – Exercise

Important Distributions

Discrete

Continuous