Random Variables

Random Variables

• Definition: A random variable is a variable whose value is a numerical outcome of a random phenomenon.

Discrete Random Variables

• Examples:
  • The outcome of throwing a die.
  • The number of heads when tossing 'n' coins.
  • The number of customers arriving at a bookstore.
  • The position of participants in a running race.

Continuous Random Variables

• Examples:
  • Heights of randomly selected persons (age > 16).
  • Weights of randomly selected persons (age > 16).
  • The time one has to wait in a queue.
  • The service time in a public service system.
  • The lifetime distribution of electric bulbs.

Probability Distribution

• Discrete Example: A random variable X with probability distribution:

  • X: -2, -1, 0, 1, 2, 3
  • f(X): 0.1, K, 0.2, 2K, 0.3, 3K

• Continuous Example: X is a continuous random variable with PDF:

  $f(x) = \begin{cases} kx, & 0 \leq x \leq 2 \ 2k, & 2 \leq x \leq 4 \ 6k - kx, & 4 \leq x \leq 6 \ 0, & \text{elsewhere} \end{cases}$

Probability Mass Function (PMF)

• Sketching: If the probability distribution of X is given:
  • X: 1, 2, 3, 4
  • f(X): 0.4, 0.3, 0.2, 0.1

Independence

• An assembly consists of three mechanical components with meeting specifications probabilities: 0.95, 0.98, and 0.99 respectively. Assuming independence, determine the probability mass function of the number of components meeting specifications.

Probability Density Function (PDF)

• Let X denote the current in a thin copper wire in milliamperes, with a range of [0, 20mA] and a probability density function f(x) = 0.05 for $0 \leq x \leq 20$.
  • Sketch the PDF
  • Probability that current is less than 10 amperes.
  • Probability that current is greater than 5 amperes.
  • Probability that current is between 10 and 15 amperes.
  • P(X=12) = 0 (for continuous RVs)

Important Note on Continuous Random Variables

• For continuous random variables, $P(X = x) = 0$ for every x.
• The probability is obtained by integrating the PDF.
• A PDF can be greater than 1.
• A PDF can be unbounded.
  • Example: If $f(x) = (2/3)x^{-1/3}$ for 0 < x < 1, then $\int f(x) dx = 1$ even though f is not bounded.

Cumulative Distribution Function (CDF)

• Definition: Cumulative Distribution Function (CDF) for discrete and continuous random variables.

PMF and CMF

• If the random variable X takes values 1, 2, 3, and 4, such that $2P(X=1) = 3P(X=2) = P(X=3) = 5P(X=4)$, find the PMF and cumulative distribution function (CMF) for X. Also, sketch the PMF and CMF of X.

CDF for Continuous Random Variables

• If $f_X(x)$ is the PDF of continuous RV X, then CDF of X is defined.

• Examples:

  1. $f(x) = 0.05, 0 \leq x \leq 20$
  2. $f(x) = \begin{cases} \frac{2}{9}x, &amp; 0 \leq x \leq 3 \ 0, &amp; \text{elsewhere} \end{cases}$

Properties of CDF for Continuous Random Variables

• Let F be the CDF for a random variable X. Then:
  • P(x < X \leq y) = F(y) - F(x)
  • P(X > x) = 1 - F(x)
  • P(a < X \leq b) = P(a \leq X < b) = F(b) - F(a)
  • P(a < x <b) = P(a \leq X \leq b)

Cumulative Density Function - Problems

1. Given CDF for a random variable X, $F(x) = \begin{cases} 0, &amp; x &lt; 0 \ 1 - e^{-x}, &amp; x \geq 0 \end{cases}$, find:

   • The probability mass function
   • P(1<x<3)
   • P(1 \le x<5)
   • P(2 < x \le 5)
   • $P(3 \le x \le 5)$
   • $P(x \le 5)$
   • $P(x \ge 3)$

2. Given the PDF of X, $f(x) = \begin{cases} \frac{2}{9}x, &amp; 0 \leq x \leq 3 \ 0, &amp; \text{elsewhere} \end{cases}$, find the cumulative density function (CDF) for X and evaluate the following probabilities using CDF.

   • $P(x \ge 2)$
   • $P(x \le 1.5)$
   • P(0.5 < x \le 2.5)
   • $P(1 \le x \le 3)$
   • $P(x \le 5)$

Mean and Variance of Discrete and Continuous Random Variables

Definition

• The expected value, or mean, or first moment of X is defined to be

  $E(X) = \begin{cases} \sum x f(x) &amp; \text{if X is discrete} \ \int x f(x) dx &amp; \text{if X is continuous} \end{cases}$

• If X is a random variable with pmf/pdf $f(x)$ and if g(x) is a function of X. Then, $E[g(x)] = \int g(x) f(x) dx$

Discrete Random Variables

1. Find the mean and variance of the number of heads obtained while tossing three coins.

2. Let X equal the number of bits in error in the next four bits transmitted. The possible values of X are {0, 1, 2, 3, 4} and corresponding probabilities are P(X=0)=0.6561; P(X=1)=0.2916; P(X=2)=0.0486; P(X=3)=0.0036 and P(X=4)=0.0001. Find the mean, variance, and standard deviation of X.

Continuous Random Variables

1. Find the mean and variance of X if the probability density function of X is f(x) = 0.125x, 0 < x < 4.

2. Given the pdf of X is $f(x) = ax^2, 0 \leq x \leq 1$.

   • Find ‘a’.
   • Find the value of ‘k’ if P(X \leq k) = P(X > k).
   • Find the mean and standard deviation of X.

Rules of Expectation as an Operator

• Let X and Y be two Random variables
  • $E(cX) = cE(X), c \in R$
  • $E(X + Y) = E(X) + E(Y)$
  • $E(c) = c, c \in R$
  • Example: E(5) = 5
• If X is a random variable with pmf/pdf and if g(x) is a function of X. Then, $E[g(x)] = \int g(x) f(x) dx$

Central Moments and Row Moments

• nth central moment: $\mu_n = E[(X - E(X))^n]$
• nth row moment: $m_n = E[X^n]$
• Central moments are used.
  • $m_1 = \mu = \text{Average(Mean) of X}$
  • $\mu<em>1 = E[(X - E(X))^1] = E(X) - E(m</em>1) = m<em>1 - m</em>1 = 0$
  • $Skewness = \frac{\mu<em>3}{ (\mu</em>2)^{3/2} }$
    • If skewness > 0 ⇒ skewed towards the left.
  • $\mu_4 = E[(X - E(X))^4]$
  • $Kurtosis = \frac{\mu<em>4}{(\mu</em>2)^2}$
    • If kurtosis = 3 ⇒ Normal($\mu$=0, $\sigma$=1)
    • Logistic($\alpha$=0, $\beta$=0.55153) kurtosis = 4.2

Uniform Row Moments

• Uniform ∼ (0,1)
  • $E(X) = \int_0^1 x f(x) dx = \frac{1}{2}$
    • Generate (Draw) N Random values $x<em>i, i=1:N$ from X. Find average $\frac{1}{N} \sum</em>{i=1}^N x_i$. This will tend to $\frac{1}{2}$ when $N \rightarrow \infty$.
  • $E(X^2) = \int_0^1 x^2 f(x) dx = \frac{1}{3}$
    • Generate(Draw) N Random values $x<em>i, i=1:N$ from X. Compute $x</em>i^2 \forall i$. Find average: $\frac{1}{N} \sum<em>{i=1}^N x</em>i^2$. This will tend to $\frac{1}{3}$ when $N \rightarrow \infty$.

Variance - Second Central Moment

• $\mu<em>2 = E[(X - m</em>1)^2] = E(X^2 - 2m<em>1X + m</em>1^2); m_1 = E(X)$
  • $= E(X^2) - E(2m<em>1X) + E(m</em>1^2)$
  • $= E(X^2) - 2m<em>1E(X) + m</em>1^2$
  • $= E(X^2) - 2m<em>1^2 + m</em>1^2$
  • $= E(X^2) - m_1^2$
  • $= E(X^2) - (E(X))^2$
• $Var(X) = E[(X - m_1)^2] = E(X^2) - (E(X))^2$

Variance of Z = X + Y

• $Var(X) = E[(X - \mu<em>x)^2] = E(X^2) - (E(X))^2; \mu</em>x = E(X)$
• Let X and Y be two RVs, Let Z = X + Y; Var(Z) = ?.
• $E(Z) = E(X + Y) = E(X) + E(Y) = \mu<em>x + \mu</em>y$
• $Var(Z) = E[(Z - E(Z))^2]$
• $Var(Z) = Var(X + Y) = E[(X + Y - (\mu<em>x + \mu</em>y))^2]$
  • $= E[((X - \mu<em>x) + (Y - \mu</em>y))^2]$
  • $= E[(X - \mu<em>x)^2 + (Y - \mu</em>y)^2 + 2(X - \mu<em>x)(Y - \mu</em>y)]$
  • $= E[(X - \mu<em>x)^2] + E[(Y - \mu</em>y)^2] + 2 \times E[(X - \mu<em>x)(Y - \mu</em>y)]$
  • $= Var(X) + Var(Y) + 2Cov(X, Y)$
• If X and Y are independent, Cov(X,Y) = 0
• Therefore, $Var(X + Y) = Var(X) + Var(Y)$
• Definition: Let X and Y be random variables with means $\mu<em>x$ and $\mu</em>y$ and standard deviations $\sigma<em>x$ and $\sigma</em>y$. Define the covariance between X and Y by
  • $Cov(X, Y) = E[(X - \mu<em>x)(Y - \mu</em>y)]$

Variance of Z = X - Y

• Let X and Y be two RVs,
• $Var(X - Y) = E[(X - \mu<em>x - Y - \mu</em>y)^2]$
  • $= E[(X - \mu<em>x)^2] + E[(Y - \mu</em>y)^2] - 2 \times E[(X - \mu<em>x)(Y - \mu</em>y)]$
  • $= Var(X) + Var(Y) - 2Cov(X, Y)$
• If X and Y are independent
• $Var(X - Y) = Var(X) + Var(Y)$

Rules of Variance as an Operator

• Let X and Y be two independent RVs, $a, b \in R$
• If $X_i$ are random variables, then

Variance and Co-variance of Multivariate Distribution

• $X = \begin{bmatrix} X<em>1 \ X</em>2 \ X<em>3 \ X</em>4 \end{bmatrix}; E(X) = \begin{bmatrix} E(X<em>1) \ E(X</em>2) \ E(X<em>3) \ E(X</em>4) \end{bmatrix} = \begin{bmatrix} \mu<em>1 \ \mu</em>2 \ \mu<em>3 \ \mu</em>4 \end{bmatrix} = \mu$

• $Cov(X) = E[(X - \mu)(X - \mu)']$

  $= E \begin{bmatrix} (X<em>1 - \mu</em>1)(X<em>1 - \mu</em>1) &amp; (X<em>1 - \mu</em>1)(X<em>2 - \mu</em>2) &amp; (X<em>1 - \mu</em>1)(X<em>3 - \mu</em>3) &amp; (X<em>1 - \mu</em>1)(X<em>4 - \mu</em>4) \ (X<em>2 - \mu</em>2)(X<em>1 - \mu</em>1) &amp; (X<em>2 - \mu</em>2)(X<em>2 - \mu</em>2) &amp; (X<em>2 - \mu</em>2)(X<em>3 - \mu</em>3) &amp; (X<em>2 - \mu</em>2)(X<em>4 - \mu</em>4) \ (X<em>3 - \mu</em>3)(X<em>1 - \mu</em>1) &amp; (X<em>3 - \mu</em>3)(X<em>2 - \mu</em>2) &amp; (X<em>3 - \mu</em>3)(X<em>3 - \mu</em>3) &amp; (X<em>3 - \mu</em>3)(X<em>4 - \mu</em>4) \ (X<em>4 - \mu</em>4)(X<em>1 - \mu</em>1) &amp; (X<em>4 - \mu</em>4)(X<em>2 - \mu</em>2) &amp; (X<em>4 - \mu</em>4)(X<em>3 - \mu</em>3) &amp; (X<em>4 - \mu</em>4)(X<em>4 - \mu</em>4) \end{bmatrix}$

  $= \begin{bmatrix} \sigma<em>{11}^2 & \sigma</em>{12}^2 &amp; \sigma<em>{13}^2 & \sigma</em>{14}^2 \ \sigma<em>{21}^2 & \sigma</em>{22}^2 &amp; \sigma<em>{23}^2 & \sigma</em>{24}^2 \ \sigma<em>{31}^2 & \sigma</em>{32}^2 &amp; \sigma<em>{33}^2 & \sigma</em>{34}^2 \ \sigma<em>{41}^2 & \sigma</em>{42}^2 &amp; \sigma<em>{43}^2 & \sigma</em>{44}^2 \end{bmatrix}$

• Variance Co-Variance Matrix

• It is a symmetric Positive definite matrix.

Exercises

1. Find the first 4 row moments of the variable with pmf, f(x) = 0.2, x = -2, -1, 0, 1, 2.

2. Find the first two row moments of a continuous random variable with probability density function, $f(x) = 2e^{-2x}$, $x \geq 0$.

3. Find the first 4 central moments of the discrete random variable, the cumulative mass function for which is given by, $F(x) = \begin{cases} 0, &amp; x &lt; 1 \ 0.5, &amp; 1 \leq x &lt; 3 \ 1, &amp; x \geq 3 \end{cases}$.

4. Given the mean of two independent random variables X and Y are 12, -50 and the standard deviation of X and Y are respectively 2 and 5, then find:

   1. Average and Variance of 3X - 5Y
   2. Mean and Standard deviation of 5X + 2Y + 100.

5. Find the mean and variance of X.

6. Let X be a random variable with the following probability distribution: Find E(X) and E($X^2$) and then, using these values, evaluate $E[(2X + 1)^2]$.

Moments and Moment Generating Function of X (MGF)

• The moments uniquely characterize a distribution
• If all moments of two distributions are the same, then the distributions are also the same.
• The functional form of the moment generating function gives a clue regarding the resulting distribution.
• Definition: The moment−generating function (MGF) of the random variable is the function $M<em>X(t)$ of a real parameter t defined by $M</em>X(t) = E[e^{tx}]$, for all $t \in R$

Why $E(e^{tx})$ is called a Moment Generating Function of X (MGF)

• The k−th row moment $m<em>k$ of a random variable with the moment−generating function $M</em>X(t)$ is given by
  • $m<em>k = \frac{d^k}{dt^k} M</em>X(t)|_{t=0}$
  • $M<em>X(t) = \sum</em>{k=0}^{\infty} \frac{E[X^k] t^k}{k!}$

Moments and Moment Generating Function of X (MGF) - Examples

1. Find the MGF of X, the outcome while throwing a die, and hence find the mean and variance.

2. If the random variable X has the MGF $M_X(t) = (\frac{3}{3-t})$. Find the mean and variance of X.

3. If the density function of a continuous random variable X is given by: f(x) = \frac{1}{2} e^{-|x|}, -\infty < x < \infty. Find the moment generating function of X, the mean of X and the variance of X.

4. If the rth row moment of a continuous random variable X about the origin is r!, find the MGF of X.

Chebyshev's Theorem

• The probability that any random variable X will assume a value within k standard deviations of the mean is at least $1 - \frac{1}{k^2}$. That is,

  • P(\mu - k\sigma < X < \mu + k\sigma) \ge 1 - \frac{1}{k^2}.

• Example: A random variable X has a mean $\mu = 8$, a variance $\sigma^2 = 9$, and an unknown probability distribution. Find

  • P(-4 < X < 20)
  • P(|X - 8| $\ge$ 6).

Chebyshev’s Theorem – Exercise

1. A random variable X has a mean $\mu = 10$ and a variance $\sigma^2 = 4$. Using Chebyshev’s theorem, find:

   • (a) $P(|X − 10| \ge 3)$;
   • (b) P(|X − 10| < 3);
   • (c) P(5 < X < 15);
   • (d) the value of the constant c such that, $P(|X − 10| \ge c) \le 0.04$.

2. Compute P(μ − 2σ < X < μ + 2σ), where X has the density function given below, and compare with the result given in Chebyshev’s theorem:

   $f(x) = \begin{cases} 6x(1 − x), &amp; 0 &lt; x &lt; 1 \ 0, &amp; \text{elsewhere} \end{cases}$

Important Distributions

Discrete

• Discrete Uniform Distribution
• Binomial Distribution
• Poisson Distribution

Continuous

• Continuous Uniform Distribution
• Exponential Distribution
• Gaussian/Normal Distribution