Multiple Random Variables and Operations

Multiple Random Variables

Concept of Multiple RV

• Multiple random variables describe random points in multi-dimensions.
• A random point in a 2D plane can be represented as $\theta = (\theta<em>1, \theta</em>2)$.

Two Random Variables

• A random point is a pair of numbers $(x, y)$ in the $xy$ plane mapped from a sample space $S$.
• Joint Event: Domain in the plane representing the intersection of two events $A$ and $B$.
• For $N$ Random Variables: Denoted by $X<em>1, X</em>2, …, X_N$. The joint sample space becomes $N$-dimensional.

Joint Distribution Function

• Probability of the joint event $\lbrace X \le x, Y \le y \rbrace$ is denoted as $F_{X,Y}(x, y) = P\lbrace X \le x, Y \le y \rbrace$ and is called the JOINT PROBABILITY DISTRIBUTION FUNCTION (JOINT CDF).
• For N Random Variables: $F<em>{X</em>1,X<em>2,…,X</em>N}(x<em>1, x</em>2, …, x<em>N) = P\lbrace X</em>1 \le x<em>1, X</em>2 \le x<em>2, …, X</em>N \le x_N \rbrace$
• Properties:
  • $F<em>{X,Y}(-\infty, -\infty) = 0$, $F</em>{X,Y}(-\infty, y) = 0$, $F_{X,Y}(x, -\infty) = 0$
  • $F_{X,Y}(\infty, \infty) = 1$
  • $0 \le F_{X,Y}(x, y) \le 1$

Joint Density Function

• Definition: Second derivative of Joint CDF $f<em>{X,Y}(x, y) = \frac{\partial^2 F</em>{X,Y}(x, y)}{\partial x \partial y}$. For discrete RVs, it is called JOINT PROBABILITY MASS FUNCTION (PMF).
• Joint PDF of N RVs: $f<em>{X</em>1,X<em>2,…,X</em>N}(x<em>1, x</em>2, …, x_N)$
• Joint CDF of N RVs: $F<em>{X</em>1,X<em>2,…,X</em>N}(x<em>1, x</em>2, …, x<em>N) = \int</em>{-\infty}^{x<em>N} … \int</em>{-\infty}^{x<em>2} \int</em>{-\infty}^{x<em>1} f</em>{X<em>1,X</em>2,…,X<em>N}(\xi</em>1, \xi<em>2, …, \xi</em>N) d\xi<em>1 d\xi</em>2 … d\xi_N$
• Properties:
  • $f_{X,Y}(x, y) \ge 0$
  • $\int<em>{-\infty}^{\infty} \int</em>{-\infty}^{\infty} f_{X,Y}(x, y) dx dy = 1$

Marginal Density Function

• Marginal Density Function of two RVs:
  $f<em>X(x) = \int</em>{-\infty}^{\infty} f<em>{X,Y}(x, y) dy$$f</em>Y(y) = \int<em>{-\infty}^{\infty} f</em>{X,Y}(x, y) dx$
• Marginal density function of N RVs:
  $f<em>{X</em>1,X<em>2,…,X</em>{N-1}}(x<em>1, x</em>2, …, x<em>{N-1}) = \int</em>{-\infty}^{\infty} … \int<em>{-\infty}^{\infty} f</em>{X<em>1,X</em>2,…,X<em>N}(x</em>1, x<em>2, …, x</em>N) dx<em>{N+1} dx</em>{N+2} … dx_N$

Conditional Distribution and Density

• Conditioning on another RV, i.e., Y. Event $B = \lbrace y-\Delta \le Y \le y + \Delta \rbrace$
• Conditional PMF:
  $P<em>{X|Y}(x|y) = \frac{P</em>{X,Y}(x, y)}{P<em>Y(y)}$ for $P</em>Y(y) \neq 0$
• Bayes Rule for pmfs:
  $P<em>{X|Y}(x|y) = \frac{P</em>{Y|X}(y|x) P<em>X(x)}{\sum</em>{x' \in X} P<em>{X}(x')P</em>{Y|X}(y|x')}$
• Density function of continuous RVs:
  $f<em>{X|Y}(x|y) = \frac{f</em>{X,Y}(x, y)}{f_Y(y)}$