Joint Distribution Notes

Practice Questions for Examination

Practice questions are for practice purposes only, but they cover the syllabus. This session (number 10) continues the discussion on joint distribution, which began in the last class.

Joint Distribution

When dealing with more than one random variable, joint distribution is used. This session covers three main parts:

1. Joint distribution and marginal distribution

2. Dependence and independence of random variables

3. Expectation, mean, and variance

Questions similar to the practice ones might appear in the exam. The session focuses on three parts:

• Discrete bivariate

• Continuous bivariate

• Multivariate

Discrete Bivariate Random Variables

For two discrete random variables $x$ and $y$, the probability that $x$ takes the value $x$ and $y$ takes the value $y$ is written as: $P(X = x, Y = y)$.

This is an intersection, and the joint density function is defined as:

$f(x, y) = P(X = x, Y = y)$

This is the joint probability mass function or joint probability density function.

Example: Rolling two dice. Let $x$ be the number on the first die and $y$ be the number on the second die. Both $x$ and $y$ vary from 1 to 6, and their density function is $1/36$ for all $x$ and $y$.

Joint Probability Table

Each entry represents the joint density function for the corresponding $x$ and $y$ values. Also, probabilities are always greater than zero and less than one. The summation for all values should be equal to one:

$\sum<em>{x} \sum</em>{y} f(x, y) = 1$

Example: Let $x$ be the value on the first die and $t$ be the total on both dice. $x$ varies from 1 to 6, while $t$ varies from 2 to 12.

Table for x and t

Example: Event $B: y - x \geq 2$. Find the probability. Add their density functions.

Marginal Distribution

Marginal distribution involves taking a projection from two dimensions to one dimension.

For example, given two random variables $x$ and $y$, transform the two-dimensional random variable into one-dimensional random variables, $x$ alone and $y$ alone. This allows the application of concepts from previous chapters (3 & 4) to find PDFs, CDFs, means, etc.

Marginal density function for $x$ is given as:

$f<em>X(x) = \sum</em>{y} f(x, y)$

This is a function of $x$ alone, as the effect of $y$ is accumulated.

Marginal density function for $y$ is given as:

$f<em>Y(y) = \sum</em>{x} f(x, y)$

This is a function of $y$ alone, accumulating the effect of $x$ on $y$.

Notation

• $f(x, y)$ or $f_{XY}(x, y)$: Joint density function

• $f_X(x)$: Marginal density function for $x$

• $f_Y(y)$: Marginal density function for $y$

Example: Calculating marginal density functions from a joint distribution table. Add row-wise to find $f<em>X(x)$ and column-wise to find $f</em>Y(y)$.

f_X(x) = \begin{cases}
1/6, & x = 1, 2, 3, 4, 5, 6 \
0, & \text{otherwise}
\end{cases}

f_Y(y) = \begin{cases}
1/6, & y = 1, 2, 3, 4, 5, 6 \
0, & \text{otherwise}
\end{cases}

Cumulative Distribution Function (CDF)

For $x$ alone:

F_X(x) = \begin{cases}
0, & x < 1 \
1/6, & 1 \leq x < 2 \
2/6, & 2 \leq x < 3 \
3/6, & 3 \leq x < 4 \
4/6, & 4 \leq x < 5 \
5/6, & 5 \leq x < 6 \
1, & x \geq 6
\end{cases}

Dependence and Independence of Random Variables

Two random variables are independent if the joint density function is equal to the product of the marginal density functions:

$f(x, y) = f<em>X(x) \cdot f</em>Y(y)$

Otherwise, they are dependent.

Example: Using the dice example, if $f(x=3, y=4) = f<em>X(3) \cdot f</em>Y(4)$, then they are independent.

If variables are not independent, the concept of conditional probability applies. Recall from chapter 3:

$P(A \cap B) = P(A) \cdot P(B)$

For conditional probability:

$P(A | B) = \frac{P(A \cap B)}{P(B)}, \text{provided } P(B) \neq 0$

Conditional Density Function

The conditional density function can be written as:

$f(x | y) = \frac{f(x, y)}{f<em>Y(y)}, \text{provided } f</em>Y(y) \neq 0$

This is the probability of $X = x$ given that $Y = y$.

Example: Finding the total value of 4, given the first die is 3. $f(t=4 | x=3) = \frac{1/36}{1/6} = 1/6$

Independence and Conditional Probability

If two random variables are independent, the conditional probability becomes the same as the marginal probability:

$f(x | y) = f_X(x)$

Expectation

Expectation involves adding over the entire sample space. With two random variables, it involves adding for all $x$ and all $y$.

The expectation of any function $h(x, y)$ is:

$E[h(x, y)] = \sum<em>{x} \sum</em>{y} h(x, y) \cdot f(x, y)$

Mean

The mean for x ($\mu_x$) is:

$E[x] = \sum<em>{x} \sum</em>{y} x \cdot f(x, y) = \sum<em>{x} x \cdot f</em>X(x)$

The mean for y ($\mu_y$) is:

$E[y] = \sum<em>{x} \sum</em>{y} y \cdot f(x, y) = \sum<em>{y} y \cdot f</em>Y(y)$

Expectation of xy

To find the expected value of $xy$:

$E[xy] = \sum<em>{x} \sum</em>{y} xy \cdot f(x, y)$

Marginal density functions cannot be used here.

Covariance

Covariance is defined as:

$Cov(x, y) = E[(x - \mu<em>x)(y - \mu</em>y)]$

A direct formula is:

$Cov(x, y) = E[xy] - E[x] \cdot E[y]$

Variance of x

Instead of $x - \mu_x$ squared, covariance of $x$ and $y$ is used.

Tossing two fair coins: Let $x$ be the number of heads and $y$ be the number of tails. Find the joint density function, CDF, marginal density function, and check whether $x$ and $y$ are independent. If dependent, find $Cov(x, y)$.

Covariance and Independence

Covariance measures the joint variability of two random variables. If $x$ and $y$ are independent, then $Cov(x, y) = 0$. However, if $Cov(x, y) = 0$, it does not necessarily mean that $x$ and $y$ are independent.

Zero covariance means no linear relationship.

Important

Covariance formula: $Cov(x, y) = E[xy] - E[x] \cdot E[y]$.

Continuous Joint Distribution

For continuous random variables, the concepts are similar to the discrete case, but summations are replaced with integrations.

For two continuous random variables $x$ and $y$, a function $f(x, y)$ such that $f(x, y) \geq 0$ and $\int<em>{-\infty}^{\infty} \int</em>{-\infty}^{\infty} f(x, y) dx dy = 1$ is the joint probability density function.

The CDF is:

$F(x, y) = \int<em>{-\infty}^{x} \int</em>{-\infty}^{y} f(x, y) dx dy$

Marginal density functions are:

$<br>f<em>X(x) = \int</em>{-\infty}^{\infty} f(x, y) dy<br>$

$<br>f<em>Y(y) = \int</em>{-\infty}^{\infty} f(x, y) dx<br>$

Independence: $f(x, y) = f<em>X(x) \cdot f</em>Y(y)$

Expectation:

$E[h(x, y)] = \int<em>{-\infty}^{\infty} \int</em>{-\infty}^{\infty} h(x, y) \cdot f(x, y) dx dy$

Example

Bank operates drive-up and walk-up. Let $x$ be the proportion of time the drive-up is in use, and $y$ be the proportion of time the walk-up is in use. The joint density function is given. Calculate the probability that neither facility is busy more than one quarter of the time. (Limits are 0 to 1/4 for both integrals.)

Also marginal pdf for x gives probability distribution of busy time for drive up facility without reference to the walk up window. The same is true for y.

Conditional PDF

To find the conditional PDF of $y$ given that $x = 0.8$, use:

$f(y | x) = \frac{f(x, y)}{f_X(x)}$

Important Results on Expectation

$<br>E[Ax + B] = A \cdot E[x] + B<br>$

$\operatorname{Var}(A x+B)=A^{2} \operatorname{Var}(x)<br>$

$<br>E[Ax + By] = A \cdot E[x] + B \cdot E[y]<br>$

If x and y are Independent

$E[X \cdot Y] = E[X] \cdot E[Y]$