Joint Probability Distributions – Discrete & Continuous

Learning Objectives

• Enumerate the different types of discrete/continuous random variables and their probability distributions.

• Compute probabilities of events for each type of distribution.

• Build the joint probability distribution (pmf/pdf) for two or more random variables.

• Derive marginal and conditional distributions from a joint distribution.

• Extend the ideas to more than two random variables and to the continuous‐valued case.

Joint Probability Distributions – DISCRETE Case

• For two discrete r.v.’s $X$ and $Y$, the joint probability mass function (pmf) is

  $f(x,y)=P\bigl(X=x,\,Y=y\bigr).$

• Requirements for a valid joint pmf

  • $f(x,y)\ge 0$ for all $(x,y)$ in the support.

  • $\sum<em>x\sum</em>y f(x,y)=1.$

  • For any region $A$ in the $xy$–plane,

  $P\bigl((X,Y)\in A\bigr)=\sum_{(x,y)\in A}f(x,y).$

Bridge-Hand Example

• Experiment: Choose a bridge hand (13 cards) from a standard deck.

• Random variables

  • $X=$ number of spades,

  • $Y=$ number of hearts,

  • Constraints: $x,y\in{0,1,\dots,13},\;x+y\le 13.$

• Joint pmf

  $p(x,y)=\frac{\binom{13}{x}\,\binom{13}{y}\,\binom{26}{13-x-y}}{\binom{52}{13}}.$

• A tabulated (and graphed) set of probabilities shows the mass concentrated near the centre; all cells sum to $1$.

• Interpretation: $p(5,3)$ is the probability that the hand contains 5 spades and 3 hearts.

Dice-Rolling Example (Binomial–Multinomial flavour)

• Roll a fair die $n=5$ times.

  • $X=$ number of sixes,

  • $Y=$ number of fives,

  • Support: $x,y\in{0,1,\dots,5},\;x+y\le 5.$

• Each sequence of length 5 containing $x$ sixes and $y$ fives has probability

  $(1/6)^x\,(1/6)^y\,(4/6)^{5-x-y}.$

• Number of such sequences $=\dfrac{5!}{x!\,y!\,(5-x-y)!}.$ .

• Hence joint pmf

  $p(x,y)=\frac{5!}{x!y!(5-x-y)!}\left(\frac16\right)^{x+y}\left(\frac46\right)^{5-x-y},\qquad x+y\le 5.$

• A $6\times6$ probability table is provided; values again sum to $1$.

• Sample query: Probability we roll more sixes than fives

  $P[X>Y]=0.3441.$

General Properties Recap

1. Non-negativity: $0\le p(x,y)\le 1.$

2. Total probability: $\sum<em>x\sum</em>y p(x,y)=1.$

3. Region probabilities via double summation.

Marginal Distributions (DISCRETE)

• Marginal pmf of $X$: $p<em>X(x)=P[X=x]=\sum</em>y p(x,y).$

• Marginal pmf of $Y$: $p<em>Y(y)=P[Y=y]=\sum</em>x p(x,y).$

• Example (dice, $n=5$):

  The marginal row and column totals in the table give $p<em>X(\cdot)$ and $p</em>Y(\cdot)$.

  For instance $pX(0)=0.4019,\;pY(2)=0.1608,$ etc.

Conditional Distributions (DISCRETE)

• Conditional pmf of $X$ given $Y=y$:

  p{X\mid Y}(x\mid y)=\frac{p(x,y)}{pY(y)},\quad p_Y(y)>0.

• Conditional pmf of $Y$ given $X=x$:

  p{Y\mid X}(y\mid x)=\frac{p(x,y)}{pX(x)},\quad p_X(x)>0.

• In example (dice): Complete conditional tables show how the distribution of one variable changes when the other variable is fixed.

  • Given $Y=0$, $P[X=0\mid Y=0]=0.3277,\;P[X=5\mid Y=0]=0.0003,$ etc.

  • Given $X=4$, $P[Y=0\mid X=4]=0.8000,$ etc.

• Interpretation: Marginals “ignore” the other variable; conditionals “zoom in” on a slice where the other variable is known.

Joint Distributions of MORE THAN TWO Discrete Variables

• Ideas extend via higher-dimensional tables or lists.

• Joint pmf $p(x<em>1,x</em>2,\dots,x_k)$ still obeys non-negativity and $\sum p=1.$

• Marginalisation: sum over all unwanted indices.

• Conditioning: divide joint pmf by appropriate marginal.

Joint Probability Distributions – CONTINUOUS Case

• A pair of continuous r.v.’s has joint probability density function (pdf) $f(x,y).$

• Requirements for a valid joint pdf

  • $f(x,y)\ge0.$

  • $\int<em>{-\infty}^{\infty}\int</em>{-\infty}^{\infty} f(x,y)\,dx\,dy =1.$

  • For a region $A$, $P[(X,Y)\in A]=\iint_A f(x,y)\,dx\,dy.$

Multiple Integration Basics

• For a rectangular region $A={(x,y):a\le x\le b,\;c\le y\le d}$,

  $\iint<em>A f(x,y)\,dx\,dy = \int</em>c^d\int<em>a^b f(x,y)\,dx\,dy = \int</em>a^b\int_c^d f(x,y)\,dy\,dx.$

• “Inner integral first” computes the area (volume) slice for fixed outer variable, then integrates the slices.

• Geometric picture: inner integral = area under surface above a line where the outer variable is held constant; outer integral = sum of infinitesimal volumes.

Worked Integral Example

• Evaluate $\int<em>0^1\int</em>0^1 (x^2+3y^2+2xy)\,dx\,dy.$

• By either order, result is $\frac76+\frac{8}{24}=1.5.$ (Demonstrates equality of iterated integrals and ease of changing order.)

Marginal Densities (CONTINUOUS)

• $f<em>X(x)=\int</em>{-\infty}^{\infty} f(x,y)\,dy.$

• $f<em>Y(y)=\int</em>{-\infty}^{\infty} f(x,y)\,dx.$

Conditional Densities (CONTINUOUS)

• f{Y\mid X}(y\mid x)=\dfrac{f(x,y)}{fX(x)},\quad f_X(x)>0.

• f{X\mid Y}(x\mid y)=\dfrac{f(x,y)}{fY(y)},\quad f_Y(y)>0.

THE BIVARIATE NORMAL DISTRIBUTION

• Parameter set: means $\mu<em>1,\mu</em>2,$ standard deviations $\sigma<em>1,\sigma</em>2,$ correlation \rho\, (||\rho||<1).

• Quadratic form

  $Q(x<em>1,x</em>2)=\frac{(x<em>1-\mu</em>1)^2}{\sigma<em>1^2}-\frac{2\rho(x</em>1-\mu<em>1)(x</em>2-\mu<em>2)}{\sigma</em>1\sigma<em>2}+\frac{(x</em>2-\mu<em>2)^2}{\sigma</em>2^2}.$

• Density function

  f(x1,x2)=\frac{1}{2\pi\sigma1\sigma2\sqrt{1-\rho^2}}\exp\Biggl{-\frac{1}{2(1-\rho^2)}Q(x1,x2)\Biggr}.

• Contours of constant $Q$ are ellipses centred at $(\mu<em>1,\mu</em>2).$

  • $\rho=0$ gives circular (independent) contours.

  • $\rho=0.90$ stretches along the main diagonal; $\rho=-0.90$ stretches along the anti-diagonal.

Practical Connections & Remarks

• Joint distributions underpin correlation, regression, hypothesis testing, and simulation.

• Discrete joint pmfs appear in quality control (defects per unit), card games, genetics (allele counts), etc.

• Continuous joint pdfs (notably the bivariate Normal) drive multivariate analytics, e.g.

  • Risk management – asset returns,

  • Signal processing – joint noise models,

  • Engineering – joint tolerances of component dimensions.

Key Formulae Summary

• Joint pmf: $f(x,y)=P[X=x,Y=y].$

• Joint pdf: $\iint f(x,y)\,dx\,dy=1.$

• Marginals: $p<em>X(x)=\sum</em>y f(x,y),\;f_X(x)=\int f(x,y)\,dy.$

• Conditionals: $p<em>{X\mid Y}(x\mid y)=\dfrac{f(x,y)}{p</em>Y(y)},\;f<em>{X\mid Y}(x\mid y)=\dfrac{f(x,y)}{f</em>Y(y)}.$

• Multinomial-style joint pmf (dice):

  $p(x,y)=\dfrac{n!}{x!y!(n-x-y)!}p<em>1^x p</em>2^y p_3^{n-x-y}.$

• Bridge hand pmf:

  $p(x,y)=\dfrac{\binom{13}{x}\,\binom{13}{y}\,\binom{26}{13-x-y}}{\binom{52}{13}}.$

• Bivariate Normal density: see formula above.

Study Tips

• Always sketch the support region $(x,y)$; constraints like $x+y\le n$ are crucial.

• Verify pmf/pdf conditions before doing further work.

• For marginals/conditionals, write the summation/integration limits explicitly.

• In probability queries (e.g. $P[X>Y]$), highlight the set $(x,y):x>y$ on the table/plane before summing/integrating.

• When changing order of