Probability 2

What are the three axioms of a probability measure $\mathbb{P}$?

• $$\mathbb{P}(\Omega)=1$$

• $\mathbb{P}(\emptyset)=0$ ;

• for disjoint $A_1,A_2,\dots$ , $\mathbb{P}\left(\bigcup_{i=1}^\infty A_i\right)=\sum_{i=1}^\infty \mathbb{P}(A_i)$

List the four key properties every c.d.f. $F_X$ must satisfy.

• Monotonic non-decreasing

• $F_X(x)\le 1$ with $\lim_{x\to\infty}F_X(x)=1$

• $F_X(x)\ge 0$ with $\lim_{x\to-\infty}F_X(x)=0$

• right continuous ($\lim_{x\to a^+}F_X(x)=F_X(a)$).

Is $F_X$ always left continuous? What is the exact condition for left continuity at a point $a$?

Not always. $F_X$ is left continuous at $a$ if and only if $\mathbb{P}(X=a)=0$.

How do you compute $\mathbb{P}(X<a)$ using $F_X$?

$\mathbb{P}(X<a) = \lim_{x\to a^-} F_X(x)$ (the left-hand limit).

How do you compute $\mathbb{P}(X=a)$ using $F_X$?

$\mathbb{P}(X=a) = F_X(a) - \lim_{x\to a^-}F_X(x)$ — the size of any jump in $F_X$ at $a$.

Bernoulli$(p)$: range, pmf, mean, variance?

Range $\{0,1\}$ $\mathbb{P}(X=1)=p$, $\mathbb{P}(X=0)=1-p$; mean $p$; variance $p(1-p)$.

Binomial$(n,p)$: range, pmf, mean, variance?

Range ${0,1,\dots,n}$;

$\mathbb{P}(X=k)=\binom{n}{k}p^k(1-p)^{n-k}$;

mean $np$;

variance $np(1-p)$.

Geometric$(p)$: range, pmf, mean, variance?

Range ${1,2,\dots}$;

$\mathbb{P}(X=k)=p(1-p)^{k-1}$;

mean $1/p$;

variance $(1-p)/p^2$.

Poisson$(\lambda)$: range, pmf, mean, variance?

Range ${0,1,2,\dots}$;

$\mathbb{P}(X=k)=e^{-\lambda}\lambda^k/k!$ ;

mean $\lambda$;

variance $\lambda$.

Discrete Uniform on ${1,2,\dots,n}$: pmf, mean, variance?

$\mathbb{P}(X=k)=1/n$;

mean $(n+1)/2$;

variance $(n^2-1)/12$.

Uniform$[a,b]$: pdf, cdf, mean, variance?

$f_X(x)=\frac{1}{b-a}$ on $[a,b]$;

$F_X(x)=\frac{x-a}{b-a}$ on $[a,b]$;

mean $\frac{a+b}{2}$;

variance $\frac{(b-a)^2}{12}$.

Exponential$(\lambda)$: pdf, cdf, mean, variance?

$f_X(x)=\lambda e^{-\lambda x}$ for $x>0$;

$F_X(x) = 1 - e^{-\lambda x}$ for $x>0$;

mean $1/\lambda$;

variance $1/\lambda^2$.

Normal$(\mu,\sigma^2)$: pdf, mean, variance?

$f_X(x) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)$;

mean $\mu$;

variance $\sigma^2$.

What are the two general steps for finding the p.d.f. of $Y=g(X)$, for any transformation $g$?

Find the c.d.f. $F_Y(y) = \mathbb{P}(g(X)\le y)$; then differentiate $F_Y$ to get $f_Y$.

Before using the shortcut formula $f_Y(y) = f_X(\phi(y))\lvert\phi’(y)\rvert$ for $Y=g(X)$, what three conditions on $g$ must hold?

• The range of $X$ is an interval.

• $g$ is differentiable with $g'(x)\neq 0$ on the interior of that range

• $g$ is strictly increasing OR strictly decreasing throughout.

Walk through the full procedure for applying the shortcut transformation formula to find $f_Y$ from $f_X$ and $g$.

• Confirm $g$ is monotone and differentiable

• find the inverse function $\phi = g^{-1}$

• differentiate to get $\phi’(y)$

• substitute into $f_Y(y) = f_X(\phi(y))\lvert\phi’(y)\rvert$ over the correct range of $y$.

If $Y = aX+b$ for constants $a\neq 0, b$, what is $f_Y(y)$ in terms of $f_X$?

$$f_Y(y) = \frac{1}{|a|}f_x(\frac{y-b}{a})$$

If $g$ is NOT monotone over the full range of $X$ (e.g. $g(x)=x^2$ with $X$ taking both positive and negative values), what approach must you use instead of the shortcut formula?

Go back to the general two-step procedure directly: write out $F_Y(y)=\mathbb{P}(g(X)\le y)$ by hand, splitting into cases if needed, then differentiate to get $f_Y$.

What two properties must a joint p.m.f. $p_{X,Y}$ satisfy?

• $p_{X,Y}(x,y) > 0$ for $(x,y)$ in the range (and $0$ otherwise

• $\sum_{x}\sum_{y} p_{X,Y}(x,y) = 1$ over the full range.

What are the marginal p.m.f.s of a discrete bivariate random variable, in terms of the joint p.m.f.?

$p_X(x) = \sum_{y} p_{X,Y}(x,y)$ and $p_Y(y) = \sum_{x} p_{X,Y}(x,y)$ — sum out the other variable.

What is a continuous bivariate random variable, and what defines its joint p.d.f.?

$(X,Y)$ is continuous if $F_{X,Y}(x,y) = \int_{-\infty}^y\int_{-\infty}^x f_{X,Y}(s,t),ds,dt$ for some non-negative function $f_{X,Y}$.

More generally, how do you compute $\mathbb{P}((X,Y)\in A)$ for an arbitrary region $A$?

$\mathbb{P}((X,Y)\in A) = \iint_A f_{X,Y}(x,y),dx,dy$ — integrate the joint density over the region $A$.

Two-step check for whether continuous $X$ and $Y$ are independent, by inspecting the joint pdf directly?

(1) The support/domain of $f_{X,Y}$ must be a rectangle or product of intervals, not a region like $x<y$.

(2) $f_{X,Y}(x,y)$ must factor exactly as (function of $x$) $\times$ (function of $y$). If both hold, $X$ and $Y$ are independent.