Continuous probability distributions

What’s a continuous random variable?

We call a random variable X continuously distributed if

P\left\lbrack X=x\right\rbrack=0 for all real numbers X

What’s the distribution function of the continuous RV X?

F_{X}\left(x\right)=P\left\lbrack X\le x\right\rbrack For all x in the reals

What’s the joint distribution function of continuous RVs X and Y

F_{X,Y}\left(x,y\right)=P\left\lbrack X\le x,Y\le y\right\rbrack

What does it mean for X to be distributed like Y

X~y is F_{X}\left(x\right)=F_{Y}\left(x\right) for all x in the reals

What does it mean for two random variables X and Y to be independent?

F_{X,Y}\left(x,y\right)=F_{X}\left(x\right)F_{y}\left(y\right) Fo all x,y in the reals

What does F_{X}\left(\infty\right) for a real-valued random variable X

F_{X}\left(\infty\right)=\lim_{x\to\infty}F_{X}\left(x\right)=1

What does F_{X}\left(-\infty\right) equal for a real-valued random variable X.

F_{X}\left(-\infty\right)=\lim_{x\to-\infty}F_{X}\left(x\right)=0

What is one important property of distribution functions?

F_{X} Is a monotone increasing function.

How do you get a single distribution function (marginal) a joint distribution function F_{X,Y}(x,y)?

From the joint CDF

F_{X,Y}(x,y) = P(X \le x,; Y \le y),

you get the marginal CDFs by letting the other variable go to +\infty:

• For X:
  F_{X}(x)=P(X\le x)=\lim_{y\to\infty}F_{X,Y}(x,y).

• For Y:
  F_Y(y) = P(Y \le y) = \lim_{x \to +\infty} F_{X,Y}(x,y).

What is the probability density function?

What is the expectation of a RV X with density f_{X}

E\left\lbrack X\right\rbrack=\int_{-\infty}^{\infty}u\times\!\,f\left(u\right)du whenever the integral is finite.

Considering X and Y RVs with existing f_{X} and existing f_{X,Y}, what’s the epectation of functions g:R→R and h:RxR→R

E[g(X)] = \int_{-\infty}^{\infty} g(x)\, f_X(x)\, dx

E[h(X,Y)] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} h(x,y)\, f_{X,Y}(x,y)\, dy\, dx

What is the indicator function?

Let A be a subset of some space S. The indicator function 1_A : S \to \mathbb{R} is defined by

1_A(x) = 1 \text{ if } x \in A,\quad 0 \text{ if } x \notin A.

Here S is typically the sample space \Omega or \mathbb{R}, and A is usually an event or an interval.

Whats the expectation of aX+b where X is an RV with existing f_{X} and finite expectation.

E\left\lbrack aX+b\right\rbrack=a\times E\left\lbrack X\right\rbrack+b

Whats the uniform distribution?

A random variable X is uniformly distributed on [a,b], written X\sim U(a,b), if it has density f_X(x)=\begin{cases}\dfrac{1}{b-a},&x\in[a,b],\\0,&x\notin[a,b].\end{cases}

Whats the expectation and variance of a uniform distributed random variable X.

If X\sim U(a,b), then

E[X]=\dfrac{a+b}{2}

\mathrm{Var}(X)=\dfrac{(b-a)^2}{12}

What’s the Exponential Distribution

A random variable X is exponentially distributed with parameter \lambda>0, written X\sim\mathrm{Exp}(\lambda), if its density is

f_X(x)=\begin{cases}\lambda e^{-\lambda x},&x>0,\\0,&x\le0.\end{cases}

What’s the expectation and variance of a exponentially distributed RV X?

If X\sim\mathrm{Exp}(\lambda), then:

E[X]=\dfrac{1}{\lambda}

\mathrm{Var}(X)=\dfrac{1}{\lambda^2}.

Whats E\left\lbrack X^{n}\right\rbrack for RV X~Exp(\lambda)

\frac{n!}{\lambda^{n}}

What does the exponential distribution measure?

How long we have to wait for something.

Explain the memorylessness of the exponential distribution

Past waiting does’t affect future waiting times.

If X\sim\mathrm{Exp}(\lambda) then for all s,t\ge 0 with P(X>s)>0 we have P(X>s+t\mid X>s)=P(X>t).

Whats the geometric distribbution used for

If the time between individual events (like one customer arriving) follows an Exponential distribution, the Gamma distribution models the total time for multiple such events to occur.

Whats the probability density function of the gamma distribution?

f_{X}(x)=\frac{\lambda^{\alpha}}{\Gamma\left(\alpha\right)}x^{\alpha-1}e^{-\lambda x} for x>0

in which \Gamma is the ‘gamma function’

\Gamma\left(\alpha\right)=\int_0^{\infty}\!x^{\alpha-1}e^{-x}\,dx

Whats \Gamma\left(\alpha\right) if \alpha is an integer n?

(n-1)!

What does \Gamma(\alpha+1)= ?

\alpha\Gamma(\alpha)

What’s the Gamma identity used in expectation proofs?

For

\alpha>0,

\Gamma(\alpha+1)=\int_0^\infty x^{\alpha}e^{-x}\,dx=\alpha\int_0^\infty x^{\alpha-1}e^{-x}\,dx=\alpha\Gamma(\alpha).

Proof: integrate by parts with u=x^\alpha, dv=e^{-x}dx so du=\alpha x^{\alpha-1}dx and v=-e^{-x}; the boundary term [-x^\alpha e^{-x}]_0^\infty=0, leaving \Gamma(\alpha+1)=\alpha\Gamma(\alpha)."

What’s the expectation and variance of the gamma distribution?

\mathbb{E}(X)=\frac{\alpha}{\lambda}

Var(X)=\frac{\alpha}{\lambda²}

Whats the poisson distribution used for?

“The number of events in a given time”

What is the pmf of the poisson distribution?

P_{X}(k)=\frac{\lambda^k}{k!}e^-\lambda for k=0,1,2,3,…

Likely to see two things

X~Po(\lambda) (one unit time)

N_{t}~Po(\lambda t) (“rate per unit time, where t = time)

What is the expectation and variance of a poisson distributed random variable X~Po(\lambda)?

\mathbb{E}(X)=\lambda

Var(X)=\lambda

What is the power series expansion of the exponential?

\Sigma_0^{\infty}\frac{x^{n}}{n!}=e^{x}

How can we rewrite a gamma distributed RV in terms of a poisson RV?

If N_{t}~Po(\lambda t) and Y~Gam(n,\lambda)

then P\left(Y\le t\right)=P\left(N_{t}\ge n\right)

Whats the density of the standard normal distribution (X~N(0,1))?

f_{X}(x)=\frac{1}{\sqrt{2\pi}}\times e^{\frac{-x^{2}}{2}}

often denoted with \varphi(x) (phi)

Whats the distribution function of a standard normally distributed RV?

F_{X}(x)=P(X\le x)=\Phi\left(x\right)=\int_{-\infty}^{x}\frac{1}{\sqrt{2\pi}}\!e^{-\frac{u^2}{2}}\,du

Facts about the symmetry of the normal distribution

\Phi\left(-x\right)=1-\Phi\left(x\right)

whats the expectation and variance of the standard normal distribution?

\mathbb{E}(X)=0

Var(X)=1

What is the Central Limit Theorem?

Let \left(X_{i}\right)_{i=1}^{\infty} be i.i.d. with \mu=E[X_1] and 0<\sigma^2=\mathrm{Var}(X_1)<\infty.

Define S_{n}=\sum_{i=1}^{n}X_{i} and Z_n=\frac{S_n-n\mu}{\sigma\sqrt{n}}.

Then as n\to\infty, Z_n \xrightarrow{d} \mathcal N(0,1),

i.e. \lim_{n\to\infty}P(Z_n\le z)=\Phi(z) for all real z (where \Phi is the standard normal CDF)

What is the normal distribution?

A RV X is normally distributed with parameters \mu\in\mathbb R and \sigma>0, written X\sim N(\mu,\sigma^2), if

• F_X(x)=\Phi\!\left(\frac{x-\mu}{\sigma}\right) for all x\in\mathbb R, or equivalently

• X\sim \mu+\sigma Z where Z\sim N(0,1) is standard normal.

Then X has density f_X(x)=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-(x-\mu)^2/(2\sigma^2)} for x\in\mathbb R."

What’s the expectation and variance of a normal distributed RV X

\mathbb{E}(X)=\mu

Var(X)=\sigma^2

If X~N(\mu,\sigma²), what is c x X where c is a constant ≠ 0

c\times X~N(c\mu,c²\sigma²)

If X and Y are two normally distributed RVs what’s the distribution of X + Y

X+Y~N(\mu + \mu_{2}, \sigma² + \sigma²_{2})