Z

6. Probability Distributions


6.1 Discrete Distributions


1. Discrete Uniform DistributionNotation

X∼U(a,b)X∼U(a,b)

where XX takes integer values from aa to bb with equal probability.

PMF

P(X=x)=1b−a+1P(X=x)=b−a+11​

for x=a,a+1,…,bx=a,a+1,…,b.

Mean

E[X]=a+b2E[X]=2a+b​

Variance

Var⁡(X)=(b−a+1)2−112Var(X)=12(b−a+1)2−1​


2. Bernoulli DistributionNotation

X∼Bernoulli⁡(p)X∼Bernoulli(p)

A Bernoulli random variable has two outcomes:

  • Success: X=1X=1

  • Failure: X=0X=0

PMF

P(X=x)=px(1−p)1−xP(X=x)=px(1−p)1−x

for x∈{0,1}x∈{0,1}.

Mean

E[X]=pE[X]=p

Variance

Var⁡(X)=p(1−p)Var(X)=p(1−p)

MGF

MX(t)=(1−p)+petMX​(t)=(1−p)+pet


3. Geometric DistributionNotation

X∼Geo⁡(p)X∼Geo(p)

XX is the number of trials until the first success.

PMF

P(X=x)=(1−p)x−1pP(X=x)=(1−p)x−1p

for x=1,2,3,…x=1,2,3,…

CDF

F(x)=1−(1−p)xF(x)=1−(1−p)x

Mean

E[X]=1pE[X]=p1​

Variance

Var⁡(X)=1−pp2Var(X)=p21−p​

MGF

MX(t)=pet1−(1−p)etMX​(t)=1−(1−p)etpet​

for t<−ln⁡(1−p)t<−ln(1−p).


4. Binomial DistributionNotation

X∼Bin⁡(n,p)X∼Bin(n,p)

XX is the number of successes in nn independent Bernoulli trials.

PMF

P(X=k)=(nk)pk(1−p)n−kP(X=k)=(kn​)pk(1−p)n−k

for k=0,1,…,nk=0,1,…,n.

Mean

E[X]=npE[X]=np

Variance

Var⁡(X)=np(1−p)Var(X)=np(1−p)

MGF

MX(t)=[(1−p)+pet]nMX​(t)=[(1−p)+pet]n


5. Negative Binomial DistributionNotation

X∼NegBin⁡(r,p)X∼NegBin(r,p)

XX is the number of trials needed to get rr successes.

PMF

P(X=x)=(x−1r−1)pr(1−p)x−rP(X=x)=(r−1x−1​)pr(1−p)x−r

for x=r,r+1,…x=r,r+1,…

Mean

E[X]=rpE[X]=pr​

Variance

Var⁡(X)=r(1−p)p2Var(X)=p2r(1−p)​

MGF

MX(t)=(pet1−(1−p)et)rMX​(t)=(1−(1−p)etpet​)r


6. Poisson DistributionNotation

X∼Poisson⁡(λ)X∼Poisson(λ)

Used to model the number of events occurring in a fixed interval of time or space.

PMF

P(X=k)=e−λλkk!P(X=k)=k!e−λλk​

for k=0,1,2,…k=0,1,2,…

Mean

E[X]=λE[X]=λ

Variance

Var⁡(X)=λVar(X)=λ

MGF

MX(t)=exp⁡(λ(et−1))MX​(t)=exp(λ(et−1))

Poisson Approximation to Binomial

If nn is large, pp is small, and λ=npλ=np is moderate, then:

Bin⁡(n,p)≈Poisson⁡(λ)Bin(n,p)≈Poisson(λ)


6.2 Continuous Distributions


1. Continuous Uniform DistributionNotation

X∼U(a,b)X∼U(a,b)

PDF

f(x)=1b−af(x)=b−a1​

for a≤x≤ba≤x≤b.

CDF

F(x)=x−ab−aF(x)=b−ax−a​

for a≤x≤ba≤x≤b.

Mean

E[X]=a+b2E[X]=2a+b​

Variance

Var⁡(X)=(b−a)212Var(X)=12(b−a)2​

MGF

MX(t)=etb−etat(b−a)MX​(t)=t(b−a)etb−eta​

for t≠0t=0.


2. Exponential DistributionNotation

X∼Exp⁡(λ)X∼Exp(λ)

Used to model waiting times between events in a Poisson process.

PDF

f(x)=λe−λxf(x)=λe−λx

for x≥0x≥0.

CDF

F(x)=1−e−λxF(x)=1−e−λx

Mean

E[X]=1λE[X]=λ1​

Variance

Var⁡(X)=1λ2Var(X)=λ21​

MGF

MX(t)=λλ−tMX​(t)=λ−tλ​

for t<λt<λ.

Memoryless Property

P(X>s+t∣X>s)=P(X>t)P(X>s+t∣X>s)=P(X>t)


3. Normal DistributionNotation

X∼N(μ,σ2)X∼N(μ,σ2)

PDF

f(x)=12πσ2exp⁡(−(x−μ)22σ2)f(x)=2πσ2​1​exp(−2σ2(x−μ)2​)

Mean

E[X]=μE[X]=μ

Variance

Var⁡(X)=σ2Var(X)=σ2

MGF

MX(t)=exp⁡(μt+12σ2t2)MX​(t)=exp(μt+21​σ2t2)

Standardization

If:

X∼N(μ,σ2)X∼N(μ,σ2)

then:

Z=X−μσ∼N(0,1)Z=σX−μ​∼N(0,1)


4. Standard Normal DistributionNotation

Z∼N(0,1)Z∼N(0,1)

PDF

f(z)=12πe−z2/2f(z)=2π​1​e−z2/2

Mean

E[Z]=0E[Z]=0

Variance

Var⁡(Z)=1Var(Z)=1

CDF

The CDF is denoted by:

Φ(z)Φ(z)

There is no elementary closed form.

MGF

MZ(t)=exp⁡(t22)MZ​(t)=exp(2t2​)


6.3 Moment Generating Function

The moment generating function of XX is:

MX(t)=E[etX]MX​(t)=E[etX]

Why MGF Is Useful

It helps to find moments.

The nnth moment about the origin is:

μn′=dndtnMX(t)∣t=0μn′​=dtndn​MX​(t)​t=0​

Therefore:

E[X]=MX′(0)E[X]=MX′​(0)

E[X2]=MX′′(0)E[X2]=MX′′​(0)

and:

Var⁡(X)=MX′′(0)−(MX′(0))2Var(X)=MX′′​(0)−(MX′​(0))2

MGF Summary Table

Distribution

MGF

Bernoulli(p)(p)

(1−p)+pet(1−p)+pet

Binomial(n,p)(n,p)

[(1−p)+pet]n[(1−p)+pet]n

Geometric(p)(p)

pet1−(1−p)et1−(1−p)etpet​

Poisson(λ)(λ)

exp⁡(λ(et−1))exp(λ(et−1))

Uniform continuous (a,b)(a,b)

etb−etat(b−a)t(b−a)etb−eta​

Exponential(λ)(λ)

λλ−tλ−tλ​

Normal(μ,σ2)(μ,σ2)

exp⁡(μt+12σ2t2)exp(μt+21​σ2t2)

Standard normal

exp⁡(t22)exp(2t2​)