Z
6. Probability Distributions
6.1 Discrete Distributions
1. Discrete Uniform DistributionNotation
X∼U(a,b)X∼U(a,b)
where XX takes integer values from aa to bb with equal probability.
PMF
P(X=x)=1b−a+1P(X=x)=b−a+11
for x=a,a+1,…,bx=a,a+1,…,b.
Mean
E[X]=a+b2E[X]=2a+b
Variance
Var(X)=(b−a+1)2−112Var(X)=12(b−a+1)2−1
2. Bernoulli DistributionNotation
X∼Bernoulli(p)X∼Bernoulli(p)
A Bernoulli random variable has two outcomes:
Success: X=1X=1
Failure: X=0X=0
PMF
P(X=x)=px(1−p)1−xP(X=x)=px(1−p)1−x
for x∈{0,1}x∈{0,1}.
Mean
E[X]=pE[X]=p
Variance
Var(X)=p(1−p)Var(X)=p(1−p)
MGF
MX(t)=(1−p)+petMX(t)=(1−p)+pet
3. Geometric DistributionNotation
X∼Geo(p)X∼Geo(p)
XX is the number of trials until the first success.
PMF
P(X=x)=(1−p)x−1pP(X=x)=(1−p)x−1p
for x=1,2,3,…x=1,2,3,…
CDF
F(x)=1−(1−p)xF(x)=1−(1−p)x
Mean
E[X]=1pE[X]=p1
Variance
Var(X)=1−pp2Var(X)=p21−p
MGF
MX(t)=pet1−(1−p)etMX(t)=1−(1−p)etpet
for t<−ln(1−p)t<−ln(1−p).
4. Binomial DistributionNotation
X∼Bin(n,p)X∼Bin(n,p)
XX is the number of successes in nn independent Bernoulli trials.
PMF
P(X=k)=(nk)pk(1−p)n−kP(X=k)=(kn)pk(1−p)n−k
for k=0,1,…,nk=0,1,…,n.
Mean
E[X]=npE[X]=np
Variance
Var(X)=np(1−p)Var(X)=np(1−p)
MGF
MX(t)=[(1−p)+pet]nMX(t)=[(1−p)+pet]n
5. Negative Binomial DistributionNotation
X∼NegBin(r,p)X∼NegBin(r,p)
XX is the number of trials needed to get rr successes.
PMF
P(X=x)=(x−1r−1)pr(1−p)x−rP(X=x)=(r−1x−1)pr(1−p)x−r
for x=r,r+1,…x=r,r+1,…
Mean
E[X]=rpE[X]=pr
Variance
Var(X)=r(1−p)p2Var(X)=p2r(1−p)
MGF
MX(t)=(pet1−(1−p)et)rMX(t)=(1−(1−p)etpet)r
6. Poisson DistributionNotation
X∼Poisson(λ)X∼Poisson(λ)
Used to model the number of events occurring in a fixed interval of time or space.
PMF
P(X=k)=e−λλkk!P(X=k)=k!e−λλk
for k=0,1,2,…k=0,1,2,…
Mean
E[X]=λE[X]=λ
Variance
Var(X)=λVar(X)=λ
MGF
MX(t)=exp(λ(et−1))MX(t)=exp(λ(et−1))
Poisson Approximation to Binomial
If nn is large, pp is small, and λ=npλ=np is moderate, then:
Bin(n,p)≈Poisson(λ)Bin(n,p)≈Poisson(λ)
6.2 Continuous Distributions
1. Continuous Uniform DistributionNotation
X∼U(a,b)X∼U(a,b)
f(x)=1b−af(x)=b−a1
for a≤x≤ba≤x≤b.
CDF
F(x)=x−ab−aF(x)=b−ax−a
for a≤x≤ba≤x≤b.
Mean
E[X]=a+b2E[X]=2a+b
Variance
Var(X)=(b−a)212Var(X)=12(b−a)2
MGF
MX(t)=etb−etat(b−a)MX(t)=t(b−a)etb−eta
for t≠0t=0.
2. Exponential DistributionNotation
X∼Exp(λ)X∼Exp(λ)
Used to model waiting times between events in a Poisson process.
f(x)=λe−λxf(x)=λe−λx
for x≥0x≥0.
CDF
F(x)=1−e−λxF(x)=1−e−λx
Mean
E[X]=1λE[X]=λ1
Variance
Var(X)=1λ2Var(X)=λ21
MGF
MX(t)=λλ−tMX(t)=λ−tλ
for t<λt<λ.
Memoryless Property
P(X>s+t∣X>s)=P(X>t)P(X>s+t∣X>s)=P(X>t)
3. Normal DistributionNotation
X∼N(μ,σ2)X∼N(μ,σ2)
f(x)=12πσ2exp(−(x−μ)22σ2)f(x)=2πσ21exp(−2σ2(x−μ)2)
Mean
E[X]=μE[X]=μ
Variance
Var(X)=σ2Var(X)=σ2
MGF
MX(t)=exp(μt+12σ2t2)MX(t)=exp(μt+21σ2t2)
Standardization
If:
X∼N(μ,σ2)X∼N(μ,σ2)
then:
Z=X−μσ∼N(0,1)Z=σX−μ∼N(0,1)
4. Standard Normal DistributionNotation
Z∼N(0,1)Z∼N(0,1)
f(z)=12πe−z2/2f(z)=2π1e−z2/2
Mean
E[Z]=0E[Z]=0
Variance
Var(Z)=1Var(Z)=1
CDF
The CDF is denoted by:
Φ(z)Φ(z)
There is no elementary closed form.
MGF
MZ(t)=exp(t22)MZ(t)=exp(2t2)
6.3 Moment Generating Function
The moment generating function of XX is:
MX(t)=E[etX]MX(t)=E[etX]
Why MGF Is Useful
It helps to find moments.
The nnth moment about the origin is:
μn′=dndtnMX(t)∣t=0μn′=dtndnMX(t)t=0
Therefore:
E[X]=MX′(0)E[X]=MX′(0)
E[X2]=MX′′(0)E[X2]=MX′′(0)
and:
Var(X)=MX′′(0)−(MX′(0))2Var(X)=MX′′(0)−(MX′(0))2
MGF Summary Table
Distribution | MGF |
|---|---|
Bernoulli(p)(p) | (1−p)+pet(1−p)+pet |
Binomial(n,p)(n,p) | [(1−p)+pet]n[(1−p)+pet]n |
Geometric(p)(p) | pet1−(1−p)et1−(1−p)etpet |
Poisson(λ)(λ) | exp(λ(et−1))exp(λ(et−1)) |
Uniform continuous (a,b)(a,b) | etb−etat(b−a)t(b−a)etb−eta |
Exponential(λ)(λ) | λλ−tλ−tλ |
Normal(μ,σ2)(μ,σ2) | exp(μt+12σ2t2)exp(μt+21σ2t2) |
Standard normal | exp(t22)exp(2t2) |