The discrete uniform random variable X∼DUnif(a,b) models an experiment where outcomes take on integer values between a and b inclusive.
The support of X consists of b−a+1 consecutive integer values: {a,a+1,a+2,…,b−1,b}.
Every possible outcome within this range is equally likely.
Probability Mass Function (PMF):
The PMF assigns equal probability to every integer in the support:
pX(x)=b−a+11for x∈{a,a+1,…,b}
For any value outside this set, pX(x)=0.
Expected Value and Variance:
The expectation represents the arithmetic midpoint of the support interval:
E(X)=2a+b
The variance measures the dispersion around the mean:
Var(X)=12(b−a+1)2−1
Bernoulli and Binomial Random Variables
The Bernoulli Random Variable:
Describes a single trial with exactly two possible outcomes: Success (S) and Failure (F).
The parameter is p∈[0,1], denoting the probability of success: P(S)=p, and P(F)=1−p.
Often utilized as an indicator random variable, mapped numerically as:
x=1if Sx=0if F
The tabular probability mass function is:
pX(0)=1−p
pX(1)=p
The compact algebraic definition of the PMF for X∼Ber(p) is:
pX(x)=px(1−p)1−xwhere x∈{0,1}
Expectation and variance for a Bernoulli random variable:
E(X)=0⋅(1−p)+1⋅p=pE(X2)=02⋅(1−p)+12⋅p=pVar(X)=E(X2)−[E(X)]2=p−p2=p(1−p)
The Binomial Experiment Framework:
Consists of n identical, independent trials.
Each individual trial results in either Success (S) with probability p, or Failure (F) with probability 1−p.
The sample space Ω consists of all possible sequences of S and F of length n, containing 2n outcomes.
The Binomial random variable X counts the total number of successes in n trials: X=number of successes.
Support of X: x∈{0,1,2,…,n}.
Derivation of the Binomial PMF (n=3 Case Study):
For n=3, the sample space Ω partitions according to the number of successes X:
X=0: Outcome {FFF}. Probability is P(FFF)=(1−p)3=(03)p0(1−p)3−0.
X=1: Outcomes {SFF,FSF,FFS}. Each sequence has probability p(1−p)2. Total probability is PX(1)=3p(1−p)2=(13)p1(1−p)3−1.
X=2: Outcomes {SSF,SFS,FSS}. Each sequence has probability p2(1−p). Total probability is PX(2)=P(SSF)+P(SFS)+P(FSS)=3p2(1−p)=(23)p2(1−p)3−2.
X=3: Outcome {SSS}. Probability is P(SSS)=p3=(33)p3(1−p)3−3.
Extending to general n and x successes, there are (xn) mutually exclusive sequences, each having probability px(1−p)n−x.
General Definition of the PMF for X∼Bin(n,p):
pX(x)=(xn)px(1−p)n−xwhere x=0,1,…,n
Representation as a Sum of Independent Bernoulli Trials:
Let X1,X2,…,Xn be independent and identically distributed (i.i.d.) Bernoulli random variables such that Xi∼Ber(p).
Each Xi indicates the outcome of trial i (Xi=1 for success, Xi=0 for failure).
The Binomial random variable can be written directly as their sum:
X=X1+X2+⋯+Xn=∑i=1nXi
Tabular correspondence between Ω outcomes and Bernoulli components (n=3):
Outcome FFF⟹X1=0,X2=0,X3=0⟹X=0
Outcomes SFF,FSF,FFS⟹ each has one 1 and two 0s ⟹X=1
Outcomes SSF,SFS,FSS⟹ each has two 1s and one 0⟹X=2
Outcome SSS⟹X1=1,X2=1,X3=1⟹X=3
Moment Generating Function (MGF) of the Binomial Distribution:
The Binomial Theorem states:
∑x=0n(xn)axbn−x=(a+b)n
Derivation of the MGF ϕX(t)=E[etX]:
ϕX(t)=∑x=0netxpX(x)ϕX(t)=∑x=0netx(xn)px(1−p)n−xϕX(t)=∑x=0n(xn)(pet)x(1−p)n−x
Setting a=pet and b=1−p in the Binomial Theorem gives:
ϕX(t)=(pet+1−p)n
Evaluating at t=0 verifies standard MGF normalization:
ϕX(0)=(pe0+1−p)n=(p+1−p)n=1n=1
Derivation of Mean and Variance:
Using the MGF first derivative:
ϕX′(t)=n(pet+1−p)n−1⋅petE(X)=ϕX′(0)=n(p+1−p)n−1p=np
Using the MGF second derivative:
ϕX′′(t)=n(n−1)(pet+1−p)n−2(pet)2+n(pet+1−p)n−1petE(X2)=ϕX′′(0)=n(n−1)p2+np=n2p2−np2+np
Calculating the variance:
Var(X)=E(X2)−[E(X)]2=(n2p2−np2+np)−(np)2=np(1−p)
Alternatively, by the linear combination of i.i.d. Bernoulli variables:
E(X)=E(∑i=1nXi)=∑i=1nE(Xi)=∑i=1np=npVar(X)=Var(∑i=1nXi)=∑i=1nVar(Xi)=∑i=1np(1−p)=np(1−p)
Models the number of occurrences of an event over a specified period of time or region of space.
Governed by rate parameter λ>0, which denotes the average number of occurrences per measurement interval: λ=np.
Canonical real-world processes:
Number of phone calls received by a server per hour.
Number of laser photons hitting a detector per second.
Number of spam emails arriving in an inbox per day.
Mathematical Derivation from the Binomial PMF (Poisson Limit):
Arises as the limiting form of the Binomial distribution under the following conditions:
The number of trials grows infinitely large: n→∞
The success probability per trial becomes infinitesimally small: p→0
The product remains constant: λ=np
Derivation of the limit:
pX(x)=limn→∞(xn)px(1−p)n−x=limn→∞x!n(n−1)⋯(n−x+1)(nλ)x(1−nλ)n−xpX(x)=x!λxlimn→∞[nxn(n−1)⋯(n−x+1)(1−nλ)n(1−nλ)−x]
Since limn→∞nxn(n−1)⋯(n−x+1)=1, limn→∞(1−nλ)n=e−λ, and limn→∞(1−nλ)−x=1:
pX(x)=x!e−λλxwhere x=0,1,2,…
Poisson Approximation to the Binomial:
When n is large and p is small, the Binomial PMF is approximated as:
(xn)px(1−p)n−x≈x!e−np(np)x
Example comparison for X∼Bin(20,0.02), computing P(X=2), where n=20 and p=0.02:
a) Exact Binomial evaluation:
P(X=2)=(220)(0.02)2(1−0.02)20−2=190×(0.0004)×(0.98)18≈0.0528
b) Poisson approximation evaluation (λ=np=20×0.02=0.4):
P(X=2)≈2!e−0.4(0.4)2=2e−0.4×0.16≈0.0536
Moment Generating Function (MGF) of the Poisson Distribution:
Maclaurin series expansion of the exponential function:
∑x=0∞x!bx=eb
Derivation of ϕX(t)=E[etX]:
ϕX(t)=∑x=0∞etxpX(x)=∑x=0∞etxx!e−λλx=e−λ∑x=0∞x!(λet)x
Substituting b=λet into the Maclaurin expansion yields:
ϕX(t)=e−λeλet=exp(λ(et−1))
Expectation and Variance:
From the MGF:
ϕX′(t)=λetexp(λ(et−1))⟹E(X)=ϕX′(0)=λϕX′′(t)=λetexp(λ(et−1))+(λet)2exp(λ(et−1))⟹E(X2)=ϕX′′(0)=λ+λ2Var(X)=E(X2)−[E(X)]2=(λ+λ2)−λ2=λ
A distinctive property of the Poisson distribution is that its mean and variance are identical: E(X)=Var(X)=λ.
Application Example: Traffic Accident Rate:
Suppose accidents occur at an average rate of 14 accidents per week.
Probability of at least one accident in a week (X∼Poisson(14)):
P(X≥1)=1−P(X=0)=1−0!e−14140=1−e−14≈1−8.315×10−7≈0.999999
Probability of at least one accident today:
The daily rate is scaled linearly: λday=714=2 accidents per day.
Let Y∼Poisson(2) represent daily accidents:
P(Y≥1)=1−P(Y=0)=1−0!e−220=1−e−2≈1−0.1353=0.8647
Visual Behavior across Parameter Values:
For small rates (λ=2): The distribution is strongly right-skewed with a peak at small integer counts (x=1,2), with E(Y)=2 and Var(Y)=2.
For larger rates (λ=14): The distribution shifts rightward, becomes substantially more symmetric, and approximates a continuous Gaussian bell curve centered at E(X)=14 with Var(X)=14.
Geometric Random Variable
Definition and Experiment Framework:
Formulated upon an infinite sequence of independent, identical Bernoulli trials.
Each trial has two possible outcomes, Success (S) and Failure (F), with constant success probability P(S)=p.
The random variable X counts the total number of trials required up to and including the first occurrence of success.
Support of X: x∈{1,2,3,…}.
Derivation of the Geometric PMF:
The event X=x requires that the first x−1 trials end in failure, and trial x ends in success (x−1 timesFF…FS).
Because trials are independent, probabilities multiply directly:
pX(x)=(1−p)x−1pwhere x=1,2,3,…
Moment Generating Function (MGF) of the Geometric Distribution:
Geometric series summation formula:
∑x=1∞bx=1−bbfor ∣b∣<1
Derivation of ϕX(t)=E[etX]:
ϕX(t)=∑x=1∞etxpX(x)=∑x=1∞etx(1−p)x−1p=1−pp∑x=1∞((1−p)et)x
Using the geometric series formula with b=(1−p)et, valid when (1−p)et<1 (i.e., t<−ln(1−p)) gives:
ϕX(t)=1−pp⋅1−(1−p)et(1−p)et=1−(1−p)etpet
Mean and Variance:
The expected number of trials until the first success is the reciprocal of the success probability:
E(X)=p1
The variance is given by:
Var(X)=p21−p
Application Example: Trials Until Passing the IELTS Examination:
Let the probability of passing the IELTS exam on any given attempt be p=0.25.
Let X∼Geom(0.25) represent the attempt number on which the candidate passes.
Probability of passing on the third trial:
P(X=3)=pX(3)=(1−0.25)3−1(0.25)=(0.75)2(0.25)=0.5625×0.25=0.140625=649
Expected attempt on which the candidate passes:
E(X)=p1=0.251=4
Variance of the attempts:
Var(X)=p21−p=0.2521−0.25=0.06250.75=12
Visual Behavior across Different Probability Levels:
Geometric distributions always display a monotonic decreasing probability profile; the mode is consistently at x=1.
Comparison for varied p values:
p=0.25⟹E(X)=4, Var(X)=12. The PMF starts at pX(1)=0.25 and has a long tail.
p=0.50⟹E(X)=2, Var(X)=2. The PMF starts at pX(1)=0.50 and halves at every step.
p=0.75⟹E(X)≈1.33, Var(X)≈0.44. The PMF starts at pX(1)=0.75 and decays rapidly.
Hypergeometric Random Variable
Sampling Without Replacement Framework:
Models a finite population of size N+M containing:
N items classified as Successes (e.g., Red balls).
M items classified as Failures (e.g., Black balls).
A sample of n items is drawn completely at random without replacement.
The random variable X represents the total number of successes (Red balls) selected in the sample.
Derivation of the Hypergeometric PMF:
Total number of equally likely ways to select n items from a population of N+M items is (nN+M).
The number of ways to choose x successes from N available successes is (xN).
The remaining n−x items in the sample must be failures, selected from the M available failures in (n−xM) ways.
General Definition of the PMF for X∼Hypergeom(N,M,n):
pX(x)=(nN+M)(xN)(n−xM)for x=max(0,n−M),…,min(n,N)
Parameters: N=4, M=6, total items N+M=10, draw size n=3.
Support of X: x∈{0,1,2,3}.
Total outcomes in sample space: (310)=3×2×110×9×8=120.
Computation for each outcome:
x=0 (No red balls, 3 black balls):
pX(0)=(310)(04)(36)=1201×20=12020=61
x=1 (1 red ball, 2 black balls):
pX(1)=(310)(14)(26)=1204×15=12060=21
x=2 (2 red balls, 1 black ball):
pX(2)=(310)(24)(16)=1206×6=12036=103
x=3 (3 red balls, 0 black balls):
pX(3)=(310)(34)(06)=1204×1=1204=301
Sum check: 12020+12060+12036+1204=120120=1.
Moment Generating Function, Mean, and Variance:
The MGF ϕX(t) of the Hypergeometric distribution is mathematically not tractable in closed analytic form.
Expected value:
E(X)=n(N+MN)
Defining p=N+MN as the initial proportion of successes in the population, the expectation matches the Binomial expectation exactly: E(X)=np.
Variance:
Var(X)=n(N+MN)(N+MM)[1−N+M−1n−1]
Using the algebraic identity 1−N+M−1n−1=N+M−1(N+M−1)−(n−1)=N+M−1N+M−n, the variance is expressed as:
Var(X)=np(1−p)(N+M−1N+M−n)
The factor N+M−1N+M−n is the finite population correction factor, reflecting the reduction in variance caused by sampling without replacement compared to independent trials.