Special Parametric Families of Univariate Distributions

Parametric family of density functions

A collection of density functions that is indexed by a quantity called a parameter.

Discrete uniform distribution

Each member of the family of probability mass function p(x; N) = 1/N I{1,2,...,N}(x) where the parameter N ranges over the positive integers.

Discrete uniform random variable

A random variable X having the probability mass function of a discrete uniform distribution.

Mean of discrete uniform distribution

E[X] = (N + 1) / 2.

Variance of discrete uniform distribution

Var(X) = (N^2 - 1) / 12.

Moment Generating Function for discrete uniform distribution

mX(t) = (1/N) Σ (e^(jt)) from j=1 to N.

Bernoulli distribution

A random variable X is defined to have a Bernoulli distribution if the pmf of X is given by pX(x; p) = p^x(1 − p)^(1−x) I{0,1}(x), where 0 ≤ p ≤ 1.

Mean of Bernoulli distribution

E[X] = p.

Variance of Bernoulli distribution

Var(X) = pq.

Moment Generating Function for Bernoulli distribution

mX(t) = pe^t + q.

Binomial distribution

A random variable X is defined to have a binomial distribution if the pmf of X is given by pX(x; n, p) = (n choose x) p^x q^(n−x) I{0,1,2,...,n}(x).

Mean of binomial distribution

E[X] = np.

Variance of binomial distribution

Var(X) = npq.

Moment Generating Function for binomial distribution

mX(t) = (q + pe^t)^n.

Bernoulli trial

A random experiment whose outcomes have been classified into categories called 'success' and 'failure.'

Point binomial

Another name for the Bernoulli distribution when n = 1.

Independent Bernoulli trials

Trials where the probability of success remains the same from trial to trial.

Sample space for Bernoulli trials

Ω= {(z1, z2, . . . , zn) : zi = s or zi = f}.

Probability of specified outcome in Bernoulli trials

Given by qqpqpp · · · qp.

Probability mass function (pmf)

A function that gives the probability that a discrete random variable is equal to a specific value.

Parameter in distributions

A quantity that indexes a family of density functions.

Real-world phenomena modeling

Applying specific parametric distributions to model situations like waiting times, population counts, or reliability measures.

Hypergeometric distribution

A random variable X is defined to have a hypergeometric distribution if the pmf of X is given by fX(x; M, K, n) = (K choose x)(M−K choose n−x)/(M choose n) I{0,1,...,n}(x) where M is a positive integer, K is a nonnegative integer that is at most M, and n is a positive integer that is at most M.

E[X] for Hypergeometric distribution

If X is a hypergeometric distribution, then E[X] = n · K/M.

Var[X] for Hypergeometric distribution

If X is a hypergeometric distribution, then Var[X] = n · K/M · (M − n)/(M · (M − K)/(M − 1)).

Poisson distribution

A random variable X is defined to have a Poisson distribution if the pmf of X is given by pX(x; λ) = e−λ(λ^x)/x! I{0,1,2,...}(x) where the parameter λ satisfies λ > 0.

E[X] for Poisson distribution

If X is a Poisson distributed random variable, then E[X] = λ.

Var[X] for Poisson distribution

If X is a Poisson distributed random variable, then Var[X] = λ.

Moment generating function for Poisson distribution

If X is a Poisson distributed random variable, then mX(t) = e^(λ(e^t−1)).

Mean rate of occurrence

The mean rate of occurrence v can be interpreted as the mean rate at which happenings occur per unit of time.

Geometric distribution

A random variable X is defined to have geometric (or Pascal) distribution if the density of X is given by pX(x; p) = p(1 − p)^x I{0,1,2,...}(x) where the parameter p satisfies 0 < p ≤ 1.

Hypergeometric distribution example

Let X denote the number of defectives in a sample of size n when sampling is done without replacement from an urn containing M balls, K of which are defective. Then X has a hypergeometric distribution.

Poisson distribution example

The Poisson distribution provides a realistic model for many random phenomena, such as the number of fatal traffic accidents per week in a given state.

Probability of no calls in 3-minute period

P[no calls in 3-minute period] = e^(−vt) = e^(−(0.5)(3)) = e^(−1.5) ≈ 0.223.

Probability of more than five calls in 5-minute interval

P[more than five calls in 5-minute interval] = Σ (from k=6 to ∞) (e^(−2.5)(2.5^k)/k!).

Mean of hypergeometric distribution

If we let K/M = p, then the mean of the hypergeometric distribution coincides with the mean of the binomial distribution.

Variance of hypergeometric distribution

The variance of the hypergeometric distribution is (M − n)/(M − 1) times the variance of the binomial distribution.

Poisson density

The density given above is called a Poisson density.

Hypergeometric density example

Figure 3.3: Hypergeometric density with M = 20, K = 10 and n = 8.

Poisson density example

Figure 3.4: Poisson density with λ = 4.

Conditions for Poisson distribution

The conditions for a Poisson distribution include: (i) The probability that exactly one happening will occur in a small time interval of length h is approximately equal to vh. (ii) The probability of more than one happening in a small time interval of length h is negligible compared to the probability of just one happening. (iii) The numbers of happenings in nonoverlapping time intervals are independent.

Parameter λ in Poisson distribution

The parameter λ in a Poisson distribution is equal to vt, where v is the mean rate of occurrence.

Geometric Distribution

If the random variable X has a geometric distribution, then E[X] = q/p, Var(X) = q/p^2, and mX(t) = p/(1 − q e^t).

Negative Binomial Distribution

A random variable X with density pX(x; r, p) = (r + x − 1 choose x) p^r q^x I{0,1,2,...}(x) where the parameters r and p satisfy r = 1, 2, 3, ... and 0 < p ≤ 1.

Negative Binomial Expectation

If X has a negative binomial distribution, then E[X] = rq/p.

Negative Binomial Variance

If X has a negative binomial distribution, then Var(X) = rq/p^2.

Negative Binomial Moment Generating Function

If X has a negative binomial distribution, then mX(t) = (p)^r / (1 − q e^t).

Geometric Density

The random variable X represents the number of trials required before the first success; then X has the geometric density.

Uniform Distribution

If the probability density function of a random variable X is given by fX(x; a, b) = 1/(b − a) I[a,b](x) where the parameters a and b satisfy −∞ < a < b < ∞, then the random variable X is defined to be uniformly distributed over the interval [a, b].

Uniform Expectation

If X is uniformly distributed over [a, b], then E[X] = (a + b)/2.

Uniform Variance

If X is uniformly distributed over [a, b], then Var(X) = (b − a)^2 / 12.

Uniform Moment Generating Function

If X is uniformly distributed over [a, b], then mX(t) = e^(bt) − e^(at) / ((b − a)t).

Cumulative Distribution Function (CDF) of Uniform Distribution

The cdf of the uniform random variable is given by FX(x) = (x − a)/(b − a) I[a,b](x) + I(b,∞)(x).

Normal Distribution

A random variable X is defined to be normally distributed if its density is given by fX(x; µ, σ) = 1/(√(2πσ^2)) e^−((x−µ)^2)/(2σ^2) I_R(x), where the parameters µ and σ satisfy −∞ < µ < ∞ and σ > 0.

Normal Expectation

If random variable X is normally distributed with mean µ and variance σ^2, then E[X] = µ.

Normal Variance

If random variable X is normally distributed with mean µ and variance σ^2, then Var(X) = σ^2.

Normal Moment Generating Function

If X is a normal random variable, then mX(t) = e^(µt + σ^2t^2/2).

Probability for Normal Distribution

If X ∼ N(µ, σ^2), then P[a < X < b] = Φ((b − µ)/σ) − Φ((a − µ)/σ).

Standard Normal Random Variable

If the normal random variable has mean 0 and variance 1, it is called a standard or normalized normal random variable.

Standard Normal Density

For a standard normal random variable, the density notation is φ(x) = 1/(√(2π)) e^−(x^2/2).

Standard Normal CDF

For a standard normal random variable, the cumulative distribution function notation is Φ(x) = ∫ from -∞ to x φ(t) dt.

Example of Normal Distribution

Suppose that the diameters of shafts manufactured by a certain machine are normal random variables with mean 10 centimeters and standard deviation 0.1 centimeter.

Exponential distribution

If a random variable X has density given by fX(x; λ) = λe−λx I[0,∞)(x) where λ > 0, then X is defined to have an (negative) exponential distribution.

Gamma distribution

If a random variable X has density given by fX(x; r, λ) = λ / Γ(r)(λx)^(r−1)e−λx I[0,∞)(x), where r > 0 and λ > 0, then X is defined to have a gamma distribution.

E[X] for Exponential distribution

E[X] = 1 / λ.

Var(X) for Exponential distribution

Var(X) = 1 / λ².

Moment generating function for Exponential distribution

mX(t) = λ / (λ − t) for t < λ.

E[X] for Gamma distribution

E[X] = r / λ.

Var(X) for Gamma distribution

Var(X) = r / λ².

Moment generating function for Gamma distribution

mX(t) = (λ)^(r) / (λ − t) for t < λ.

Probability density function of Exponential distribution

fX(x) = λe−λx, x ≥ 0.

Mean of Exponential distribution

Mean = 1 / λ.

Cumulative distribution function (CDF) of Exponential distribution

FX(x) = P(X ≤ x) = 1 − e−λx.

Probability that X < 3 for Exponential distribution

P(X < 3) ≈ 0.4512 or 45.12%.

Probability that X > 7 for Exponential distribution

P(X > 7) ≈ 0.2466 or 24.66%.

Median time for Exponential distribution

The median time is approximately 3.47 minutes.

Beta distribution

If a random variable X has a density given by fX(x; a, b) = 1 / B(a, b)xa−1(1 − x)b−1 I(0,1)(x), where a > 0 and b > 0, then X is defined to have a beta distribution.

Beta function

B(a, b) = ∫(0 to 1) x^(a−1)(1 − x)^(b−1) dx.

E[X] for Beta distribution

E[X] = a / (a + b).

Var(X) for Beta distribution

Var(X) = ab / ((a + b + 1)(a + b)²).

Special case of Beta distribution

The beta distribution reduces to the uniform distribution over (0, 1) if a = b = 1.

Theorem 36

If the random variable X has an exponential distribution with parameter λ, then P[X > a + b|X > a] = P[X > b].

Remark on Gamma distribution

If in the gamma density r = 1, the gamma density specializes to the exponential density.

Remark on Exponential distribution

The exponential distribution has been used as a model for lifetimes of various things.

Theorem 33

If X has an exponential distribution, then E[X] = 1 / λ, Var(X) = 1 / λ², and mX(t) = λ / (λ − t) for t < λ.

Theorem 34

If X has a gamma distribution with parameters r and λ, then E[X] = r / λ, Var(X) = r / λ², and mX(t) = (λ)^(r) for t < λ.