Statistics 1 for Economics: Discrete Probability Distributions

Overview of Discrete Probability Distributions

Discrete probability distributions are foundational tools in statistics for modeling scenarios where outcomes are countable. The primary distributions covered include:

• Bernoulli Distribution: Models a single trial with two possible outcomes.
• Binomial Distribution: Models a sequence of independent Bernoulli trials (drawing with replacement).
• Hypergeometric Distribution: Models trials where the probability of success changes because items are not replaced (drawing without replacement).
• Poisson Distribution: Models the number of occurrences of an event over a specific interval of time or space.

Key relationships and approximations involve:

• Hypergeometric → Binomial: Applicable when the sample size is small relative to the population ($n/N < 5\%$ ).
• Binomial → Poisson: Applicable when the probability of success is very low ($p < 5\%$ ).

The Hypergeometric Distribution

The Hypergeometric distribution is used when drawing samples from a finite population without replacement. This means the trials are dependent, and the probability of success changes after each draw.

Mathematical Formulation

The probability of obtaining exactly $x$ successes in a sample of size $n$ from a population of size $N$ containing $k$ successes is given by:

$P(X = x) = \frac{\binom{k}{x} \binom{N-k}{n-x}}{\binom{N}{n}}$

Parameters and Variables

• $N$: Population size.
• $n$: Sample size.
• $k$: Total number of successes in the population.
• $x$: Number of successes in the sample.
• $N-k$: Total number of failures in the population.
• $n-x$: Number of failures in the sample.

Mean and Variance

• Expected Value (Mean): $E[X] = n \times \frac{k}{N} = n \times p$
• Variance: $V[X] = n \times p(1 - p) \times \frac{N - n}{N - 1}$
  • Note: The term $\frac{N - n}{N - 1}$ is known as the finite population correction factor.

Practical Example: Stock Index

A stock index consists of $500$ stocks ($N = 500$). On a specific day, $400$ stocks increased in value ($k = 400$). A professor selects $4$ stocks at random ($n = 4$).

a. Probability that exactly 3 increased in value:$P(X = 3) = \frac{\binom{400}{3} \binom{100}{1}}{\binom{500}{4}} = 0.411$

b. Probability that at least 3 increased in value:$P(X \ge 3) = P(X = 3) + P(X = 4)$$P(X \ge 3) = \frac{\binom{400}{3} \binom{100}{1}}{\binom{500}{4}} + \frac{\binom{400}{4} \binom{100}{0}}{\binom{500}{4}} = 0.411 + 0.409 = 0.820$

The Bernoulli Distribution

The Bernoulli distribution is the simplest discrete distribution, representing a single experiment with only two outcomes: success or failure.

Characteristics

• Defined for a single trial ($n = 1$).
• $X = 1$ if success occurs (with probability $p$ ).
• $X = 0$ if failure occurs (with probability $1 - p$ ).

Mathematical Formulation

$P(X = x) = \begin{cases} p & \text{if } x = 1 \\ 1 - p & \text{if } x = 0 \end{cases}$

Moments

• Mean: $E[X] = (1 - p) \times 0 + p \times 1 = p$
• Variance: $V[X] = E[X^2] - (E[X])^2 = (1 - p) \times (0 - p)^2 + p \times (1 - p)^2 = p(1 - p)$

The Binomial Distribution

The Binomial distribution describes the number of successes in a fixed number of independent Bernoulli trials where the probability of success remains constant (drawing with replacement).

Mathematical Formulation

$P(X = x) = \binom{n}{x} p^x (1 - p)^{n-x}$

Parameters

• $n$: Number of trials (sample size).
• $p$: Probability of success in each trial ($p = k/N$ ).
• $x$: Number of successes ($x = 0, 1, 2, \dots, n$ ).

Mean and Variance

A Binomial random variable $X$ is the sum of $n$ independent Bernoulli random variables ($X = X_1 + X_2 + \dots + X_n$ ).

• Mean: $E[X] = n \times p$
• Variance: $V[X] = n \times p(1 - p)$

Example Comparison: Hypergeometric vs. Binomial

Consider a group of $20$ clients, $6$ of whom are in debt. If you pick $5$ clients:

• Hypergeometric (Without replacement): $P(X = 1) = \frac{\binom{6}{1} \binom{14}{4}}{\binom{20}{5}} = 0.387$
• Binomial (With replacement): $P(X = 1) = \binom{5}{1} (\frac{6}{20})^1 (\frac{14}{20})^4 = 0.360$

Approximations Between Distributions

Approximating Hypergeometric by Binomial

This approximation is used when the population size $N$ is much larger than the sample size $n$, making the change in probability after each draw negligible.

• Condition: $\frac{n}{N} < 0.05$ (sample is less than $5\%$ of population).
• Calculation: Use Binomial formula with $p = \frac{k}{N}$.
• Illustration: In a group of $200$ clients with $60$ in debt ($N = 200, k = 60, n = 5$ ), $\frac{n}{N} = \frac{5}{200} = 0.025$. The approximation holds. The Hypergeometric result ($0.363$ ) is very close to the Binomial result ($0.360$ ).

Approximating Binomial by Poisson

This approximation is used for "rare events" where the number of trials is large and the probability of success is very small.

• Condition: $p < 0.05$ (success occurs sporadically).
• Calculation: Set $\mu = n \times p$ and use the Poisson formula.
• Example (Sunglasses): A shipment of sunglasses has a $2\%$ defect rate ($p = 0.02$ ). For a sample of $50$ ($n = 50$ ):
  • Exact Binomial: $P(X \le 2) = 0.9216$
  • Poisson Approximation ($\mu = 50 \times 0.02 = 1$ ): $P(Y \le 2) = 0.9197$

The Poisson Distribution

The Poisson distribution models the number of successes as a "number per interval" (time, space, volume, etc.).

Mathematical Formulation

$P(X = x) = \frac{e^{-\mu} \mu^x}{x!}$

Parameters

• $\mu$: Mean number of successes per interval.
• $x$: Actual number of successes observed ($x = 0, 1, 2, \dots$ ).

Assumptions

1. Independence: The number of successes in one interval is independent of successes in other intervals.
2. Constant Rate: The probability distribution is the same for all intervals of equal size.
3. No Simultaneity: As intervals become increasingly smaller, the probability of two or more successes occurring simultaneously goes to zero.

Properties

• Mean: $E[X] = \mu$
• Variance: $V[X] = \mu$

Examples of Poisson Variables

• Number of insurance claims per year.
• Number of clients arriving in a $20$-minute window.
• Number of flaws per square meter of fabric.
• Number of meteorites hitting Earth in a year.

Problem: Defective Products in Boxes

On average, there are $6$ defective products per box ($\mu = 6$ ).

• Probability of at most 3 defectives:$P(X \le 3) = P(X=0) + P(X=1) + P(X=2) + P(X=3) = 0.151$
• Probability of exactly 3 defectives:$P(X = 3) = \frac{e^{-6} 6^3}{3!} = 0.089$     Using tables: $P(X \le 3) - P(X \le 2) = 0.151 - 0.062 = 0.089$

Questions & Discussion

1. Guessing on an Exam Question: What is the probability of getting at least $60\%$ correct on a $20$-question multiple-choice exam (4 options per question) by guessing?

• Setup: Drawing with replacement ($n = 20, p = 0.25$ ). $X \sim Bin(20, 0.25)$.
• Target: $60\%$ of $20 = 12$. Find $P(X \ge 12)$.
• Calculation: $P(X \ge 12) = 1 - P(X \le 11)$. Using tables, $1 - 0.999 = 0.001$.

2. WhatsApp Messages A student receives an average of one text per hour ($\mu = 1$ ).

• a. Probability of 2 messages in 1 hour:$P(X = 2) = \frac{e^{-1} 1^2}{2!} = 0.184$
• b. Probability of 6 messages in 4 hours:     New interval implies new $\mu_{new} = 1 \times 4 = 4$.     $P(Y = 6) = \frac{e^{-4} 4^6}{6!} = 0.104$
• c. Probability of at least 18 messages in 12 hours:$\mu_{new} = 1 \times 12 = 12$.     $P(Z \ge 18) = 1 - P(Z \le 17)$. From tables ($\mu = 12$ ), $1 - 0.937 = 0.063$.

3. Marbles Example A box contains $5$ blue and $5$ green marbles. Draw a sample of size $3$. Find $P(\text{no green})$.

• Without Replacement (Hypergeometric):$P(X = 0) = \frac{\binom{5}{3} \binom{5}{0}}{\binom{10}{3}} = 0.083$
• With Replacement (Binomial):$p = \frac{5}{10} = 0.5$.     $P(Y = 0) = \binom{3}{0} (0.5)^0 (0.5)^3 = 0.125$

4. Hospital Biopsies A hospital performs $4000$ biopsies per year with a complication probability of $0.001$. Find the probability of at least $4$ complications.

• Setup: $X \sim Bin(n = 4000, p = 0.001)$.
• Approximation: Since $p = 0.001 < 0.05$, use Poisson with $\mu = 4000 \times 0.001 = 4$.
• Calculation: $P(X \ge 4) \approx 1 - P(X_{Poi} \le 3) = 1 - 0.433 = 0.567$.
• Exact Binomial: $1 - [P(0) + P(1) + P(2) + P(3)] \approx 1 - 0.4334 = 0.5666$.