Probability and Statistics: Discrete Probability Distributions

Binomial and Multinomial Distributions

• An experiment often consists of repeated trials.

• Each trial has two possible outcomes: success or failure.

• We can define either outcome as a success.

• This process is called a Bernoulli process.

• Each trial is called a Bernoulli trial.

Binomial Distribution

• The number $X$ of successes in $n$ Bernoulli trials is a binomial random variable.

• The probability distribution of this discrete random variable is the binomial distribution, denoted by $b(x; n, p)$, dependent on the number of trials and the probability of success on a given trial.

• Defined as the probability of $x$ successes in $n$ independent trials.

• A Bernoulli trial can result in a success with probability $p$ and a failure with probability $q = 1 - p$.

• The probability distribution of the binomial random variable $X$ is:
  $b(x; n, p) = \binom{n}{x} p^x q^{n-x}, \quad x = 0, 1, 2, …, n$

The Bernoulli Process

• Consider selecting three items at random from a manufacturing process, inspecting them, and classifying them as defective (success) or nondefective.

• The number of successes (defective items) is a random variable $X$ with values from 0 to 3.

Example

• If the process produces 25% defectives, then:
  $P(NDN) = P(N)P(D)P(N) = (\frac{3}{4})(\frac{1}{4})(\frac{3}{4}) = \frac{9}{64}$

• The probability distribution of $X$ is:

Example: Binomial Distribution

• The probability that a component survives a shock test is $\frac{3}{4}$.

• Find the probability that exactly 2 of the next 4 components tested survive.

• Solution:
  $b(2; 4, \frac{3}{4}) = \binom{4}{2} (\frac{3}{4})^2 (\frac{1}{4})^2 = 6 (\frac{9}{16}) (\frac{1}{16}) = \frac{54}{256} = \frac{27}{128}$

Where Does the Name Binomial Come From?

• The binomial distribution derives its name from the binomial expansion of $(q + p)^n$, where the $n + 1$ terms correspond to the values of $b(x; n, p)$ for $x = 0, 1, 2, …, n$.

• $(q + p)^n = \binom{n}{0} q^n + \binom{n}{1} p q^{n-1} + \binom{n}{2} p^2 q^{n-2} + … + \binom{n}{n} p^n$

• $(q + p)^n = b(0; n, p) + b(1; n, p) + b(2; n, p) + … + b(n; n, p)$

• Since $p + q = 1$:
  $\sum_{x=0}^{n} b(x; n, p) = 1$

• Binomial sums $B(r; n, p) = \sum_{x=0}^{r} b(x; n, p)$ are often used to find P(X < r) or $P(a \leq X \leq b)$.

Example 1

• The probability that a patient recovers from a rare blood disease is 0.4. If 15 people are known to have contracted this disease, what is the probability that:

  • (a) at least 10 survive

  • (b) from 3 to 8 survive

  • (c) exactly 5 survive?

• Solution: Let $X$ be the number of people who survive.

  • (a) P(X \geq 10) = 1 - P(X < 10) = 1 - \sum_{x=0}^{9} b(x; 15, 0.4) = 1 - 0.9662 = 0.0338

  • (b) $P(3 \leq X \leq 8) = \sum<em>{x=3}^{8} b(x; 15, 0.4) = \sum</em>{x=0}^{8} b(x; 15, 0.4) - \sum_{x=0}^{2} b(x; 15, 0.4) = 0.9050 - 0.0271 = 0.8779$

  • (c) $P(X = 5) = b(5; 15, 0.4) = \sum<em>{x=0}^{5} b(x; 15, 0.4) - \sum</em>{x=0}^{4} b(x; 15, 0.4) = 0.4032 - 0.2173 = 0.1859$

Example 2

• A retailer purchases electronic devices with a 3% defective rate.

  • (a) If the inspector picks 20 items, what is the probability of at least one defective item?

  • (b) If the retailer receives 10 shipments and tests 20 devices per shipment, what is the probability of exactly 3 shipments containing at least one defective device?

• Solution:

  • (a) Let $X$ be the number of defective devices among 20. $X \sim b(x; 20, 0.03)$. Therefore,
    $P(X \geq 1) = 1 - P(X = 0) = 1 - b(0; 20, 0.03) = 1 - (0.03)^0 (1 - 0.03)^{20-0} = 0.4562$

  • (b) Let $Y$ be the number of shipments containing at least one defective item, so $Y$ follows a binomial distribution $b(y; 10, 0.4562)$. Therefore,
    $P(Y = 3) = \binom{10}{3} (0.4562)^3 (1 - 0.4562)^7 = 0.1602$

Areas of Application

Theorem 5.1

• The mean and variance of the binomial distribution $b(x; n, p)$ are: $\mu = np$ and $\sigma^2 = npq$

Example 1

• A rural community conjectures that 30% of drinking wells have an impurity. 10 wells are randomly selected for testing.

  • (a) What is the probability that exactly 3 wells have the impurity, assuming the conjecture is correct?

  • (b) What is the probability that more than 3 wells are impure?

• Solution:

  • (a) $b(3; 10, 0.3) = \sum<em>{x=0}^{3} b(x; 10, 0.3) - \sum</em>{x=0}^{2} b(x; 10, 0.3) = 0.6496 - 0.3828 = 0.2668$

  • (b) P(X > 3) = 1 - 0.6496 = 0.3504

Example 2

• Using the previous example where $n = 15$ and $p = 0.4$, find the mean and variance, and interpret the interval $\mu \pm 2\sigma$ using Chebyshev's theorem.

• Solution:
  $\mu = (15)(0.4) = 6$
  $\sigma^2 = (15)(0.4)(0.6) = 3.6$
  $\sigma = \sqrt{3.6} = 1.897$

• The interval is $6 \pm 2(1.897)$, or from 2.206 to 9.794.

• By Chebyshev's theorem, the number of recoveries among 15 patients has a probability of at least $\frac{3}{4}$ of falling between 2 and 10 inclusive.

Multinomial Distribution

• If a trial can result in $k$ outcomes $E<em>1, E</em>2, …, E<em>k$ with probabilities $p</em>1, p<em>2, …, p</em>k$, then the probability distribution of the random variables $X<em>1, X</em>2, …, X<em>k$, representing the number of occurrences for $E</em>1, E<em>2, …, E</em>k$ in $n$ independent trials, is:

  $f(x<em>1, x</em>2, …, x<em>k; p</em>1, p<em>2, …, p</em>k, n) = \binom{n}{x<em>1, x</em>2, …, x<em>k} p</em>1^{x<em>1} p</em>2^{x<em>2} … p</em>k^{x<em>k}$ where $\sum</em>{i=1}^{k} x<em>i = n \quad \text{and} \quad \sum</em>{i=1}^{k} p_i = 1$

Example

• An airport with three runways has the following probabilities for runway access by a randomly arriving jet: Runway 1: $p<em>1 = \frac{2}{9}$, Runway 2: $p</em>2 = \frac{1}{6}$, Runway 3: $p_3 = \frac{11}{18}$.

• What is the probability that 6 randomly arriving airplanes are distributed as follows: 2 on Runway 1, 1 on Runway 2, and 3 on Runway 3?

• Solution:

  $f(2, 1, 3; \frac{2}{9}, \frac{1}{6}, \frac{11}{18}, 6) = \binom{6}{2, 1, 3} (\frac{2}{9})^2 (\frac{1}{6})^1 (\frac{11}{18})^3 = \frac{6!}{2!1!3!} (\frac{2}{9})^2 (\frac{1}{6}) (\frac{11}{18})^3 \approx 0.1127$

Hypergeometric Distribution

Areas of Application

• Applications for the hypergeometric distribution are found in acceptance sampling, electronic testing, and quality assurance.

• Deals with the probability of selecting $x$ successes from $k$ items labeled as successes and $n - x$ failures from $N - k$ items labeled as failures when a sample of size $n$ is selected from $N$ items.

Hypergeometric Experiment Properties

1. A random sample of size $n$ is selected without replacement from $N$ items.

2. Of the $N$ items, $k$ are classified as successes and $N - k$ are classified as failures.

Hypergeometric Random Variable

• The number $X$ of successes is called a hypergeometric random variable.

• The probability distribution is denoted by $h(x; N, n, k)$.

Hypergeometric Distribution in Acceptance Sampling

• Similar to the binomial distribution, it's used in acceptance sampling to determine whether to accept a lot of materials or parts.

Example

• A part is sold in lots of 10. A lot is acceptable if it has no more than one defective. A sampling plan tests 3 parts out of 10, and the lot is accepted if none are defective.

• Comment on the utility of this plan.

• Solution: Suppose the lot has 2 defective parts. The probability of accepting the lot is:
  $P(X = 0) = \frac{\binom{2}{0} \binom{8}{3}}{\binom{10}{3}} = \frac{1 \cdot 56}{120} = 0.467$

• Thus, the plan allows acceptance of an unacceptable lot (with 2 defectives) about 47% of the time, so it is considered faulty.

Hypergeometric Distribution

• The probability distribution of the hypergeometric random variable $X$ (number of successes in a sample of size $n$ from $N$ items, where $k$ are successes and $N - k$ are failures) is:

  $h(x; N, n, k) = \frac{\binom{k}{x} \binom{N-k}{n-x}}{\binom{N}{n}}$

• with $\max{0, n - (N - k)} \leq x \leq \min{n, k}$

Example

• Lots of 40 components are unacceptable if they contain 3 or more defectives. The sampling procedure is to select 5 components and reject the lot if a defective is found.

• What is the probability of finding exactly 1 defective in the sample if there are 3 defectives in the lot?

• Solution:
  $h(1; 40, 5, 3) = \frac{\binom{3}{1} \binom{37}{4}}{\binom{40}{5}} = \frac{3 \cdot 66045}{658008} \approx 0.3011$

• This plan detects a bad lot (3 defectives) only about 30% of the time, so it is not desirable.

Theorem 5.2

• The mean and variance of the hypergeometric distribution $h(x; N, n, k)$ are:

$\mu = \frac{nk}{N}$

$\sigma^2 = \frac{N-n}{N-1} \cdot n \cdot \frac{k}{N} \cdot (1 - \frac{k}{N})$

Example

• Find the mean and variance for the previous example (N = 40, n = 5, k = 3), and interpret the interval $\mu \pm 2\sigma$ using Chebyshev's theorem.

• Solution:

$\mu = \frac{(5)(3)}{40} = \frac{15}{40} = \frac{3}{8} = 0.375$

$\sigma^2 = \frac{40-5}{40-1} (5) (\frac{3}{40}) (1 - \frac{3}{40}) = \frac{35}{39} (5) (\frac{3}{40}) (\frac{37}{40}) \approx 0.3113$

$\sigma = \sqrt{0.3113} \approx 0.558$

• The interval is $0.375 \pm 2(0.558)$, or from -0.741 to 1.491.

• By Chebyshev's theorem, the number of defectives obtained has a probability of at least $\frac{3}{4}$ of falling between -0.741 and 1.491. Therefore, at least three-fourths of the time, the 5 components include fewer than 2 defectives.

Multivariate Hypergeometric Distribution

• If $N$ items can be partitioned into $k$ cells $A<em>1, A</em>2, …, A<em>k$ with $a</em>1, a<em>2, …, a</em>k$ elements, respectively, then the probability distribution of the random variables $X<em>1, X</em>2, …, X<em>k$ representing the number of elements selected from $A</em>1, A<em>2, …, A</em>k$ in a random sample of size $n$, is:

  $f(x<em>1, x</em>2, …, x<em>k; a</em>1, a<em>2, …, a</em>k, N, n) = \frac{\binom{a<em>1}{x</em>1} \binom{a<em>2}{x</em>2} … \binom{a<em>k}{x</em>k}}{\binom{N}{n}}$
  where

$\sum<em>{i=1}^{k} x</em>i = n$
and

$\sum<em>{i=1}^{k} a</em>i = N$

Example

• A group of 10 individuals is used for a biological case study: 3 with blood type O, 4 with blood type A, and 3 with blood type B.

• What is the probability that a random sample of 5 will contain 1 person with blood type O, 2 people with blood type A, and 2 people with blood type B?

• Solution:

$f(1, 2, 2; 3, 4, 3, 10, 5) = \frac{\binom{3}{1} \binom{4}{2} \binom{3}{2}}{\binom{10}{5}} = \frac{3 \cdot 6 \cdot 3}{252} = \frac{54}{252} \approx 0.2143$

Negative Binomial and Geometric Distributions

• Instead of the probability of $x$ successes in $n$ trials, where $n$ is fixed, we are now interested in the probability that the $k^{th}$ success occurs on the $x^{th}$ trial.

• Experiments of this kind are called negative binomial experiments.

What is the Negative Binomial Random Variable?

• The number $X$ of trials required to produce $k$ successes in a negative binomial experiment is called a negative binomial random variable.

• Its probability distribution is called the negative binomial distribution and denoted as $b*(x; k, p)$.

Negative Binomial Distribution

• If repeated independent trials can result in a success with probability $p$ and a failure with probability $q = 1 - p$, then the probability distribution of the random variable $X$, the number of the trial on which the $k^{th}$ success occurs, is:

$b*(x; k, p) = \binom{x-1}{k-1} p^k q^{x-k}, \quad x = k, k+1, k+2, …$

Example

• In an NBA championship series (best of 7), teams A and B face each other. Team A has a probability of 0.55 of winning a game.

  • (a) What is the probability that team A will win the series in 6 games?

  • (b) What is the probability that team A will win the series?

  • (c) If teams A and B were facing each other in a regional playoff series that is decided by winning three out of five games, what is the probability that team A would win the series?

• Solution:

  • (a) $b*(6; 4, 0.55) = \binom{6-1}{4-1} (0.55)^4 (1 - 0.55)^{6-4} = \binom{5}{3} (0.55)^4 (0.45)^2 \approx 0.1853$

  • (b)
    $P(\text{team A wins}) = b<em>(4; 4, 0.55) + b</em>(5; 4, 0.55) + b<em>(6; 4, 0.55) + b</em>(7; 4, 0.55)$
    $= 0.0915 + 0.1647 + 0.1853 + 0.1668 \approx 0.6083$

  • (c)
    $P(\text{team A wins}) = b<em>(3; 3, 0.55) + b</em>(4; 3, 0.55) + b*(5; 3, 0.55)$
    $= 0.1664 + 0.2246 + 0.2021 \approx 0.5931$

Geometric Distribution

• If repeated independent trials can result in a success with probability $p$ and a failure with probability $q = 1 - p$, then the probability distribution of the random variable $X$, the number of the trial on which the first success occurs, is:
  $g(x; p) = pq^{x-1}, \quad x = 1, 2, 3, …$

Example 1

• For a manufacturing process, 1 in every 100 items is defective. What is the probability that the fifth item inspected is the first defective item found?

• Solution:
  $g(5; 0.01) = (0.01) (0.99)^{5-1} = (0.01) (0.99)^4 \approx 0.0096$

Example 2

• At a busy time, the probability of a telephone connection is $p = 0.05$. What is the probability that 5 attempts are necessary for a successful call?

• Solution:

  $P(X = x) = g(5; 0.05) = (0.05) (0.95)^4 \approx 0.041$

Theorem 5.3

• The mean and variance of a random variable following the geometric distribution are:
  $\mu = \frac{1}{p}$
  $\sigma^2 = \frac{1-p}{p^2}$

Poisson Distribution and the Poisson Process

• Experiments yielding numerical values of a random variable $X$, the number of outcomes occurring during a given time interval or in a specified region, are called Poisson experiments.

• A Poisson experiment is derived from the Poisson process and possesses the following properties.

Properties of the Poisson Process

1. The number of outcomes occurring in one time interval or specified region of space is independent of the number that occurs in any other disjoint time interval or region (no memory).

2. The probability that a single outcome will occur during a very short time interval or in a small region is proportional to the length of the time interval or the size of the region.

3. The probability that more than one outcome will occur in a short time interval or small region is negligible.

Poisson Random Variable

• The number $X$ of outcomes occurring during a Poisson experiment is called a Poisson random variable, and its probability distribution is called the Poisson distribution.
  $\mu = \lambda t$

• $\lambda$ is the rate of occurrence of outcomes, denoted by $p(x; \lambda t)$.

Poisson Distribution

• The probability distribution of the Poisson random variable $X$, representing the number of outcomes occurring in a given time interval or specified region denoted by $t$, is:

  $p(x; \lambda t) = \frac{e^{-\lambda t} (\lambda t)^x}{x!}, \quad x = 0, 1, 2, …$

• $\lambda$ is the average number of outcomes per unit time, distance, area, or volume, and $e = 2.71828…$

Approximation of Binomial Distribution by a Poisson Distribution

• If $n$ is large and $p$ is close to 0, the Poisson distribution can be used, with $\mu = np$, to approximate binomial probabilities.

• If $p$ is close to 1, interchange success and failure to change $p$ to a value close to 0.

Theorem 5.4

• Both the mean and the variance of the Poisson distribution $p(x; \lambda t)$ are $\lambda t$

Nature of the Poisson Probability Function

• Plots of the probability function for $\mu = 0.1$, $\mu = 2$, and $\mu = 5$ show the distribution for different means.

Approximation of Binomial Distribution

• If $n$ is quite large and $p$ is small, the conditions simulate the continuous space or time implications of the Poisson process.

• The independence among Bernoulli trials is consistent with the Poisson process.

Theorem 5.5

• Let $X$ be a binomial random variable with probability distribution $b(x; n, p)$. When $n \rightarrow \infty$, $p \rightarrow 0$, and $np \rightarrow \mu$ remains constant,
  $b(x; n, p) \approx p(x; \mu)$

Example 1

• In an industrial facility, the probability of an accident on any given day is 0.005, and accidents are independent.

  • (a) What is the probability that in any given period of 400 days there will be an accident on one day?

  • (b) What is the probability that there are at most three days with an accident?

• Solution:
  Let $X$ be a binomial random variable with $n = 400$ and $p = 0.005$. Thus, $\mu = np = 2$.

  • (a) $P(X = 1) = \frac{e^{-2} 2^1}{1!} \approx 0.271$

  • (b) $P(X \leq 3) = \sum_{x=0}^{3} \frac{e^{-2} 2^x}{x!} \approx 0.857$

Example 2

• In a manufacturing process where glass products are made, defects or bubbles occur, and on average, 1 in every 1000 has one or more bubbles.

• What is the probability that a random sample of 8000 will yield fewer than 7 items possessing bubbles?

• Solution: $n = 8000$ and $p = 0.001$. Approximate using the Poisson distribution with $\mu = (8000) (0.001) = 8$.

  P(X < 7) = \sum{x=0}^{6} b(x; 8000, 0.001) \approx \sum{x=0}^{6} p(x; 8) \approx 0.3134