Bernoulli and Binomial Probability Distributions

Statistics and Probability

Bernoulli Distribution

• Named after Swiss mathematician Jacob Bernoulli (1654-1705).

• Models a single trial with two possible outcomes: success (1) or failure (0).

• A sequence of independent Bernoulli trials forms a Bernoulli process.

Properties of the Bernoulli Process:

• The experiment consists of repeated trials.

• Each trial results in an outcome classified as success or failure.

• The probability of success, denoted by $p$, remains constant from trial to trial.

• The repeated trials are independent.

Bernoulli Distribution Definition

• A Bernoulli trial is a random experiment with exactly two possible outcomes: success and failure.

• A random variable $X$ associated with such a trial, where $X = 1$ indicates success and $X = 0$ indicates failure, is called a Bernoulli random variable.

Probability Distribution

• The probability distribution of $X$, known as the Bernoulli distribution, is given by:
  $f(x) = \begin{cases} p, &amp; \text{if } x = 1 \ 1 - p, &amp; \text{if } x = 0 \end{cases}$ $0 \le p \le 1$, the probability of success.</p></li></ul><h5 id="050d752f-c777-4a1f-b7d1-5ef0e536743d" data-toc-id="050d752f-c777-4a1f-b7d1-5ef0e536743d" collapsed="false" seolevelmigrated="true">Theorem</h5><ul><li><p>The mathematical expectation and variance of the Bernoulli distribution are:<br>$E(X) = p$<br>$Var(X) = p(1 - p)$</p></li></ul><h5 id="8ceab1aa-2311-4b09-aa7c-aaf207c0a28a" data-toc-id="8ceab1aa-2311-4b09-aa7c-aaf207c0a28a" collapsed="false" seolevelmigrated="true">Examples of Bernoulli Trials:</h5><ol><li><p>Selecting a defective item from a manufacturing process with a probability of$\frac{1}{5}$. A defective item is designated as a success.</p><ul><li><p>Mean:$E(X) = \frac{1}{5}$</p></li><li><p>Variance:$Var(X) = \frac{4}{25}$</p></li></ul></li><li><p>Opening an email to check for a malicious attachment, with a probability of 0.02 of being malicious. A malicious attachment is designated as a success.</p><ul><li><p>Mean:$E(X) = 0.02$</p></li><li><p>Variance:$Var(X) = 0.0196$</p></li></ul></li><li><p>Checking if a webpage contains a specific keyword, with a probability of 0.1. The presence of the keyword is designated as a success.</p><ul><li><p>Mean:$E(X) = 0.1$</p></li><li><p>Variance:$Var(X) = 0.09$</p></li></ul></li><li><p>Detecting a fraudulent transaction among online purchases, with a fraud probability of 0.001. A fraudulent transaction is designated as a success.</p><ul><li><p>Mean:$E(X) = 0.001$</p></li><li><p>Variance:$Var(X) = 0.000999$</p></li></ul></li><li><p>Observing whether a website visitor makes a purchase, with a conversion probability of 0.07. A purchase is designated as a success.</p><ul><li><p>Mean:$E(X) = 0.07$</p></li><li><p>Variance:$Var(X) = 0.0651$</p></li></ul></li></ol><h4 id="1508ecfc-bd28-4322-95cb-5e8be263fc75" data-toc-id="1508ecfc-bd28-4322-95cb-5e8be263fc75" collapsed="false" seolevelmigrated="true">Binomial Distribution</h4><ul><li><p>Models the$x$number of successes in$n$Bernoulli trials.</p></li><li><p>Characterizes the sum of multiple independent Bernoulli trials, specifically$n$such trials.</p></li><li><p>When a Bernoulli trial is repeated$n$times, each with an identical probability of success$p$, the total number of successes, denoted by$X$, follows a binomial distribution:<br>$X = X1 + X2 + \cdots + Xn \sim \text{Binomial}(n, p)$where each$Xi \sim \text{Bernoulli}(p)$.</p></li><li><p>The binomial distribution emerges as the distribution of the sum of these independent Bernoulli random variables.</p></li></ul><h5 id="9880e732-9dd7-4f27-8b4f-09ceb9765cc7" data-toc-id="9880e732-9dd7-4f27-8b4f-09ceb9765cc7" collapsed="false" seolevelmigrated="true">Example</h5><ul><li><p>Let$X$be a random variable representing the number of defective items among three items randomly selected from a manufacturing process. Each item is inspected and classified as either defective (designated as a success) or non-defective (a failure).</p></li><li><p>The random variable$X$can take integer values from 0 to 3, corresponding to the number of defective items observed.</p></li><li><p>The eight possible outcomes and their associated values of$X$are:</p><ul><li><p>Sample Space: NNN, NND, NDN, DNN, NDD, DND, DDN, DDD</p></li><li><p>$x$: 0, 1, 1, 1, 2, 2, 2, 3</p></li></ul></li></ul><h5 id="81fc2795-f74f-40ec-8832-ca3565a4a1ae" data-toc-id="81fc2795-f74f-40ec-8832-ca3565a4a1ae" collapsed="false" seolevelmigrated="true">Definition</h5><ul><li><p>A variable described as the number of successes in a sequence of independent Bernoulli trials follows a binomial distribution.</p></li><li><p>The probability mass function of the binomial distribution is given by:</p><p>$P(X) = f(x) = b(x; n, p) = {n \choose x} p^x (1 - p)^{n-x}, \quad x = 0, 1, 2,…, n$</p></li><li><p>The binomial distribution derives its name from the fact that the$n + 1$terms in the binomial expansion of$((1 - p) + p)^n$correspond to the values of the binomial probability mass function$b(x; n, p)$for$x = 0, 1, 2,…, n$.</p><p>$((1-p)+p)^n = {n \choose 0}(1-p)^n+{n \choose 1}p(1-p)^{n-1}+{n \choose 2}p^2(1-p)^{n-2}+\cdots+{n \choose n}p^n = b(0; n, p)+b(1; n, p)+\cdots+b(n; n, p)$</p></li><li><p>Since$p + (1 - p) = 1$, it follows that$\sum_{x=0}^{n} b(x; n, p) = 1$, which confirms a fundamental property of any probability distribution: the total probability must sum to 1.</p></li></ul><h5 id="4d5f35f0-c477-4e70-948e-c6b3f7cdd1f2" data-toc-id="4d5f35f0-c477-4e70-948e-c6b3f7cdd1f2" collapsed="false" seolevelmigrated="true">Examples</h5><ol><li><p>As part of a business strategy, 20$P(X < a)$or$P(a \le X \le b)$.</p></li></ul><h5 id="fa5ffb52-8680-4192-86bd-f112d1cf3cd8" data-toc-id="fa5ffb52-8680-4192-86bd-f112d1cf3cd8" collapsed="false" seolevelmigrated="true">Examples Continued</h5><ol><li><p>As part of a business strategy, 20$E(X) = np$<br>$Var(X) = np(1 - p)$</p></li><li><p>If$X = X1 + X2 + \cdots + X_n$then,</p></li></ul><p>$E(X) = E(X1 + X2 + … + Xn) = E(X1) + E(X2) + … + E(Xn) = p + p + … + p = np$</p><p>$Var(X) = Var(X1 + X2 + … + Xn) = Var(X1) + Var(X2) + … + Var(Xn) = p(1 - p) + p(1 - p) + … + p(1 - p) = np(1 - p)$</p><h5 id="dc051dba-0912-4857-bb1b-31f7a7cd0922" data-toc-id="dc051dba-0912-4857-bb1b-31f7a7cd0922" collapsed="false" seolevelmigrated="true">Examples Continued</h5><ol><li><p>As part of a business strategy, 20$\mu \pm 2$$

More Examples

1. An exciting computer game is released. Sixty percent of players complete all the levels. Thirty percent of them will then buy an advanced version of the game. Among 15 users,

   • what is the expected number of people who will buy the advanced version?

   • what is the probability that at least two people will buy it?

References:

• Probability and Statistics for Engineers and Scientists by Walpole, 9th Edition

• Probability and Statistics for Computer Scientists by Baron, Michael, 2nd Edition