Chapter 1: Probability Theory: Sample Spaces, Events, Counting Rules, and Bayes' Theorem

Sample Space

• Observations and Statistical Experiments:

  • An observation refers to any recorded information, whether numerical (such as counts or measurements) or categorical (classified according to specific criteria).

  • Numerical observation examples: Recording monthly accident counts at the intersection of Driftwood Lane and Royal Oak Drive as $2$, $0$, $1$, and $2$.

  • Categorical observation examples: Inspecting five manufactured items and recording their quality status as nondefective ($N$) or defective ($D$): $N$, $D$, $N$, $N$, $D$.

  • An experiment is any process that generates a set of data.

  • Statistical studies—including designed experiments, observational studies, and retrospective studies—are all classified as statistical experiments because the data generated is subject to uncertainty.

  • Examples of statistical experiments include tossing a coin, launching a missile to record velocity over time, surveying voter opinions on a new sales tax, analyzing corrosion measurements, evaluating blood cholesterol and sodium levels, or evaluating historical monthly electric power consumption against ambient temperature.

• Definition of Sample Space:

  • The sample space of a statistical experiment is the set of all possible outcomes and is represented by the symbol $S$.

  • Each individual outcome in a sample space is called an element, a member, or a sample point.

  • When a sample space contains a finite number of elements, they can be listed explicitly inside braces separated by commas.

  • For a single coin flip: $S = \{H, T\}$, where $H$ represents heads and $T$ represents tails.

• Multiple Sample Spaces and Detailed Representation:

  • Depending on the objective of an investigation, an experiment can be described by more than one sample space.

  • Example 2.1: Tossing a single six-sided die.

  • If interested in the exact number showing on the top face: $S_1 = \{1, 2, 3, 4, 5, 6\}$.

  • If interested only in whether the top face is even or odd: $S_2 = \{\text{even}, \text{odd}\}$.

  • $S_1$ contains more detailed information than $S_2$. Knowing which element occurs in $S_1$ uniquely determines the outcome in $S_2$, whereas knowing the outcome in $S_2$ does not uniquely determine the element in $S_1$. The sample space providing the maximum information is generally preferred.

• Tree Diagrams:

  • Tree diagrams systematically list the sample points of multi-stage experiments.

  • Example 2.2: Flipping a coin once. If heads occurs, the coin is flipped a second time. If tails occurs on the first flip, a six-sided die is tossed once.

  • First-stage paths: $H$ or $T$.

  • Second-stage paths following $H$: $H$ or $T$.

  • Second-stage paths following $T$: 1, 2, 3, 4, 5, 6$.\n - Resulting sample space: S = {HH, HT, T1, T2, T3, T4, T5, T6}.\n\n\n\n - **Example 2.3:** Selecting three items at random from a manufacturing process and classifying each as defective (D$) or nondefective ($N).\n - Path outcomes across three successive items generate 2^3 = 8 sample points.\n - Resulting sample space: S = {DDD, DDN, DND, DNN, NDD, NDN, NND, NNN}.\n\n\n\n- **Statement or Rule Method:**\n - When a sample space contains a large or infinite number of outcomes, it is represented using rule notation: S = {x \mid \text{condition on } x}$, read as "$S$is the set of all$x such that the condition holds."\n - Examples:\n - Cities with population over 1 million: S = {x \mid x \text{ is a city with a population over } 1\,\text{million}}.\n - Points on or inside a circle of radius 2$centered at the origin:$S = {(x, y) \mid x^2 + y^2 \le 4}.\n - Sampling items sequentially until one defective item is observed: S = {D, ND, NND, NNND, \dots}.\n\n# Events\n\n- **Definition of Events and Special Subsets:**\n - An **event** is a subset of a sample space S.\n - **Example 2.4:** Let S = {t \mid t \ge 0}$represent the life length in years of an electronic component. The event$A$that the component fails before the end of the fifth year is$A = {t \mid 0 \le t < 5}.\n - **Entire Sample Space (S):** The subset containing all outcomes.\n - **Null Set (\phi):** A subset containing no elements, representing an impossible event.\n - Example: Detecting a microscopic organism with the naked eye (A = \phi).\n - Example: B = {x \mid x \text{ is an even factor of } 7} = \phi$, because the factors of$7$are$1$and$7, which are both odd.\n\n- **Complements of Events:**\n - The **complement** of an event A$with respect to$S$is the subset of all elements in$S$that are not in$A$, denoted by$A'\n - **Example 2.5:** If R$is the event of selecting a red card from a standard deck of$52$playing cards$S$, then$R' is the event of selecting a card that is not red (i.e., a black card).\n - **Example 2.6:** Given S = {\text{book}, \text{cell phone}, \text{mp3}, \text{paper}, \text{stationery}, \text{laptop}}$. If$A = {\text{book}, \text{stationery}, \text{laptop}, \text{paper}}$, then$A' = {\text{cell phone}, \text{mp3}}.\n\n- **Operations on Events:**\n - **Intersection (A \cap B$):** The event containing all elements common to both$A$and$B.\n - **Example 2.7:** Let E$be the event that a student chosen in a classroom is majoring in engineering, and$F$be the event that the student is female. The intersection$E \cap F is the event of choosing a female engineering student.\n - **Example 2.8:** Let V = {a, e, i, o, u}$and$C = {l, r, s, t}$. Their intersection is$V \cap C = \phi.\n - **Mutually Exclusive / Disjoint Events:**\n - Two events A$and$B$are mutually exclusive (or disjoint) if$A \cap B = \phi; they cannot occur simultaneously.\n - **Example 2.9:** Let A$be the event that a television program is affiliated with NBC, and$B$be the event that it is affiliated with CBS. Since a program cannot belong to more than one network,$A \cap B = \phi$, so$A$and$B are mutually exclusive.\n - **Union (A \cup B$):** The event containing all elements belonging to$A$, to$B, or to both.\n - **Example 2.10:** If A = {a, b, c}$and$B = {b, c, d, e}$, then$A \cup B = {a, b, c, d, e}.\n - **Example 2.11:** Let P$be the event that an employee smokes cigarettes, and$Q$be the event that the employee drinks alcoholic beverages.$P \cup Q is the event that the employee drinks, smokes, or does both.\n - **Example 2.12:** If M = {x \mid 3 < x < 9}$and$N = {y \mid 5 < y < 12}$, then$M \cup N = {z \mid 3 < z < 12}.\n\n- **Venn Diagrams and Fundamental Set Identities:**\n - Graphical representation: Sample space S is depicted as a rectangle, and events are represented as circles drawn inside.\n\n\n\n - Fundamental Identities:\n 1. A \cap \phi = \phi\n 2. A \cup \phi = A\n 3. A \cap A' = \phi\n 4. A \cup A' = S\n 5. S' = \phi\n 6. \phi' = S\n 7. (A')' = A\n 8. (A \cap B)' = A' \cup B'\n 9. (A \cup B)' = A' \cap B'\n\n# Counting Sample Points\n\n- **Multiplication Rules:**\n - **Rule 2.1 (Two Operations):** If an operation can be performed in n_1$ways, and for each of these ways a second operation can be performed in$n_2$ways, then the two operations can be performed together in$n_1 n_2 ways.\n - **Example 2.13:** Tossing a pair of dice. The first die lands face-up in n_1 = 6$ways, and the second in$n_2 = 6$ways. Total outcomes =$n_1 n_2 = (6)(6) = 36.\n - **Example 2.14:** Choosing a home with 4$exterior styles (Tudor, rustic, colonial, traditional) and$3$floor plans (ranch, two-story, split-level). Total choices =$n_1 n_2 = (4)(3) = 12.\n\n\n\n - **Example 2.15:** Electing a chair (22$options) and a treasurer ($21$options) from a$22$-member club. Total ways =$n_1 n_2 = (22)(21) = 462$.

  • Rule 2.2 (Generalized Multiplication Rule): If a sequence of $k$ operations can be performed in $n_1, n_2, \dots, n_k$ ways respectively, then the sequence can be performed in $n_1 n_2 \dots n_k$ ways.

  • Example 2.16: Assembling a computer with $2$ chip brands, $4$ hard drives, $3$ memory options, and $5$ accessory stores. Total ways = n_1 n_2 n_3 n_4 = (2)(4)(3)(5) = 120$.\n - **Example 2.17:** Forming even four-digit numbers from digits 0, 1, 2, 5, 6, 9 without repeating any digit.\n - Case 1: Units digit is 0$($n_1 = 1$choice). Thousands position has$n_2 = 5$choices, hundreds position has$n_3 = 4$choices, tens position has$n_4 = 3$choices. Count =$(1)(5)(4)(3) = 60$.

    • Case 2: Units digit is non-zero ($2$ or $6$, so $n_1 = 2$ choices). Thousands position cannot be $0$ or the units digit ($n_2 = 4$ choices), hundreds position has $n_3 = 4$ choices, tens position has $n_4 = 3$ choices. Count = (2)(4)(4)(3) = 96$.\n - Total even four-digit numbers = 60 + 96 = 156$.

• Permutations:

  • A permutation is an arrangement of all or part of a set of objects.

  • Factorial Notation: For any non-negative integer $n$, $n!$ ("$n$ factorial") is defined as:     $n! = n(n-1) \dots (2)(1)$     with special case 0! = 1$.\n - **Theorem 2.1:** The number of permutations of n$distinct objects is$n!$.

  • Example: Permutations of four letters $a, b, c, d$ equal 4! = 24$.\n - **Theorem 2.2:** The number of permutations of n$distinct objects taken$r at a time is:\n    {}n P_r = \frac{n!}{(n - r)!}\n - **Example 2.18:** Distributing 3$distinct awards (research, teaching, service) to a class of$25 graduate students with at most one award per student:\n      {}{25} P_3 = \frac{25!}{(25 - 3)!} = \frac{25!}{22!} = (25)(24)(23) = 13800\n - **Example 2.19:** Electing a president and a treasurer from a student club of 50 people.\n - (a) Without restrictions: {}_{50} P_2 = \frac{50!}{48!} = (50)(49) = 2450$.

    • (b) $A$ serves only if he is president: Either $A$ is president ($49$ treasurer options) or $A$ is not selected (${}_{49} P_2 = (49)(48) = 2352$ options). Total = 49 + 2352 = 2401$.\n - (c) B$and$C$serve together or not at all: Serving together yields$2$options ($B$pres/$C$treas, or$C$pres/$B$treas); neither serving yields${}_{48} P_2 = 2256$options. Total =$2 + 2256 = 2258$.

    • (d) $D$ and $E$ will not serve together: Total ways without restriction ($2450$) minus ways they serve together ($2$) = 2450 - 2 = 2448$.\n\n- **Circular Permutations:**\n - **Theorem 2.3:** The number of permutations of n$distinct objects arranged in a circle is$(n - 1)!$.

  • Explanation: Fixing one object in a single position leaves $(n - 1)$ remaining objects to be arranged relative to it in $(n - 1)!$ distinct ways.

• Permutations of Similar / Non-Distinct Objects:

  • Theorem 2.4: The number of distinct permutations of $n$ things of which $n_1$ are of a first kind, $n_2$ of a second kind, …, $n_k$ of a $k$-th kind is:     $\frac{n!}{n_1! n_2! \dots n_k!}$

  • Example 2.20: Arranging $10$ football players in a row consisting of $1$ freshman, $2$ sophomores, $4$ juniors, and $3$ seniors, distinguishing only by class level:     $\frac{10!}{1! 2! 4! 3!} = 12600$

• Partitions:

  • Theorem 2.5: The number of ways of partitioning a set of $n$ objects into $r$ cells with $n_1$ elements in the first cell, $n_2$ elements in the second, …, and $n_r$ elements in the $r$-th cell (where $n_1 + n_2 + \dots + n_r = n$) is:     $\binom{n}{n_1, n_2, \dots, n_r} = \frac{n!}{n_1! n_2! \dots n_r!}$

  • Example 2.21: Assigning $7$ graduate students to $1$ triple room and $2$ double rooms:     $\binom{7}{3, 2, 2} = \frac{7!}{3! 2! 2!} = 210$

• Combinations:

  • A combination is a selection of $r$ objects from $n$ without regard to order. It represents a partition into two cells containing $r$ and $n - r$ objects.

  • Theorem 2.6: The number of combinations of $n$ distinct objects taken $r$ at a time is:     $\binom{n}{r} = \frac{n!}{r! (n - r)!}$

  • Example 2.22: Selecting $3$ arcade games from $10$ and $2$ sports games from $5$:     $\binom{10}{3} = \frac{10!}{3! 7!} = 120, \quad \binom{5}{2} = \frac{5!}{2! 3!} = 10$     Total selections = (120)(10) = 1200$.\n - **Example 2.23:** Distinct letter arrangements from STATISTICS (10$letters:$3$S's,$3$T's,$2$I's,$1$A,$1 C):\n    \binom{10}{3, 3, 2, 1, 1} = \frac{10!}{3! 3! 2! 1! 1!} = 50400\n\n# Probability of an Event\n\n- **Axioms and Definitions:**\n - **Definition 2.9:** The **probability** of an event A$, denoted by$P(A)$, is the sum of the weights assigned to all sample points in$A$.

  • Axioms:

  1. $0 \le P(A) \le 1$ for every event A$.\n 2. P(\phi) = 0$.

  2. P(S) = 1$.\n 4. If A_1, A_2, A_3, \dots is a sequence of mutually exclusive events, then:\n       P(A_1 \cup A_2 \cup A_3 \cup \dots) = P(A_1) + P(A_2) + P(A_3) + \dots\n\n- **Equally Likely Outcomes:**\n - **Rule 2.3:** If an experiment can result in any one of N$different equally likely outcomes, and exactly$n$of these outcomes correspond to event$A, then:\n    P(A) = \frac{n}{N}\n\n- **Detailed Applications and Examples:**\n - **Example 2.24:** Tossing a balanced coin twice (S = {HH, HT, TH, TT}$, each assigned weight$\omega = \frac{1}{4}$). Event$A$is at least$1$head ($A = {HH, HT, TH}):\n    P(A) = \frac{1}{4} + \frac{1}{4} + \frac{1}{4} = \frac{3}{4}\n - **Example 2.25:** A loaded die where even numbers are twice as likely as odd numbers. Assign weight w$to odd numbers and$2w to even numbers.\n - Sum of weights: w + 2w + w + 2w + w + 2w = 9w = 1 \implies w = \frac{1}{9}.\n - Odd probabilities = \frac{1}{9}$, Even probabilities =$\frac{2}{9}.\n - Event E = {1, 2, 3}$(number less than$4):\n      P(E) = \frac{1}{9} + \frac{2}{9} + \frac{1}{9} = \frac{4}{9}\n - **Example 2.26:** Using Example 2.25, let A = {2, 4, 6}$(even) and$B = {3, 6}$(divisible by$3).\n - A \cup B = {2, 3, 4, 6} \implies P(A \cup B) = \frac{2}{9} + \frac{1}{9} + \frac{2}{9} + \frac{2}{9} = \frac{7}{9}.\n - A \cap B = {6} \implies P(A \cap B) = \frac{2}{9}.\n - **Example 2.27:** Selecting a student at random from a class of 53$($25$industrial,$10$mechanical,$10$electrical,$8 civil engineering majors).\n - (a) Probability of selecting an industrial engineering major P(I) = \frac{25}{53}.\n - (b) Probability of selecting a civil or electrical engineering major P(C \cup E) = \frac{8 + 10}{53} = \frac{18}{53}.\n - **Example 2.28:** Probability of holding 2$aces and$3$jacks in a$5-card poker hand:\n - Number of ways to get 2$aces from$4$:$\binom{4}{2} = 6$.

  • Number of ways to get $3$ jacks from $4$: \binom{4}{3} = 4$.\n - Favorable hands n = (6)(4) = 24$.

  • Total possible $5$-card poker hands N = \binom{52}{5} = \frac{52!}{5! 47!} = 2598960$.\n - Probability P(C) = \frac{24}{2598960} \approx 0.9 \times 10^{-5}.\n\n- **Interpretations of Probability:**\n - **Relative Frequency:** The limiting proportion of times an event occurs in a long series of repeated identical statistical trials.\n - **Indifference Approach:** Equal likelihood assigned to symmetric outcomes (such as a fair die landing on any of its six sides with probability \frac{1}{6}).\n - **Subjective Probability:** Assigned based on personal beliefs, intuition, or historical non-repeatable information (used in Bayesian statistics).\n\n# Additive Rules\n\n- **Theorem 2.7 (Additive Rule for Two Events):**\n - For any two events A$and$B:\n    P(A \cup B) = P(A) + P(B) - P(A \cap B)\n - Proof intuition: Adding P(A)$and$P(B)$counts the probabilities in$A \cap B$twice, so$P(A \cap B) must be subtracted once.\n\n- **Corollaries of the Additive Rule:**\n - **Corollary 2.1:** If A$and$B$are mutually exclusive ($A \cap B = \phi), then:\n    P(A \cup B) = P(A) + P(B)\n - **Corollary 2.2:** If A_1, A_2, \dots, A_n are mutually exclusive, then:\n    P(A_1 \cup A_2 \cup \dots \cup A_n) = P(A_1) + P(A_2) + \dots + P(A_n)\n - **Corollary 2.3:** If A_1, A_2, \dots, A_n$form a partition of the sample space$S, then:\n    P(A_1 \cup A_2 \cup \dots \cup A_n) = P(A_1) + P(A_2) + \dots + P(A_n) = P(S) = 1\n\n- **Theorem 2.8 (Additive Rule for Three Events):**\n - For any three events A$,$B$, and$C:\n    P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(A \cap C) - P(B \cap C) + P(A \cap B \cap C)\n\n- **Theorem 2.9 (Complement Rule):**\n - For complementary events A$and$A'$,$P(A) + P(A') = 1, or:\n    P(A) = 1 - P(A')\n\n- **Examples:**\n - **Example 2.29:** Job offer probabilities: P(A) = 0.8$(company A),$P(B) = 0.6$(company B),$P(A \cap B) = 0.5 (both companies). Probability of at least one offer:\n    P(A \cup B) = 0.8 + 0.6 - 0.5 = 0.9\n - **Example 2.30:** Tossing two fair dice. Find probability of getting a total of 7$or$11$.

  • Event $A$ (sum $7$): $6$ outcomes out of $36 \implies P(A) = \frac{1}{6}$.

  • Event $B$ (sum $11$): $2$ outcomes out of $36 \implies P(B) = \frac{1}{18}$.

  • Since $A \cap B = \phi$: $P(A \cup B) = \frac{1}{6} + \frac{1}{18} = \frac{2}{9}$.

  • Example 2.31: Purchasing a car in green ($0.09$), white ($0.15$), red ($0.21$), or blue ($0.23$).

  • Since choices are mutually exclusive:       $P(G \cup W \cup R \cup B) = 0.09 + 0.15 + 0.21 + 0.23 = 0.68$

  • Example 2.32: Mechanic servicing cars daily with probabilities: $3$ ($0.12$), $4$ ($0.19$), $5$ ($0.28$), $6$ ($0.24$), $7$ ($0.10$), $\ge 8$ ($0.07$).

  • Event $E$: servicing at least $5$ cars. Complement $E'$: servicing fewer than $5$ cars ($3$ or $4$).

  • P(E') = 0.12 + 0.19 = 0.31 \implies P(E) = 1 - 0.31 = 0.69$.\n - **Example 2.33:** Computer cable specification length 2000 \pm 10\,\text{mm}.\n - Probability meeting spec P(M) = 0.99$. Probability too small$P(S) = P(L) = \frac{1 - 0.99}{2} = 0.005$.

  • (a) Probability cable selected is too large = P(L) = 0.005$.\n - (b) Probability cable is larger than 1990\,\text{mm}:\n      P(X \ge 1990) = P(M) + P(L) = 0.99 + 0.005 = 0.995\n      Or via complement: P(X \ge 1990) = 1 - P(S) = 1 - 0.005 = 0.995$.

Conditional Probability, Independence, and the Product Rule

• Conditional Probability:

  • Definition 2.10: The conditional probability of $B$ given $A$, denoted $P(B|A)$, is defined by:     $P(B|A) = \frac{P(A \cap B)}{P(A)}, \quad \text{provided } P(A) > 0$

  • Example 2.34: On-time flight departure ($D$) and arrival ($A$).

  • $P(D) = 0.83$, $P(A) = 0.82$, P(D \cap A) = 0.78$.\n - (a) On-time arrival given on-time departure:\n      P(A|D) = \frac{P(D \cap A)}{P(D)} = \frac{0.78}{0.83} \approx 0.94\n - (b) On-time departure given on-time arrival:\n      P(D|A) = \frac{P(D \cap A)}{P(A)} = \frac{0.78}{0.82} \approx 0.95\n - On-time arrival given late departure:\n      P(A|D') = \frac{P(A \cap D')}{P(D')} = \frac{0.82 - 0.78}{1 - 0.83} = \frac{0.04}{0.17} \approx 0.24\n - **Example 2.35:** Textile cloth strips failing length (L$) and texture ($T) tests.\n - P(L) = 0.10$,$P(T) = 0.05$,$P(L \cap T) = 0.008$.

  • Conditional probability of texture defect given length defect:       $P(T|L) = \frac{P(T \cap L)}{P(L)} = \frac{0.008}{0.10} = 0.08$

• Independent Events:

  • Definition 2.11: Two events $A$ and $B$ are independent if and only if:     $P(B|A) = P(B) \quad \text{or} \quad P(A|B) = P(A)$     assuming conditional probabilities exist. Otherwise, $A$ and $B$ are dependent.

• Product / Multiplicative Rules:

  • Theorem 2.10 (General Multiplicative Rule):     $P(A \cap B) = P(A) P(B|A), \quad \text{provided } P(A) > 0$

  • Theorem 2.11 (Multiplicative Rule for Independent Events): Two events $A$ and $B$ are independent if and only if:     $P(A \cap B) = P(A) P(B)$

  • Example 2.36: Fuse box containing $20$ fuses ($5$ defective). Two fuses selected sequentially without replacement.

  • Let $A$ be first defective, $B$ second defective.

  • $P(A) = \frac{5}{20} = \frac{1}{4}$, P(B|A) = \frac{4}{19}$.\n - P(A \cap B) = \left(\frac{1}{4}\right) \left(\frac{4}{19}\right) = \frac{1}{19}$.

  • Example 2.37: Bag 1 contains $4$ white, $3$ black. Bag 2 contains $3$ white, $5$ black. One ball transferred unseen from Bag 1 to Bag 2, then a ball drawn from Bag 2.

  • Probability black ball drawn from Bag 2:       $P(B_2) = P(B_1) P(B_2|B_1) + P(W_1) P(B_2|W_1) = \left(\frac{3}{7}\right) \left(\frac{6}{9}\right) + \left(\frac{4}{7}\right) \left(\frac{5}{9}\right) = \frac{18 + 20}{63} = \frac{38}{63}$

  • Example 2.38: Independent availability of fire engine ($P(A) = 0.98$) and ambulance ($P(B) = 0.92$).

  • Probability both are available: P(A \cap B) = P(A) P(B) = (0.98)(0.92) = 0.9016$.\n - **Example 2.39:** Electrical system with serial components A$($0.9$),$B$($0.9$) and parallel subsystem$C$($0.8$),$D$($0.8).\n\n\n\n - (a) Probability the system works:\n      P(A \cap B \cap (C \cup D)) = P(A) P(B) [1 - P(C') P(D')] = (0.9)(0.9)[1 - (0.2)(0.2)] = (0.81)(0.96) = 0.7776\n - (b) Probability component C does not work given system works:\n      \frac{P(A \cap B \cap C' \cap D)}{P(\text{system works})} = \frac{(0.9)(0.9)(0.2)(0.8)}{0.7776} = \frac{0.1296}{0.7776} = 0.1667\n\n- **Generalized Multiplicative Rule & Mutual Independence:**\n - **Theorem 2.12:** For events A_1, A_2, \dots, A_k:\n    P(A_1 \cap A_2 \cap \dots \cap A_k) = P(A_1) P(A_2|A_1) P(A_3|A_1 \cap A_2) \dots P(A_k|A_1 \cap A_2 \cap \dots \cap A_{k-1})\n    If events are independent:\n    P(A_1 \cap A_2 \cap \dots \cap A_k) = P(A_1) P(A_2) \dots P(A_k)\n - **Example 2.40:** Drawing 3 cards in succession without replacement.\n - A_1$: red ace ($P(A_1) = \frac{2}{52}).\n - A_2$: 10 or jack ($P(A_2|A_1) = \frac{8}{51}).\n - A_3$: card value between 3 and 7 ($4, 5, 6$, so$12$cards;$P(A_3|A_1 \cap A_2) = \frac{12}{50}).\n - P(A_1 \cap A_2 \cap A_3) = \left(\frac{2}{52}\right) \left(\frac{8}{51}\right) \left(\frac{12}{50}\right) = \frac{8}{5525}$.

  • Definition 2.12 (Mutual Independence): A collection of events $\mathcal{A} = \{A_1, \dots, A_n\}$ is mutually independent if for every subset $\{A_{i_1}, \dots, A_{i_k}\}$ (with $k \le n$):     $P(A_{i_1} \cap \dots \cap A_{i_k}) = P(A_{i_1}) \dots P(A_{i_k})$

Bayes' Rule

• Theorem of Total Probability (Rule of Elimination):

  • Theorem 2.13: Let $B_1, B_2, \dots, B_k$ form a partition of the sample space $S$ such that $P(B_i) \neq 0$ for all $i$. Then for any event $A$ of $S$:     $P(A) = \sum_{i=1}^k P(B_i \cap A) = \sum_{i=1}^k P(B_i) P(A|B_i)$

  • Example 2.41: Plant with 3 machines ($B_1, B_2, B_3$) producing $30\%$, $45\%$, and $25\%$ of products with defect rates $2\%$, $3\%$, and $2\%$ respectively.

  • $P(B_1) = 0.30$, P(A|B_1) = 0.02 \implies P(B_1) P(A|B_1) = 0.006$.\n - P(B_2) = 0.45$,$P(A|B_2) = 0.03 \implies P(B_2) P(A|B_2) = 0.0135$.

  • $P(B_3) = 0.25$, P(A|B_3) = 0.02 \implies P(B_3) P(A|B_3) = 0.005$.\n - Total defective product probability P(A) = 0.006 + 0.0135 + 0.005 = 0.0245$.

• Bayes' Rule:

  • Theorem 2.14 (Bayes' Rule): Let $B_1, B_2, \dots, B_k$ be a partition of $S$ with $P(B_i) \neq 0$. For any event $A$ with $P(A) \neq 0$:     $P(B_r|A) = \frac{P(B_r \cap A)}{\sum_{i=1}^k P(B_i \cap A)} = \frac{P(B_r) P(A|B_r)}{\sum_{i=1}^k P(B_i) P(A|B_i)}, \quad r = 1, 2, \dots, k$

  • Example 2.42: Given that a selected product from Example 2.41 is defective, probability it was made by machine $B_3$:     $P(B_3|A) = \frac{P(B_3) P(A|B_3)}{P(A)} = \frac{0.005}{0.0245} = \frac{10}{49} \approx 0.2041$

  • Example 2.43: Three analytical plans ($P_1, P_2, P_3$) used $30\%$, $20\%$, and $50\%$ of the time. Defect rates: $P(D|P_1) = 0.01$, $P(D|P_2) = 0.03$, P(D|P_3) = 0.02$.\n - Total defect probability:\n      P(D) = (0.30)(0.01) + (0.20)(0.03) + (0.50)(0.02) = 0.003 + 0.006 + 0.010 = 0.019\n - Posterior probabilities given a defective product:\n - Plan 1: P(P_1|D) = \frac{0.003}{0.019} \approx 0.158\n - Plan 2: P(P_2|D) = \frac{0.006}{0.019} \approx 0.316\n - Plan 3: P(P_3|D) = \frac{0.010}{0.019} \approx 0.526$$

  • Plan 3 is most likely responsible for the defective product.