Comprehensive Notes on Introductory Statistics: Sample Spaces, Counting Rules, Permutations, Combinations, and Contingency Table Probabilities

Fundamental Concepts of Probability and Sample Spaces

• Sample Space Definition:

  • The sample space, denoted by $S$, is the set of all possible outcomes or results of a statistical experiment.
  • The size of the sample space refers to the total count of distinct elements or outcomes contained within $S$.

• Basic Sample Space Examples:

  • Coin Toss Experiment:
  • Outcomes set: $S = \{\text{Head}, \text{Tail}\}$ (or {\text{Hit}, \text{Tear}}).
  • Sample space size: $2$ elements.
  • Single Die Throw Experiment:
  • Outcomes set based on the upper face of the die: $S = \{1, 2, 3, 4, 5, 6\}$.
  • Sample space size: $6$ elements.
  • Blood Type Classification:
  • Outcomes set for human blood types: $S = \{A, B, AB, O\}$ (combining positive and negative Rh factors into primary ABO blood group categories).
  • Sample space size: $4$ elements.

• Kolmogorov's Axioms of Probability:

  • First established in $1950$ by scientist Adriel Kogurov (Andrey Kolmogorov), these two fundamental axioms form the foundation of probability theory and modern statistical science.
  • Applied extensively in system design, randomized algorithms, and game theory (such as action randomization in computer games like Super Mario).
  • Axiom 1 (Range Criterion):
  • For any event or outcome $E_i$ within an experiment, its assigned probability $P(E_i)$ must be a real number strictly bounded between $0$ and $1$ inclusive:     $0 \le P(E_i) \le 1$
  • Axiom 2 (Completeness Criterion):
  • The sum of the probabilities of all simple outcomes across the entire sample space $S$ must equal exactly $1$:     $\sum_{i=1}^{n} P(E_i) = 1$

• Equally Probable Single-Stage Experiments:

  • Fair Coin Toss:
  • Outcomes $E_1 = \text{Head}$, $E_2 = \text{Tail}$.
  • $P(\text{Head}) = \frac{1}{2}$, $P(\text{Tail}) = \frac{1}{2}$.
  • Axiom verification: $\frac{1}{2} + \frac{1}{2} = 1$, where $0 \le \frac{1}{2} \le 1$
  • Balanced Die Throw:
  • Outcomes $E_1 = 1, E_2 = 2, E_3 = 3, E_4 = 4, E_5 = 5, E_6 = 6$.
  • Probabilities: $P(1) = P(2) = P(3) = P(4) = P(5) = P(6) = \frac{1}{6}$.
  • Axiom verification:     $\frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} = \frac{1+1+1+1+1+1}{6} = \frac{6}{6} = 1$

Calculating Sub-Event Probabilities

• Subset Probabilities for a Balanced Die:

  • Given sample space $S = \{1, 2, 3, 4, 5, 6\}$:
  • Event A (Observing an odd upper face):
  • Favorable outcomes: {1, 3, 5} ($3$ outcomes).
  • Probability: $P(A) = \frac{3}{6} = \frac{1}{2} = 0.50$.
  • Event B (Observing a face greater than or equal to $4$):
  • Favorable outcomes: {4, 5, 6} ($3$ outcomes).
  • Probability: $P(B) = \frac{3}{6} = \frac{1}{2} = 0.50$.
  • Event C (Observing a face strictly greater than $4$):
  • Favorable outcomes: {5, 6} ($2$ outcomes).
  • Probability: $P(C) = \frac{2}{6} = \frac{1}{3} \approx 0.3333$.
  • Properties of Denominators in Subset Probabilities:
  • When evaluating subset probabilities, the total sample space size ($6$) remains in the denominator because outcomes are selected from the overarching $6$-element space.
  • Selected subset probabilities do not sum to $1$ unless they cover the entire sample space; individual sub-event probabilities strictly remain bounded in the interval $[0, 1]$.

• Two Fair Coins Experiment:

  • Sample Space Construction:
  • Outcomes set: $S = \{HH, HT, TH, TT\}$.
  • Sample space size: $4$ elements.
  • Outcome $HT$ (Head on coin 1, Tail on coin 2) is distinct from $TH$ (Tail on coin 1, Head on coin 2).
  • Theoretical assumption: The probability of a coin landing on its rim is $0$.
  • Element Probabilities:
  • Each outcome has probability $\frac{1}{4}$, since $\frac{1}{4} + \frac{1}{4} + \frac{1}{4} + \frac{1}{4} = 1$
  • Composite Events:
  • Event C (Both faces are identical):
    • Favorable outcomes: {HH, TT}.
    • $P(C) = P(HH) + P(TT) = \frac{1}{4} + \frac{1}{4} = \frac{2}{4} = \frac{1}{2} = 0.50$
  • Event D (First face is Head):
    • Favorable outcomes: {HH, HT}.
    • $P(D) = P(HH) + P(HT) = \frac{1}{4} + \frac{1}{4} = \frac{2}{4} = \frac{1}{2} = 0.50$
  • Event E (Second face is Head):
    • Favorable outcomes: {HH, TH}.
    • $P(E) = P(HH) + P(TH) = \frac{1}{4} + \frac{1}{4} = \frac{2}{4} = \frac{1}{2} = 0.50$
  • Event F (Second face is Tail):
    • Favorable outcomes: {HT, TT}.
    • $P(F) = P(HT) + P(TT) = \frac{1}{4} + \frac{1}{4} = \frac{2}{4} = \frac{1}{2} = 0.50$

Fundamental Counting Rules and Multiplication Principle

• Multiplication Principle of Counting (MLO Rule):

  • Used to determine sample space sizes when direct manual enumeration is impractical.
  • Two-Stage Experiments:
  • If Stage 1 can occur in $m$ distinct ways and Stage 2 can occur in $n$ distinct ways, the total number of combined ways to complete the experiment is:     $\text{Total Outcomes} = m \times n$
  • Multi-Stage ($k$-Stage) Experiments:
  • If an experiment progresses across $k$ stages with outcomes $m_1, m_2, \dots, m_k$, the total sample space size is:     $\text{Total Outcomes} = m_1 \times m_2 \times \dots \times m_k$

• Applications of Counting Rules:

  • Tossing Two Balanced Dice:
  • Die 1 has $6$ outcomes; Die 2 has $6$ outcomes.
  • Sample space size: $6 \times 6 = 36$ total outcomes.
  • Tossing Three Balanced Dice:
  • $3$ stages with $6$ outcomes each.
  • Sample space size: $6 \times 6 \times 6 = 216$ total outcomes.
  • Tossing Three Fair Coins:
  • $3$ stages with $2$ outcomes (Head, Tail) each.
  • Sample space size: $2 \times 2 \times 2 = 8$ total outcomes.
  • Outcomes set: {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}.
  • Multi-City Travel Routes:
  • Traveling from City A (Regina) to City D (La Ronge) via City B (Saskatoon) and City C (Prince Albert):
    • Stage 1 (City A to City B): $3$ route options.
    • Stage 2 (City B to City C): $4$ route options.
    • Stage 3 (City C to City D): $3$ route options.
  • Total unique travel routes: $3 \times 4 \times 3 = 36$ distinct ways.

Permutations

• Definition and Mathematical Formula:

  • A permutation is an arrangement of $r$ objects selected from a set of $n$ distinct objects where the order of selection matters.
  • Formula:   $P(n, r) = {}_n P_r = \frac{n!}{(n-r)!}$
  • Factorial Definition:
  • $n! = n \times (n-1) \times (n-2) \times \dots \times 1$
  • Example: $6! = 6 \times 5 \times 4 \times 3 \times 2 \times 1 = 720$
  • Convention: $0! = 1$

• Permutation Problems and Calculations:

  • Selecting $3$ Genes from $10$ (Order Matters):
  • Selecting $3$ targeted disease-causing genes out of $10$ candidates:     ${}_{10}P_3 = \frac{10!}{(10-3)!} = \frac{10!}{7!} = \frac{10 \times 9 \times 8 \times 7!}{7!} = 10 \times 9 \times 8 = 720 \text{ ways}$
  • Selecting $4$ Genes from $7$ (Order Matters):
  • ${}_7 P_4 = \frac{7!}{(7-4)!} = \frac{7!}{3!} = \frac{7 \times 6 \times 5 \times 4 \times 3!}{3!} = 7 \times 6 \times 5 \times 4 = 840 \text{ ways}$
  • Arranging $2$ Books from $3$ Candidates on a Bookshelf:
  • Selecting and ordering $2$ textbooks (e.g., Biochemistry, Chemistry, Statistics) on a bookshelf holding $2$ books:     ${}_3 P_2 = \frac{3!}{(3-2)!} = \frac{3!}{1!} = \frac{6}{1} = 6 \text{ ways}$
  • Explicit outcomes: {(Bio, Chem), (Bio, Stat), (Stat, Bio), (Chem, Stat), (Stat, Chem), (Chem, Bio)}.
  • Assembling $5$ Equipment Parts in Sequence:
  • Assembling a piece of equipment made of $5$ unique parts where assembly sequence matters:     ${}_5 P_5 = \frac{5!}{(5-5)!} = \frac{5!}{0!} = \frac{120}{1} = 120 \text{ ways}$

Combinations

• Definition and Mathematical Formula:

  • A combination is a selection of $r$ objects from a collection of $n$ distinct objects where order does NOT matter.
  • Formula:   $C(n, r) = {}_n C_r = \binom{n}{r} = \frac{n!}{r! \, (n-r)!}$

• Relationship Between Combinations and Permutations:

  • Because order is disregarded in combinations, sample space sizes are reduced relative to permutations:   ${}_n C_r \le {}_n P_r$
  • The factor $r!$ in the denominator eliminates duplicate permutations of selected subsets.

• Combination Problems and Calculations:

  • Selecting $2$ Books from $3$ Candidates (Order Disregarded):
  • ${}_3 C_2 = \frac{3!}{2! \, (3-2)!} = \frac{3!}{2! \, 1!} = \frac{6}{2 \times 1} = 3 \text{ ways}$
  • Reduces the $6$ permuted outcomes down to $3$ unordered pairs: {(Bio, Chem), (Bio, Stat), (Chem, Stat)}.
  • Selecting $3$ Objects from $10$:
  • ${}_{10} C_3 = \frac{10!}{3! \, (10-3)!} = \frac{10!}{3! \, 7!} = \frac{10 \times 9 \times 8 \times 7!}{(3 \times 2 \times 1) \times 7!} = \frac{720}{6} = 120 \text{ ways}$
  • (Permutation equivalent was $720$).
  • Selecting $4$ Objects from $7$:
  • ${}_7 C_4 = \frac{7!}{4! \, (7-4)!} = \frac{7!}{4! \, 3!} = \frac{7 \times 6 \times 5 \times 4!}{4! \times (3 \times 2 \times 1)} = \frac{210}{6} = 35 \text{ ways}$
  • (Permutation equivalent was $840$).
  • Selecting $6$ Objects from $6$:
  • ${}_6 C_6 = \frac{6!}{6! \, 0!} = \frac{6!}{6! \times 1} = 1 \text{ way}$
  • (Permutation equivalent ${}_6 P_6 = 720$).
  • Selecting $1$ Object from $20$:
  • ${}_{20} C_1 = \frac{20!}{1! \, 19!} = \frac{20 \times 19!}{1 \times 19!} = 20 \text{ ways}$
  • Note: When $r = 1$, permutation and combination values are equal (${}_{20} C_1 = {}_{20} P_1 = 20$).

Marginal Probability and Contingency Tables

• Definition of Marginal Probability:

  • Marginal probability computes the probability of a single simple event occurring across an entire sample population without any restricting condition.
  • Calculation formula:   $\text{Marginal Probability} = \frac{\text{Total Observations in Row or Column Class}}{\text{Grand Total Observations}}$

• $2 \times 2$ Contingency Table Analysis (Employee Compensation Plan Survey):

  • A study surveyed $100$ employees ($60$ Male, $40$ Female) regarding approval of a high-salary compensation plan.

• Marginal Probability Calculations:

  • Probability of selecting a Male employee:     $P(\text{Male}) = \frac{60}{100} = 0.60 \text{ (or } 60\%$
  • Probability of selecting a Female employee:     $P(\text{Female}) = \frac{40}{100} = 0.40 \text{ (or } 40\%$
  • Probability of selecting an employee In Favor:     $P(\text{In Favor}) = \frac{19}{100} = 0.19 \text{ (or } 19\%$
  • Probability of selecting an employee Against:     $P(\text{Against}) = \frac{81}{100} = 0.81 \text{ (or } 81\%$

• Verification of Axioms in Contingency Margins:

  • Row Margins Sum: $P(\text{Male}) + P(\text{Female}) = 0.60 + 0.40 = 1.00$
  • Column Margins Sum: $P(\text{In Favor}) + P(\text{Against}) = 0.19 + 0.81 = 1.00$

Conditional Probability

• Definition and Structure:

  • Conditional probability computes the likelihood of an event occurring given that a specific subset condition or secondary attribute is already known to be satisfied.
  • The condition restricts the baseline denominator from the grand total ($100$) to the specific conditional row or column subtotal.

• Conditional Computations from Employee Survey Data:

  • Given that an employee is Male, probability they are In Favor:
  • Restricting denominator: Total Males ($60$).     $P(\text{In Favor} \mid \text{Male}) = \frac{15}{60} = 0.25 \text{ (or } 25\%$
  • Given that an employee is In Favor, probability they are Male:
  • Restricting denominator: Total In Favor ($19$).     $P(\text{Male} \mid \text{In Favor}) = \frac{15}{19} \approx 0.7895 \text{ (or } 78.95\%$
  • Given that an employee is In Favor, probability they are Female:
  • Restricting denominator: Total In Favor ($19$).     $P(\text{Female} \mid \text{In Favor}) = \frac{4}{19} \approx 0.2105 \text{ (or } 21.05\%$
  • Given that an employee is Female, probability they are In Favor:
  • Restricting denominator: Total Females ($40$).     $P(\text{In Favor} \mid \text{Female}) = \frac{4}{40} = 0.10 \text{ (or } 10\%$
  • Given that an employee is Against, probability they are Female:
  • Restricting denominator: Total Against ($81$).     $P(\text{Female} \mid \text{Against}) = \frac{36}{81} \approx 0.4444 \text{ (or } 44.44\%$
  • Given that an employee is Female, probability they are Against:
  • Restricting denominator: Total Females ($40$).     $P(\text{Against} \mid \text{Female}) = \frac{36}{40} = 0.90 \text{ (or } 90\%$
  • Given that an employee is Against, probability they are Male:
  • Restricting denominator: Total Against ($81$).     $P(\text{Male} \mid \text{Against}) = \frac{45}{81} \approx 0.5556 \text{ (or } 55.56\%$
  • Given that an employee is Male, probability they are Against:
  • Restricting denominator: Total Males ($60$).     $P(\text{Against} \mid \text{Male}) = \frac{45}{60} = 0.75 \text{ (or } 75\%$

Questions & Discussion

• Question on Coin Landing on Rim:

  • Question: Why is the outcome of a coin landing on its rim not included in the sample space?
  • Answer: In theoretical statistics, the probability of a coin landing on its edge or rim is assumed to be strictly $0$, restricting sample space outcomes strictly to Heads and Tails.

• Question on Sample Space Subset Denominators:

  • Question: Why doesn't the denominator change from $6$ to $3$ when calculating $P(\text{outcome} \ge 4)$ for a die throw?
  • Answer: The overarching outcome pool remains the original sample space of size $6$. Evaluating specific subset conditions restricts the numerator (favorable outcomes), but the sample baseline remains $6$.

• Question on Permutations vs. Combinations Criteria:

  • Question: How do we determine whether order matters in a practical scenario?
  • Answer: Order matters when sequential position changes the system state (e.g., gene expression ordering or physical arrangement sequence), dictating Permutations. When simple inclusion in a group is evaluated regardless of sequence, Combinations are applied.

• Question on Showing Mathematical Work on Exams:

  • Question: Is it required to write out full factorial expansions on exams?
  • Answer: Full itemized term-by-term multiplications are not required, but intermediate setup steps showing the factorial formula expression before writing the final simplified numeric evaluation must be provided.