Comprehensive Study Notes on Probability Theory, Independence, and Conditional Probability

Conditional Probability Framework

• Definition and Mathematical Formulation:

  • Conditional probability evaluates the likelihood of an event $X$ occurring given that another event $Y$ has already taken place, denoted formally as $P(X \mid Y)$.

  • The formal mathematical definition assumes that the conditioning event has a non-zero probability ($P(Y) > 0$):     $P(X \mid Y) = \frac{P(X \cap Y)}{P(Y)}$

  • The numerator $P(X \cap Y)$ represents the joint probability of both events $X$ and $Y$ occurring simultaneously (the intersection).

  • The denominator $P(Y)$ acts as the new, restricted sample space.

• Venn Diagram Geometric Interpretation:

  • Unconditional probability measures event sizes relative to the entire sample space $S$ (where $P(S) = 1$).

  • When event $Y$ is known to have occurred, the sample space collapses entirely to the region occupied by $Y$.

  • Visually, $P(X \mid Y)$ measures the relative area of the intersection region $X \cap Y$ as an exact proportion of the total area of region Y$.\n\n- **Information Updates and Sample Space Reduction**:\n - Acquiring new information fundamentally changes the baseline probability distribution by eliminating outcomes outside the updated condition.\n - The transition from unconditional probability P(X)$to conditional probability$P(X \mid Y)$reflects the dynamic revision of beliefs upon observing evidence$Y$.

Card Deck Probability Applications

• Standard Deck Structure:

  • Total cards in a standard deck: $52$ cards.

  • Suits: $4$ suits (Clubs, Spades, Hearts, Diamonds), each containing $13$ cards.

  • Colors: $26$ black cards (Clubs and Spades) and $26$ red cards (Hearts and Diamonds).

  • Ranks: $13$ ranks (Ace, $2$, $3$, $4$, $5$, $6$, $7$, $8$, $9$, $10$, Jack, Queen, King), with exactly $4$ cards per rank (one of each suit).

• Unconditional Probability Baseline:

  • The probability of drawing any specific single named card (e.g., the Four of Clubs) from a full, randomized $52$-card deck is:     $P(\text{Four of Clubs}) = \frac{1}{52}$

• Conditional Probability with Color Constraints:

  • Event Definitions:

  • Let Event $X$ be drawing the Four of Clubs.

  • Let Event $Y$ be observing that the drawn card is black.

  • Evaluating Component Probabilities:

  • Unconditional probability of drawing a black card:       $P(Y) = \frac{26}{52} = \frac{1}{2}$

  • Joint probability $P(X \cap Y)$ of drawing a card that is both the Four of Clubs AND black:       Since the Four of Clubs is intrinsically a black card, exactly $1$ card out of $52$ satisfies both conditions:       $P(X \cap Y) = \frac{1}{52}$

  • Applying the Conditional Formula:     $P(X \mid Y) = \frac{P(X \cap Y)}{P(Y)} = \frac{\frac{1}{52}}{\frac{1}{2}} = \frac{2}{52} = \frac{1}{26}$

  • Interpretation: Knowing the card is black eliminates all $26$ red cards, effectively shrinking the sample space from $52$ to $26$ outcomes and doubling the likelihood of drawing the Four of Clubs.

Mutually Exclusive vs. Independent Events

• Mutually Exclusive (Disjoint) Events:

  • Conceptual Definition: Two events $X$ and $Y$ are mutually exclusive if they cannot occur at the same time. The occurrence of $X$ completely precludes the occurrence of Y$.\n - **Mathematical Criterion**:\n    P(X \cap Y) = 0\n - **Venn Diagram Representation**: The regions representing X$and$Y share zero overlap.\n - **Card Example**: Drawing a single card that is simultaneously a Five and an Eight. Because a card possesses exactly one rank, P(\text{Five} \cap \text{Eight}) = 0$.

• Independent Events:

  • Conceptual Definition: Two events $X$ and $Y$ are independent if learning that $Y$ occurred provides zero additional information about whether $X$ will occur.

  • Mathematical Criteria:     $P(X \cap Y) = P(X) \times P(Y)$     Alternatively, expressed via conditional probability:     $P(X \mid Y) = P(X) \quad \text{and} \quad P(Y \mid X) = P(Y)$

  • Card Deck Example (Rank and Suit Independence):

  • Probability of drawing a Six:       $P(\text{Six}) = \frac{4}{52} = \frac{1}{13}$

  • Probability of drawing a Six given the card is a Club:       $P(\text{Six} \mid \text{Club}) = \frac{1}{13}$

  • Because every suit contains exactly one Six out of $13$ cards, knowing the suit provides no predictive value regarding the rank.

  • Verification using multiplication rule:       $P(\text{Four} \cap \text{Club}) = P(\text{Four}) \times P(\text{Club}) = \frac{1}{13} \times \frac{1}{4} = \frac{1}{52}$

• Distinction Between Mutual Exclusivity and Independence:

  • Mutually exclusive events with non-zero probabilities are never independent, because if $X$ occurs, the conditional probability of $Y$ drops to $0$, altering its likelihood.

The Two-Coin Conditional Probability Problem

• Problem Setup:

  • A bag contains $2$ coins:

  1. Coin 1 (Fair Coin): $1$ Heads face, $1$ Tails face.

  2. Coin 2 (Trick Coin): $2$ Heads faces ($0$ Tails faces).

  • A coin is selected uniformly at random from the bag and flipped.

  • Observation: The outcome of the flip is Heads.

  • Goal: Calculate the probability that the coin chosen was the fair coin given the observed Heads flip.

• Sample Space Analysis via Distinct Faces:

  • Total distinct physical faces across both coins = $4$ faces:

  • Fair Coin: Face 1 (Heads), Face 2 (Tails)

  • Trick Coin: Face 3 (Heads), Face 4 (Heads)

  • Total faces displaying Heads = $3$ faces (Face 1, Face 3, Face 4). Each face has an equal probability of $\frac{1}{4}$ of being selected and shown.

• Mathematical Calculation:

  • Define Event $X$: Selecting the fair coin.

  • Define Event $Y$: Observing a Heads result on the flip.

  • Unconditional probability of getting Heads:     $P(Y) = \frac{3}{4}$

  • Joint probability of choosing the fair coin AND flipping Heads ($X \cap Y$):     Only Face 1 satisfies both conditions (Fair coin + Heads):     $P(X \cap Y) = \frac{1}{4}$

  • Conditional Probability calculation:     $P(X \mid Y) = \frac{P(X \cap Y)}{P(Y)} = \frac{\frac{1}{4}}{\frac{3}{4}} = \frac{1}{3}$

• Intuition vs. Analytical Reality:

  • Intuition falsely suggests a $\frac{1}{2}$ chance, reasoning that after drawing a coin, there are two coins originally.

  • However, observing Heads provides empirical evidence that shifts the odds toward the trick coin (which guarantees Heads), decreasing the posterior probability of having picked the fair coin down to $\frac{1}{3}$.

General and Special Laws of Addition

• General Law of Addition (Inclusion-Exclusion Principle):

  • Computes the probability of the union of two events (event $X$, event $Y$, or both occurring), written as $P(X \cup Y)$.

  • Formula:     $P(X \cup Y) = P(X) + P(Y) - P(X \cap Y)$

  • Logic: Adding $P(X)$ and $P(Y)$ counts the overlapping region $X \cap Y$ twice. Subtracting $P(X \cap Y)$ corrects for this double-counting.

• Special Law of Addition:

  • Applies exclusively when events $X$ and $Y$ are mutually exclusive ($P(X \cap Y) = 0$).

  • Formula:     $P(X \cup Y) = P(X) + P(Y)$

• Worked Application Example (Two Fair Coin Flips):

  • Scenario: Flipping two distinct fair coins independently.

  • Define Event $X$: Heads on the first coin ($P(X) = \frac{1}{2}$).

  • Define Event $Y$: Heads on the second coin ($P(Y) = \frac{1}{2}$).

  • Intersection Event $X \cap Y$: Heads on both coins ($P(X \cap Y) = \frac{1}{4}$).

  • Probability of getting Heads on at least one coin ($P(X \cup Y)$):     $P(X \cup Y) = \frac{1}{2} + \frac{1}{2} - \frac{1}{4} = \frac{3}{4}$

  • Direct Verification via Sample Space Listing:

  • Possible outcomes: $\{(H,H), (H,T), (T,H), (T,T)\}$ ($4$ total equal outcomes).

  • Outcomes with at least one Heads: $3$ outcomes ($(H,H), (H,T), (T,H)$).

  • Ratio: $\frac{3}{4}$.