Bayes' Rule, Independence, and Probability Concepts

Bayes' Rule and Inference

• Thomas Bayes: Presbyterian minister known for his theorem (1701-1761).
• Bayes' Theorem: A systematic approach for incorporating new evidence into prior beliefs.
  • Bayesian Inference: Involves updating the initial beliefs $P(A_1)$ regarding possible causes of the observed event $B$.
  • Requires a world-model for each potential cause $A_i$:
  • $P(B | Ai)$ is used to draw conclusions about the causes, yielding the posterior probability $P(Ai | B)$.

Independence and Conditional Probabilities

• Basic Concepts:
  • For a biased coin with $P(H) = p$ (Probability of Heads), then $P(T) = 1 - p$ (Probability of Tails).
  • Multiplication Rule: If $A$ and $B$ are two events, then
    $P(A \cap B) = P(A) \cdot P(B | A)$
• Total Probability: Aggregate behaviors of $A$ leading to events $B$.

Independence of Two Events

• Intuitive Definition: For two events $A$ and $B$, if occurrence of $A$ provides no new information about $B$, then $P(B | A) = P(B)$.
• Formal Definition: Events A and B are independent if: $P(A \cap B) = P(A) \cdot P(B)$
  • Symmetric: Also implies that $P(A|B) = P(A)$ and holds even when $P(A)$ or $P(B)$ = 0.

The Product Rule (Multiplicative Rule)

• If events $A$ and $B$ can both occur:
  • $P(A \cap B) = P(A) \cdot P(B | A)$ (providing $P(A) > 0$)
• For independent events:
  • $P(A \cap B) = P(A) \cdot P(B)$

Independence of Event Complements

• If $A$ and $B$ are independent, then events not $A$ and not $B$ are also independent.

Example: Drawing Two Cards

• Experiment: Drawing cards with replacement.
  • Define Event $A$: first card is an ace.
  • Define Event $B$: second card is a spade.
  • Since the first card is replaced, outcomes are independent, yielding $P(B | A) = P(B)$.

Conditional Independence

• Conditional Independence Given C: If $A$ and $B$ are independent but conditioned upon event $C$, they may not be independent.

Independence of Collections of Events

• Events $A1, A2,…, An$ are independent if: $P(A</em>{i1} \cap A<em>{i2} \cap … \cap A</em>{im}) = P(A<em>{i1}) \cdot P(A</em>{i2}) \cdots P(A_{im})$
• For three events, if $n=3$:
  • Pairwise Independence: $P(A<em>1 \cap A</em>2) = P(A<em>1) \cdot P(A</em>2)$

Reliability and Independence

• Reliability of Independent Units: Given multiple units with probabilities $Pi$ of being operational, calculating the system's operational status requires considering those individual probabilities: $P</em>{system} = P<em>1 \cdot P</em>2 \cdots P_n$

Exercises on Probability

1. Determine if "liked recreational reading" and "disliked recreational reading" are mutually exclusive.
2. For public transportation, find the probability that a commuter had a commute of 60 or more minutes.
3. Analyze lunch forgetting among students: what is the chance neither selected student forgot their lunch.
4. Discuss a scenario in disease testing with probabilities of detection, false positives to demonstrate Bayes’ rule application.

Additional Examples & Scenarios

• Consider production lines with defective products, analyzing the probability of defects among randomly selected products.
• Count and probability examples related to chocolates show various outcomes with random selections.