MTH331 Lecture Notes 02: Probabilistic Model and Discrete Probability

Part 1: Understanding Random Phenomena

What Makes Something "Random"?

Random phenomena share these characteristics:

• Unpredictable outcomes: Results vary when you repeat the process

• Defined possibilities: Outcomes come from a known set of options

• Varying likelihoods: Some outcomes are more probable than others

Common Examples

1. Coin toss: Heads (H) or Tails (T)

2. Drawing from a box: 3 red + 2 blue balls → Red or Blue outcome

3. Student survey: Random student's major → Math, CS, Biology, English, etc.

4. Stock prediction: Tomorrow's movement → Up or Down

5. Class quiz: Any given day → Yes or No

Key Insight: Probability theory = Set theory + Randomness

Part 2: Building Blocks of Probability Theory

Essential Vocabulary

The Formal Framework

A probability system is a triple (Ω, ℱ, P):

• Ω: Sample space (all possible outcomes)

• ℱ: Collection of events we can assign probabilities to

• P: Function that assigns probabilities to events

Part 3: Kolmogorov's Probability Axioms

The Three Fundamental Rules

These axioms, established by A.N. Kolmogorov in 1933, form the foundation of modern probability theory:

1. Non-negativity

P(A) ≥ 0 for every event A

• Probabilities are never negative

• Example: P(getting heads) = 0.5 ≥ 0 ✓

2. Additivity (for disjoint events)

P(A ∪ B) = P(A) + P(B) when A and B don't overlap

• For mutually exclusive events, probabilities add up

• Example: P(H or T) = P(H) + P(T) = 0.5 + 0.5 = 1

Extended version: For infinite sequences of disjoint events A₁, A₂, A₃, ... P(A₁ ∪ A₂ ∪ A₃ ∪ ...) = P(A₁) + P(A₂) + P(A₃) + ...

3. Normalization

P(Ω) = 1

• The probability of "something happening" is always 1

• Example: P(H or T) = 1 (one of these must occur)

Why These Axioms Matter

These three simple rules generate all the properties we need for probability calculations.

Part 4: Coin Tossing Examples (Step-by-Step)

Example 1: Single Coin Toss

• Sample space: Ω = {H, T}

• Events: {H}, {T}, {H,T}, ∅ (empty set)

• Probabilities: P({H}) = P({T}) = 1/2

Example 2: Two Coin Tosses

• Sample space: Ω = {HH, HT, TH, TT}

• Equal probability: Each outcome has probability 1/4

• Calculation: P({HT}) = P({HH}) = P({TH}) = P({TT}) = 1/4

Example 3: Three Coin Tosses

• Sample space: Ω = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

• Total outcomes: 8 possibilities

• Equal probability: Each outcome has probability 1/8

Practice Problem: Find P(exactly 2 heads in 3 tosses)

• Event E: {HHT, HTH, THH}

• Calculation: P(E) = P({HHT}) + P({HTH}) + P({THH}) = 1/8 + 1/8 + 1/8 = 3/8

• Formula: P(E) = |E|/|Ω| = 3/8

Part 5: Discrete Probability Laws

General Discrete Probability Law

When the sample space has finitely many outcomes, the probability of any event equals the sum of probabilities of its individual outcomes:

P({ω₁, ω₂, ..., ωₙ}) = P({ω₁}) + P({ω₂}) + ... + P({ωₙ})

Discrete Uniform Probability Law

When all outcomes are equally likely:

P(A) = |A|/n

Where:

• |A| = number of outcomes in event A

• n = total number of possible outcomes

When to use: Perfect for symmetric situations like fair coins, dice, or random selection.

Part 6: Properties of Probability Laws

Essential Properties to Remember

(a) Monotonicity

If A ⊆ B, then P(A) ≤ P(B)

• Larger events have higher (or equal) probability

• Example: P(getting at least one head) ≥ P(getting exactly two heads)

(b) Inclusion-Exclusion (Two Events)

P(A ∪ B) = P(A) + P(B) - P(A ∩ B)

• Accounts for overlap between events

• Example: P(head on first OR second toss) = P(head on first) + P(head on second) - P(head on both)

(c) Subadditivity

P(A ∪ B) ≤ P(A) + P(B)

• Union probability never exceeds sum of individual probabilities

• Equality holds only when events are disjoint

(d) Inclusion-Exclusion (Three Events)

P(A ∪ B ∪ C) = P(A) + P(B) + P(C) - P(A ∩ B) - P(A ∩ C) - P(B ∩ C) + P(A ∩ B ∩ C)

Study Tips and Key Takeaways

1. Master the Vocabulary

• Sample space = all possibilities

• Event = subset of sample space

• Probability = number between 0 and 1

2. Remember the Axioms

• Probabilities are non-negative

• Disjoint events add up

• Total probability equals 1

3. Practice with Uniform Models

• Start with equal probability cases

• Use the formula P(A) = |A|/|Ω|

• Count carefully!

4. Understand Set Operations

• Union (∪) = "or"

• Intersection (∩) = "and"

• Complement = "not"

5. Check Your Work

• All probabilities should be between 0 and 1

• Probabilities of all possible outcomes should sum to 1

• Use inclusion-exclusion for overlapping events

Quick Reference Formulas