Basics of Probability and Probability Distributions

Learning Goals and Core Concepts of Probability

• Foundational Objective: The primary goal is to find probabilities and construct probability distributions using both theoretical and relative frequency methods.

• Outcomes: These are the most basic possible results of observations or experiments.

  • Example: In a toss of two coins, there are four distinct outcomes: $HH$, $HT$, $TH$, and $TT$.

• Events: An event is a defined collection of one or more outcomes that share a property of interest.

  • Example: In a toss of two coins, if we are interested only in the number of heads, there are only three possible events: 0 heads, 1 head, and 2 heads.

  • Outcome vs. Event Difference: While there are four outcomes for the two-coin toss ($HH, HT, TH, TT$), the outcomes $HT$ and $TH$ both represent the same single event: "1 head."

Expressing Probability Values and Scale

• Notation: The probability of an event is expressed as $P(\text{event})$.

• The Probability Scale: Probability values always fall between $0$ and $1$ inclusive ($0 \le P \le 1$).

  • A probability of $0$ indicates the event is impossible.

  • A probability of $0.5$ indicates a $50/50$ chance.

  • A probability of $1$ indicates the event is certain.

The Theoretical Method for Probability

• Assumption: Applied when all outcomes are assumed to be equally likely.

• Step-by-Step Procedure:

  1. Count the total number of possible outcomes.

  2. Among all possible outcomes, count the number of ways the event of interest (Event $A$) can occur.

  3. Determine the probability using the formula:         $P(A) = \frac{\text{number of ways A can occur}}{\text{total number of outcomes}}$

• Example 1: Guessing Birthdays

  • Scenario: Selecting a person at random and determining the probability they were born in July (assuming a 365-day year and equally likely birthdays).

  • Step 1: Total outcomes = $365$ (each day is an outcome).

  • Step 2: July has $31$ days, so there are $31$ ways the event can occur.

  • Step 3: $P(\text{July Birthday}) = \frac{31}{365}$.

  • Interpretation: This is slightly more than a $1$ in $12$ chance.

Counting Outcomes and the Multiplication Principle

• Tree Diagrams: A visual tool used to show all possible outcomes. For two coins, a tree diagram shows $4$ outcomes ($2 \times 2$). For three coins, it shows $8$ outcomes ($2 \times 2 \times 2$).

• Multiplication Principle for Counting:

  • If process $A$ has $a$ possible outcomes and process $B$ has $b$ possible outcomes, and they do not affect each other, the total combined outcomes is $a \times b$.

  • This extends to further processes: If a third process $C$ has $c$ outcomes, the total is $a \times b \times c$.

• Think About It: The outcomes for tossing one coin twice in a row are identical to the outcomes for tossing two coins at the same time ($HH, HT, TH, TT$).

• Example 2: Rolling Two Dice

  • Total outcomes: Since one die has $6$ outcomes, two fair dice result in $6 \times 6 = 36$ outcomes.

  • Probability of "Snake Eyes" (two 1s): Only $1$ outcome matches this event. $P(\text{Snake Eyes}) = \frac{1}{36}$.

• Example 3: Three-Child Families

  • Scenario: Finding the probability of a specific birth order (Oldest: Boy, Second: Girl, Youngest: Girl).

  • Total outcomes ($2^3 = 8$): $BBB, BBG, BGB, BGG, GBB, GBG, GGB, GGG$.

  • Event "2 girls and 1 boy": The outcomes matching this are $BGG, GBG, GGB$.

  • Probability for two girls and one boy: $P(\text{exactly 2 girls}) = \frac{3}{8}$.

  • Scientific Caveat: In reality, births are not perfectly equally likely. There are approximately $105$ male births for every $100$ female births, though female adults outnumber male adults due to higher male death rates.

Relative Frequency and Subjective Probabilities

• Relative Frequency (Empirical) Method: Approximates probability based on many observations.

  • Step 1: Observe a process many times and count how often Event $A$ occurs.

  • Step 2: Estimate $P(A) = \frac{\text{number of times A occurred}}{\text{total number of observations}}$.

• Example 4: 500-Year Flood

  • Data: A river crests above a certain point $4$ times in the past $2,000$ years.

  • Probability: $P = \frac{4}{2000} = \frac{1}{500} = 0.002$.

  • Nomenclature: Because this occurs once every $500$ years on average, it is termed a "500-year flood."

• Subjective Method: Estimates based on experience or intuition (e.g., "I'm sure you'll like your new car"). This is the most unreliable method as it lacks calculations or systematic observation.

Comparison of Probability Approaches

• Theoretical: Based on the assumption of equally likely outcomes.

• Relative Frequency: Based on historical data, observations, or experiments.

• Subjective: Based on personal opinion or experience.

• Identification Examples:

  • "1 in 8 people move house annually" based on housing data: Relative Frequency.

  • "Chance of rolling a 7 on a 12-sided die is 1/12": Theoretical.

  • "Certain you'll be happy": Subjective.

The Complement Rule

• Definition: The event that Event $A$ does not occur is called the complement of $A$, denoted as "not $A$" or $\bar{A}$.

• Properties:

  • The sum of the probability of an event and its complement must equal $1$: $P(A) + P(\bar{A}) = 1$.

  • The probability of an event not occurring is: $P(\text{not } A) = 1 - P(A)$.

Probability Distributions

• Definition: A complete set of probability results for all possible events in a process.

• Requirements:

  1. It must represent the probabilities of all possible events.

  2. The sum of all probabilities in the distribution must exactly equal $1$.

• Formats: Can be displayed as a table, a histogram, or a mathematical formula.

• Constructing a Distribution:

  • Step 1: List all possible outcomes.

  • Step 2: Group outcomes into specific events and find each event's probability.

  • Step 3: Create a table with events in one column and probabilities in the other.

• Distribution Example (Tossing Two Coins):

  • Event: 0 heads ($TT$) | Probability: $0.25$

  • Event: 1 head ($HT, TH$) | Probability: $0.50$

  • Event: 2 heads ($HH$) | Probability: $0.25$

  • Total: $1.0$

• Distribution Example (Tossing Three Coins):

  • Total outcomes: $8$.

  • Events for # of heads: 0, 1, 2, 3.

  • 0 heads ($TTT$): $P = \frac{1}{8}$

  • 1 head ($TTH, THT, HTT$): $P = \frac{3}{8}$

  • 2 heads ($HHT, HTH, THH$): $P = \frac{3}{8}$

  • 3 heads ($HHH$): $P = \frac{1}{8}$

• Think About It: For four coins, there are $2^4 = 16$ outcomes and $5$ possible events for the number of heads (0, 1, 2, 3, or 4).

Distribution for the Sum of Two Dice

• Outcomes Table: Rolling two dice creates a $6 \times 6$ grid ($36$ outcomes).

• Event Analysis: Possible sums range from $2$ to $12$.

• Probability Calculations:

  • Sum of 2: $(1,1)$ -> $1/36$

  • Sum of 3: $(1,2), (2,1)$ -> $2/36$

  • Sum of 8: $(2,6), (3,5), (4,4), (5,3), (6,2)$ -> $5/36$

  • Sum of 12: $(6,6)$ -> $1/36$

• This distribution is symmetrical with the peak probability at sum 7 ($6/36 = 1/6$).

Odds

• Odds Against: The ratio of the probability that an event does not occur to the probability that it does occur.

  • $\text{Odds against } A = \frac{P(\bar{A})}{P(A)}$

  • Example: Odds against rolling a 6 with a fair die are $\frac{5/6}{1/6} = \frac{5}{1}$ (or 5 to 1).

• Payoff Odds: Common in gambling, these express the net gain on a winning bet.

  • Example: Payoff odds of 3 to 1 mean for every $\$1$ bet, you win $\$3$ (and retrieve the original $\$1$).