Basic Probability Lecture Notes

Introduction to Uncertainty and Probability

• Uncertainty is a fundamental aspect of making decisions with incomplete information, reflecting the way we generally operate in the world.

• Everyday concepts used to describe the handling of uncertainty include chance, luck, and risk.

• Probability is a branch of mathematics providing the language and tools necessary to quantify the uncertainty of events and reason in a principled manner.

Instances of Uncertainty

• Common activities involving uncertain outcomes include flipping a coin, crossing the road without looking, or rolling a die.

• These outcomes require weighing uncertainties against one another.

• In a coin flip, there is no reason to expect heads appearing more often than tails.

• If one crosses the road without looking, there is a risk of being squashed by a truck; looking before crossing significantly reduces the likelihood of this outcome.

Defining Probability and Experiments

• Probability acts as the machinery used to describe and account for the fact that some outcomes occur more frequently than others.

• It is a measure that quantifies the likelihood of an event occurring.

• An Experiment is defined as any activity from which results are obtained.

• A Random Experiment is one where results or outcomes cannot be predicted with total certainty. An example of a random experiment is flipping a coin and observing whether a head or tail appears.

Sample Space and Outcomes

• An Outcome is the specific result of a single trial in a probability experiment.

• The Sample Space, represented by the symbol $S$, is the set of all possible outcomes for an experiment.

• Each individual outcome within a sample space is referred to as an element, a member, or a sample point.

Examples of Sample Spaces

• Rolling a Die: The sample space consists of six outcomes:     $S = \{1, 2, 3, 4, 5, 6\}$

• Tossing a Coin Twice: There are four possible results based on the first and second flips:     $S = \{HH, HT, TH, TT\}$

• Manufacturing Selection: If 3 items are randomly selected and classified as defective ($D$) or nondefective ($N$), a tree diagram can be used to list the elements of the sample space:     $S = \{DDD, DDN, DND, DNN, NDD, NDN, NND, NNN\}$

Types of Sample Spaces

• Finite Sample Space: A space containing a fixed, finite number of points, such as $\{1, 2, 5, 6\}$.

• Countably Infinite Sample Space (Discrete): A space that has as many points as there are natural numbers ($1, 2, 3, \dots$).

• Noncountable Infinite Sample Space (Nondiscrete): A space containing as many points as there are in an interval on the $x$ axis, such as $0 \le x \le 1$.

Events

• An Event is a subset $A$ of the sample space $S$, representing a set of possible outcomes.

• A Simple Event is an event consisting of only a single outcome.

Examples of Events

• In a die toss, the event that an outcome is divisible by 3 occurs if the outcome belongs to the subset $A = \{3, 6\}$.

• In the manufacturing selection experiment of 3 items, the event that the number of defectives is greater than 1 occurs if the outcome is in the subset:     $B = \{DDN, DND, NDD, DDD\}$

Classical and Empirical Probability

Classical Probability

• Also known as theoretical probability, it is used when every outcome in a sample space is equally likely to occur.

• The formula for the classical probability of event $E$ is:     $P(E) = \frac{\text{Number of outcomes in event}}{\text{Total number of outcomes in sample space}}$

• Example: Rolling a die to find the probability of rolling a 5 ($Event A$). Since there is only one 5 in the set $\{5\}$, the calculation is:     $P(A) = \frac{1}{6} \approx 0.167$

Empirical Probability

• Also known as statistical probability, it is based on actual observations from probability experiments.

• The empirical probability of an event $E$ is its relative frequency:     $P(E) = \frac{\text{Frequency of Event (f)}}{\text{Total frequency (n)}}$

• Example: A travel agent finds that in every 50 reservations, 12 are for a cruise. The probability that the next reservation is a cruise is:     $P(\text{cruise}) = \frac{12}{50} = 0.24$

Null Sets and Set Operations

• A Null Set ($\phi$) contains no elements. For example, if $B = \{x \mid x \text{ is an even factor of 7}\},$ $B$ is a null set because the factors of 7 (1 and 7) are odd.

Core Set Operations

• Complement: The complement of event $A$ ($A'$) consists of all elements in $S$ that are not in $A$.

  • Example: If $S = \{\text{book, cell phone, mp3, paper, stationery, laptop}\}$ and $A = \{\text{book, stationery, laptop, paper}\},$ then $A' = \{\text{cell phone, mp3}\}$.

• Intersection: Denoted by $A \cap B$, it is the event containing elements common to both $A$ and $B$.

  • Example: If $E$ is the event of engineering majors and $F$ is the event of females, $E \cap F$ is the set of all female engineering students.

• Mutually Exclusive (Disjoint): Events $A$ and $B$ are mutually exclusive if they have no common elements, denoted as $A \cap B = \phi$.

• Union: Denoted by $A \cup B$, it is the event containing elements belonging to $A$, or $B$, or both.

• Difference: The difference of events $A$ and $B$ contains all outcomes included in $A$ but excluded from $B$.

Venn Diagram Relationships

• $A \cap B$: Regions common to both.

• $B \cap C$: Regions common to $B$ and $C$.

• $A \cup C$: Regions in either $A$ or $C$.

• $B' \cap A$: Regions in $A$ that are not in $B$.

• $A \cap B \cap C$: The region common to all three.

• $(A \cup B) \cap C'$: Regions in the union of $A$ and $B$ that are not in $C$.

Operational Rules and Axioms

Logical Rules for Sets

1. $A \cap \phi = \phi$

2. $A \cup \phi = A$

3. $A \cap A' = \phi$

4. $A \cup A' = S$

5. $S' = \phi$

6. $\phi' = S$

7. $(A')' = A$

8. $(A \cap B)' = A' \cup B'$

9. $(A \cup B)' = A' \cap B'$

Axioms of Probability

• Axiom 1: For every event $A$, $P(A) \ge 0$.

• Axiom 2: For the certain event $S$, $P(S) = 1$.

• Axiom 3: For any number of mutually exclusive events $A_1, A_2, \dots, A_n$:     $P(A_1 \cup A_2 \cup \dots \cup A_n) = P(A_1) + P(A_2) + \dots + P(A_n)$

• For two mutually exclusive events $A_1, A_2$:     $P(A_1 \cup A_2) = P(A_1) + P(A_2)$

Probability Theorems

• Theorem 1: If $A \subseteq B$, then $P(A) \le P(B)$ and $P(B - A) = P(B) - P(A)$.

• Theorem 2: For every event $A$, $0 \le P(A) \le 1$.

• Theorem 3: The impossible event has zero probability: $P(\phi) = 0$.

• Theorem 4: $P(A') = 1 - P(A)$.

• Theorem 5 (Addition Rule): For any two events $A$ and $B$:     $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

• Theorem 6 (Triple Union Rule): For any three events $A_1, A_2, A_3$:     $P(A_1 \cup A_2 \cup A_3) = P(A_1) + P(A_2) + P(A_3) - P(A_1 \cap A_2) - P(A_2 \cap A_3) - P(A_3 \cap A_1) + P(A_1 \cap A_2 \cap A_3)$

Worked Examples

Tossing a Coin Twice

Requirement: Find the probability of at least 1 head.

• $S = \{HH, HT, TH, TT\}$

• Assign a probability $\omega$ to each equally likely point: $4\omega = 1 \implies \omega = 1/4$.

• Let $A$ be the event of at least 1 head: $A = \{HH, HT, TH\}$.

• $P(A) = \frac{1}{4} + \frac{1}{4} + \frac{1}{4} = \frac{3}{4}$

Loaded Die

Requirement: A die is loaded so an even number is twice as likely as an odd number. Find $P(E)$ where $E$ is a number less than $4$.

• $S = \{1, 2, 3, 4, 5, 6\}$

• Assign $\omega$ to odds (1, 3, 5) and $2\omega$ to evens (2, 4, 6).

• Total probability: $\omega + 2\omega + \omega + 2\omega + \omega + 2\omega = 9\omega = 1 \implies \omega = 1/9$.

• $P(\text{odd}) = 1/9$; $P(\text{even}) = 2/9$.

• $E = \{1, 2, 3\} \implies P(E) = \frac{1}{9} + \frac{2}{9} + \frac{1}{9} = \frac{4}{9}$.

• Extended Example: For $A = \{\text{even}\} = \{2, 4, 6\}$ and $B = \{\text{divisible by 3}\} = \{3, 6\}$.

  • $A \cup B = \{2, 3, 4, 6\} \implies P(A \cup B) = \frac{2}{9} + \frac{1}{9} + \frac{2}{9} + \frac{2}{9} = \frac{7}{9}$.

  • $A \cap B = \{6\} \implies P(A \cap B) = \frac{2}{9}$.

Job Offers (John)

Requirement: Probability of an offer from Company A is 0.8, Company B is 0.6, and both is 0.5. Find probability of at least one offer.

• $P(A) = 0.8$

• $P(B) = 0.6$

• $P(A \cap B) = 0.5$

• $P(A \cup B) = 0.8 + 0.6 - 0.5 = 0.9$

Pairs of Dice

Requirement: Find the probability of a total of 7 or 11.

• Total sample points = 36.

• Event $A$ (Total 7): 6 points. $P(A) = 6/36 = 1/6$.

• Event $B$ (Total 11): 2 points. $P(B) = 2/36 = 1/18$.

• Since they are mutually exclusive: $P(A \cup B) = \frac{1}{6} + \frac{1}{18} = \frac{4}{18} = \frac{2}{9}$.

• Alternatively: $n/N = 8/36 = 2/9$.

Car Color Preference

Requirement: Probabilities for green (0.09), white (0.15), red (0.21), and blue (0.23). Find probability of one of these colors.

• Events are mutually exclusive.

• $P(G \cup W \cup R \cup B) = 0.09 + 0.15 + 0.21 + 0.23 = 0.68$

Mechanic Servicing Cars

Requirement: Probabilities for servicing 3, 4, 5, 6, 7, or 8+ cars are 0.12, 0.19, 0.28, 0.24, 0.10, and 0.07. Find the probability of servicing at least 5 cars.

• Let $E$ be the event of servicing at least 5 cars.

• $E'$ is the event of servicing fewer than 5 cars (servicing 3 or 4 cars).

• $P(E') = 0.12 + 0.19 = 0.31$

• $P(E) = 1 - P(E') = 1 - 0.31 = 0.69$