Probability and Statistics Notes
Probability Intro: Sample Space and Events
STAT 330 Iowa State University
Outline
- In this lecture students will:
- Be introduced to Probability and Statistics
- Begin Probability Basics
- 2.1 Random Experiments
- 2.2 Outcomes and Sample Space
- 2.3 Events
Introduction
Probability and Statistics Definitions
- Probability: A mathematical theory for modeling processes where outcomes occur randomly.
- Statistics: Learning about the real world from data under the assumption that the data was generated by a random process.
Goals
- Probability: Model and assign probabilities to outcomes.
- Statistics: Use probability models to draw conclusions and learn.
Random Experiment
Random Experiment Definition
- A random experiment is an experiment (or process) for which the outcome cannot be predicted with certainty.
Example 1: Various random experiments
- A message can take two network routers to reach a recipient computer. We record the status of router 1, the status of router 2, and the status of the recipient computer, where the status is either up (U) or down (D).
- Record the time for a web page to respond.
- Roll a die and record the face up.
- Flip a coin until you get a head. Record all the faces that you obtain.
Components of Random Experiment
Outcome Definition
- The outcome ($ extbf{!}$) is the result of an experiment.
Example 2: Outcomes
- Network Routers: $ extbf{!} = ( ext{router 1 down}, ext{router 2 down}, ext{recipient computer up}) = ext{DDU}$
- Access web page: $ extbf{!} = ext{response time in seconds}$
- Roll a die: $ extbf{!} = ext{face up}$
- Toss coin until head: $ extbf{!} = ext{sequence of faces obtained}$
Sample Space Definition
- The sample space ($oldsymbol{ ext{Ω}}$) is the set of ALL possible outcomes.
Example 3: Sample Spaces
- Network Routers: $Ω = ext{{DDD, DDU, DUD, UDD, UUD, UDU, DUU, UUU}}$
- Access web page: $Ω = ext{(time in seconds)}$
- Roll a die: $Ω = ext{{1, 2, 3, 4, 5, 6}}$
- Toss coin until head: $Ω = ext{{H, TH, TTH, TTTH, …}}$
- The size of the sample space, denoted $|oldsymbol{ ext{Ω}}|$, is the number of outcomes in the sample space. For the network routers example, $|oldsymbol{ ext{Ω}}| = 8$.
Types of Sample Space
- Sample space can be:
- finite (discrete)
- countably infinite (discrete)
- uncountable infinite (not discrete)
Example 4: Discrete/Not Discrete Sample Spaces
- Network Routers: $Ω = ext{{DDD, DDU, DUD, UDD, UUD, UDU, DUU, UUU}}$ (discrete)
- Access web page: $Ω = ext{(0, 1)}$ (not discrete)
- Roll a die: $Ω = ext{{1, 2, 3, 4, 5, 6}}$ (discrete)
- Toss coin until head: $Ω = ext{{H, TH, TTH, TTTH, …}}$ (discrete)
Event Definition
- An event $(A, B, C, ext{etc.})$ is a collection of outcomes from the sample space that we are interested in. An event is a subset of the sample space: $A ⊆ oldsymbol{ ext{Ω}}$.
Example 5: Events
- Network Routers: A message is transmitted successfully if at least one router is up and the recipient computer is up. $A = ext{successful transmission} = ext{{DUU, UDU, UUU}}$
- Access web page: $B = ext{More than 10 seconds} = (10, ext{∞})$
- Roll a die: $C = ext{even outcome} = ext{{2, 4, 6}}$
- Toss coin until head: $D = ext{sequence of coin tosses until head is obtained}$
Probability
Definition of Probability
- Let $A$ be an event (set of outcomes from the sample space). Then, the probability of event A is written as $P(A)$.
Example 6:
- Consider event $C$ as successful transmission in the network router example. Suppose the chance that a message is successfully transmitted is 90%.
- $P(C) = 0.9$
- To calculate probability of events, start with understanding set theory.
Recap
- Students should now be familiar with basic concepts and terms that serve as the starting point for beginning probability:
- Random Experiments
- Outcomes and Sample Space
- Events
Event Operations
STAT 330 Iowa State University
Outline
- In this lecture, students will learn event operations to combine events into new events. These will be used when finding probabilities for various events.
Set Theory
Set Notation Review
- Symbols: $∈, ∉, ⊆, ⊂, ∩, ∪$.
- If $x$ is an element of set $B$, this is denoted $x ∈ B$.
- If $y$ is not an element of set $B$, this is denoted $y ∉ B$.
- If every element of set $A$ is also an element of set $B$, then $A$ is a subset of $B$: $A ⊆ B$.
Union
- Let $A$ and $B$ be two events.
- Union ($igcup$): $A igcup B$ is the event consisting of all outcomes in $A$ or in $B$ or in both.
- $A igcup B = ig{ extbf{!} | extbf{!} ∈ A ext{ or } extbf{!} ∈ B}$
Intersection
- Intersection ($igcap$): $A igcap B$ is the event consisting of all outcomes simultaneously in $A$ and in $B$.
- $A igcap B = ig{ extbf{!} | extbf{!} ∈ A ext{ and } extbf{!} ∈ B}$
Complement
- The complement of an event $A$ ($ar{A}$) is the event consisting of all outcomes not in $A$.
- $ar{A} = ig{ extbf{!} | extbf{!} ∉ A}$
De Morgan’s Laws
- De Morgan’s laws:
- $ar{(A igcup B)} = ar{A} igcap ar{B}$
- $ar{(A igcap B)} = ar{A} igcup ar{B}$
Empty Set
- Empty Set ($∅$): $∅$ is a set containing no elements, usually denoted by $ ext{{}}$. The empty set is a subset of every set: $∅ ⊆ A$.
Disjoint/Mutually Exclusive Sets
- Sets $A$ and $B$ are disjoint if their intersection is empty: $A igcap B = ∅$.
- Pairwise disjoint sets: Sets $A1, A2, A3, …$ are pairwise disjoint if $Ai igcap A_j = ∅$ for any $i ≠ j$.
Example 7
- Let $oldsymbol{ ext{Ω}} = ext{{1, 2, 3, 4, 5}}$, $A = ext{{1, 2, 3}}$, $B = ext{{2, 3, 4}}$, $C = ext{{4, 5}}$
- $ar{A} = ext{{4, 5}}$
- $A igcup B = ext{{1, 2, 3, 4}}$
- $A igcap B = ext{{2, 3}}$
- $A igcap C = ∅$
- Are $A$ and $B$ disjoint? No
- Are $A$ and $C$ disjoint? Yes
- Are $A, B, C$ pairwise disjoint? No
Recap
- Students should now be familiar with event operations to form new events from given events.
- Students should be familiar with Venn Diagrams as a way to visualize event operations.
Probability Axioms
STAT 330 Iowa State University
Outline
- In this lecture students will:
- Learn the three probability axioms that all probability models must follow.
- See useful consequences of the axioms for finding probabilities.
Kolmogorov's Axioms
Kolmogorov’s Axioms
- Recall: $P(A)$ is the probability that event A occurs.
- We want to assign probabilities to events as a measure of their likelihood of occurring.
- A probability model is an assignment of numbers $P(A)$ to events $A ⊆ oldsymbol{ ext{Ω}}$ such that Kolmogorov’s axioms are satisfied.
Kolmogorov’s Axioms
- $0 ≤ P(A) ≤ 1$ for all $A$.
- $P(oldsymbol{ ext{Ω}}) = 1$.
- If $A1, A2, A_3, …$ are pairwise disjoint, then
- $P(A1 igcup A2 igcup …) = P(A1) + P(A2) + … = extstyle igsum P(A_i)$.
Kolmogorov’s Axioms Cont.
- Kolmogorov’s axioms provide the logical framework that probability assignment must follow.
- However, the axioms do not specify what probabilities to assign to events.
Example 8
- Draw a single card from a standard deck of playing cards:
- $oldsymbol{ ext{Ω}} = ext{{red, black}}$.
- Two different probability models are:
- Model 1:
- $P(oldsymbol{ ext{Ω}}) = 1$, $P( ext{red}) = 0.5$, $P( ext{black}) = 0.5$.
- Model 2:
- $P(oldsymbol{ ext{Ω}}) = 1$, $P( ext{red}) = 0.3$, $P( ext{black}) = 0.7$.
- Both are valid probability models, but real-world experience tells us Model 1 is more accurate for the scenario.
Consequences of Kolmogorov’s Axioms
Let $A, B ⊆ oldsymbol{ ext{Ω}}$.
Probability of the Complementary Event:
- $P(A) = 1 - P(ar{A})$.
- Corollary: $P(∅) = 0$.
Addition Rule of Probability:
- $P(A igcup B) = P(A) + P(B) - P(A igcap B)$.
If $A ⊆ B$, then $P(A) ≤ P(B)$.
- Corollary: For any $A$, $P(A) ≤ 1$.
Recap
- Students should now be familiar with the three axioms that all probability models must follow.
- Students should also know some useful consequences that will be used going forward.
Assigning Probabilities
STAT 330 Iowa State University
Outline
- In this lecture students will learn two ways to begin assigning probabilities to events using the probability axioms.
Assigning Probabilities
- There are 2 main approaches to assign probabilities to events at this point.
When we know events are disjoint (easy!):
- Let $A$ be a collection of $k$ outcomes ($ extbf{!}1,…, extbf{!}k$) that are all pairwise disjoint.
- Use Kolmogorov’s axiom 3:
- $P(A) = P(igcup{i=1}^k extbf{!}i) = extstyle igsum{i=1}^{k} P( extbf{!}i)$.
Example 9:
- Roll a die. Suppose event $A$ is rolling an even number. (Assume all numbers are equally likely, $P( extbf{!}) = rac{1}{6}$ for all $ extbf{!}$)
- $Ω = ext{{1, 2, 3, 4, 5, 6}}$, $A = ext{{2, 4, 6}}$.
- $P(A) = P( ext{