QTM- Chapter 3
Introduction to Probability
Definition of a Random Process: A random process is a situation in which we know what outcomes could happen, but we don't know which particular outcome will happen.
Examples of Random Processes:
Coin tosses
Die rolls
iTunes shuffle
Stock market fluctuations
Modeling Random Processes: It can be helpful to model a process as random even if it is not truly random.
Mathematical Rules of Probability:
Notation: P(A) = Probability of event A
Range of Probability:
Interpretations of Probability:
Frequentist Interpretation: The probability of an outcome is the proportion of times the outcome would occur if we observed the random process an infinite number of times.
Bayesian Interpretation: A Bayesian views probability as a subjective degree of belief. This means that for the same event, two different individuals may assign different probabilities based on their perspectives.
Objective 1: Explain the Law of Large Numbers
Definition: The Law of Large Numbers states that as more observations are collected, the proportion of occurrences for a particular outcome, denoted as , converges to the probability of that outcome, .
Example Scenario: Consider tossing a fair coin.
If heads comes up on each of the first 10 tosses, the probability of getting heads on the subsequent toss remains 0.5.
The probability is still:
Clarification: The coin is not “due” for a tail; this misconception is known as the gambler’s fallacy or the law of averages.
Objective 2: Distinguish between when events are disjoint or complement
Disjoint (Mutually Exclusive) Outcomes:
Definition: Two outcomes cannot happen at the same time.
Examples:
The outcome of a single coin toss cannot be both heads and tails.
A student cannot both fail and pass a class.
One card drawn from a deck cannot be an ace and a queen.
Non-Disjoint Outcomes:
Definition: Two outcomes can happen simultaneously.
Example: A student can earn an A in both Statistics and Economics in the same semester.
Complementary Events:
Definition: Complementary events are two mutually exclusive events whose probabilities sum up to 1.
Example: If one child is known not to be a boy, the only possibilities are
{M, F}(Male, Female).Sample Space: The collection of all potential outcomes.
For one child:
Distinguishing Disjoint vs. Complementary Events:
Disjoint Events: Do not always sum to 1 due to the possibility of multiple events in the sample space.
Complementary Events: Must sum to 1, as this is part of their definition (a given event either occurs or it does not, encompassing all possibilities).
Objective 3: Compare and contrast between marginal probability, joint probability, and conditional probability
Marginal Probability:
Definition: The probability of a single event occurring, typically derived from a joint distribution or a contingency table, without regard to other variables.
Example: In a cocaine user study, the marginal probability of a user relapsing is
Joint Probability:
Definition: The probability of two or more events occurring together.
Example: In the cocaine user study, the probability of a user receiving desipramine and relapsing. This is often an intermediate step in calculating conditional probabilities.
Conditional Probability:
Definition: The probability of an event (A) occurring given that another event (B) has already occurred. It is denoted as .
Formula:
Example: In the cocaine user study, the probability of relapse given desipramine treatment is
Comparison: Conditional probability specifically focuses on the probability of an event under a given condition, unlike marginal probability which is overall, and joint probability which is about strict co-occurrence.
Objective 4: Recognize when to use the following rules, and how to apply them: General Addition Rule; General Multiplication Rule; Complement Rule
General Addition Rule
Formula: For two events A and B, the probability of A or B occurring is calculated as:
Application: Used when finding the probability of either event A or event B happening. The term is subtracted to avoid double-counting the overlap if the events are not disjoint.
For Disjoint Events: If A and B are disjoint, , simplifying the rule to:
General Multiplication Rule
Formula: For two events A and B, the probability of both A and B occurring is:
Alternatively:
Application: Used when finding the probability of both event A and event B happening. It's especially useful when events are dependent, as it accounts for how one event influences the probability of the other.
For Independent Events (Product Rule): If A and B are independent, knowing A doesn't change B's probability (), simplifying to:
More generally, for multiple independent events: P(A1, A2, \ldots, Ak) = P(A1) \times P(A2) \times \ldots \times P(Ak)A^cP(A) = 1 - P(A^c)$$
Application: This rule is particularly useful when it's easier to calculate the probability of an event not happening than to calculate the probability of it happening directly. Common scenarios include