QTM- Chapter 3

Introduction to Probability
  • Definition of a Random Process: A random process is a situation in which we know what outcomes could happen, but we don't know which particular outcome will happen.

    • Examples of Random Processes:

    • Coin tosses

    • Die rolls

    • iTunes shuffle

    • Stock market fluctuations

  • Modeling Random Processes: It can be helpful to model a process as random even if it is not truly random.

  • Mathematical Rules of Probability:

    • Notation: P(A) = Probability of event A

    • Range of Probability: 0≤P(A)≤10 \leq P(A) \leq 1

  • Interpretations of Probability:

    • Frequentist Interpretation: The probability of an outcome is the proportion of times the outcome would occur if we observed the random process an infinite number of times.

    • Bayesian Interpretation: A Bayesian views probability as a subjective degree of belief. This means that for the same event, two different individuals may assign different probabilities based on their perspectives.

Objective 1: Explain the Law of Large Numbers
  • Definition: The Law of Large Numbers states that as more observations are collected, the proportion of occurrences for a particular outcome, denoted as p^n\hat{p}_n, converges to the probability of that outcome, pp.

  • Example Scenario: Consider tossing a fair coin.

    • If heads comes up on each of the first 10 tosses, the probability of getting heads on the subsequent toss remains 0.5.

    • The probability is still: P(H on 11th toss)=P(T on 11th toss)=0.5P(H \text{ on 11th toss}) = P(T \text{ on 11th toss}) = 0.5

    • Clarification: The coin is not “due” for a tail; this misconception is known as the gambler’s fallacy or the law of averages.

Objective 2: Distinguish between when events are disjoint or complement
  • Disjoint (Mutually Exclusive) Outcomes:

    • Definition: Two outcomes cannot happen at the same time.

    • Examples:

    • The outcome of a single coin toss cannot be both heads and tails.

    • A student cannot both fail and pass a class.

    • One card drawn from a deck cannot be an ace and a queen.

  • Non-Disjoint Outcomes:

    • Definition: Two outcomes can happen simultaneously.

    • Example: A student can earn an A in both Statistics and Economics in the same semester.

  • Complementary Events:

    • Definition: Complementary events are two mutually exclusive events whose probabilities sum up to 1.

    • Example: If one child is known not to be a boy, the only possibilities are {M, F} (Male, Female).

    • Sample Space: The collection of all potential outcomes.

    • For one child: S=M,FS = {M, F}

  • Distinguishing Disjoint vs. Complementary Events:

    • Disjoint Events: Do not always sum to 1 due to the possibility of multiple events in the sample space.

    • Complementary Events: Must sum to 1, as this is part of their definition (a given event either occurs or it does not, encompassing all possibilities).

Objective 3: Compare and contrast between marginal probability, joint probability, and conditional probability
  • Marginal Probability:

    • Definition: The probability of a single event occurring, typically derived from a joint distribution or a contingency table, without regard to other variables.

    • Example: In a cocaine user study, the marginal probability of a user relapsing is P(relapsed)=Number of relapsed usersTotal number of users=4872≈0.67P(relapsed) = \frac{\text{Number of relapsed users}}{\text{Total number of users}} = \frac{48}{72} \approx 0.67

  • Joint Probability:

    • Definition: The probability of two or more events occurring together.

    • Example: In the cocaine user study, the probability of a user receiving desipramine and relapsing. This is often an intermediate step in calculating conditional probabilities.

  • Conditional Probability:

    • Definition: The probability of an event (A) occurring given that another event (B) has already occurred. It is denoted as P(A∣B)P(A | B).

    • Formula: P(A∣B)=P(A and B)P(B)P(A | B) = \frac{P(A \text{ and } B)}{P(B)}

    • Example: In the cocaine user study, the probability of relapse given desipramine treatment is P(relapse∣desipramine)=Number relapsed with desipramineTotal with desipramine=1024≈0.42P(relapse | desipramine) = \frac{\text{Number relapsed with desipramine}}{\text{Total with desipramine}} = \frac{10}{24} \approx 0.42

    • Comparison: Conditional probability specifically focuses on the probability of an event under a given condition, unlike marginal probability which is overall, and joint probability which is about strict co-occurrence.

Objective 4: Recognize when to use the following rules, and how to apply them: General Addition Rule; General Multiplication Rule; Complement Rule

General Addition Rule

  • Formula: For two events A and B, the probability of A or B occurring is calculated as:

    • P(A or B)=P(A)+P(B)−P(A and B)P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)

  • Application: Used when finding the probability of either event A or event B happening. The term P(A and B)P(A \text{ and } B) is subtracted to avoid double-counting the overlap if the events are not disjoint.

  • For Disjoint Events: If A and B are disjoint, P(A and B)=0P(A \text{ and } B) = 0, simplifying the rule to:

    • P(A or B)=P(A)+P(B)P(A \text{ or } B) = P(A) + P(B)

General Multiplication Rule

  • Formula: For two events A and B, the probability of both A and B occurring is:

    • P(A and B)=P(A)×P(B∣A)P(A \text{ and } B) = P(A) \times P(B | A)

    • Alternatively: P(A and B)=P(B)×P(A∣B)P(A \text{ and } B) = P(B) \times P(A | B)

  • Application: Used when finding the probability of both event A and event B happening. It's especially useful when events are dependent, as it accounts for how one event influences the probability of the other.

  • For Independent Events (Product Rule): If A and B are independent, knowing A doesn't change B's probability (P(B∣A)=P(B)P(B | A) = P(B)), simplifying to:

    • P(A and B)=P(A)×P(B)P(A \text{ and } B) = P(A) \times P(B)

    • More generally, for multiple independent events: P(A1, A2, \ldots, Ak) = P(A1) \times P(A2) \times \ldots \times P(Ak)</p></li></ul></li></ul><p>ComplementRule</p><ul><li><p><strong>Formula</strong>:TheprobabilityofaneventAoccurringis1minustheprobabilityofitscomplement,</p></li></ul></li></ul><p>Complement Rule</p><ul><li><p><strong>Formula</strong>: The probability of an event A occurring is 1 minus the probability of its complement,A^c(Anotoccurring):</p><ul><li><p>(A not occurring):</p><ul><li><p>P(A) = 1 - P(A^c)$$

  • Application: This rule is particularly useful when it's easier to calculate the probability of an event not happening than to calculate the probability of it happening directly. Common scenarios include