Comprehensive Notes on Sample Spaces, Events, and Axiomatic Probability

Sample Spaces and Classification

• Sample Space Concept & Fundamental Examples

  • Example 1.2 (Repeated Coin Toss): If a coin is tossed repeatedly until a head occurs, the natural sample space is:
    • $S = \{H, TH, TTH, \dots\}$
  • If interest centers strictly on the number of tosses required to obtain a head, a possible sample space for this experiment is the set of all positive integers:
    • $S^* = \{1, 2, 3, \dots\}$
  • The outcomes in $S^*$ correspond directly to the number of tosses required to obtain the first head.
  • Example 1.3 (Light Bulb Lifespan): A light bulb is placed in service and the time of operation until it burns out is measured.
  • Conceptually, the sample space for this experiment is the set of nonnegative real numbers:
    • $S = \{t : 0 \le t < \infty\}$
  • If the actual failure time is measured only to the nearest hour, the sample space for the observed failure time is the set of nonnegative integers:
    • $S^* = \{0, 1, 2, 3, \dots\}$

• Classification of Sample Spaces

  • Finite Sample Space: A sample space $S$ is finite if it consists of a finite number of outcomes:
    • $S = \{e_1, e_2, \dots, e_N\}$
  • Countably Infinite Sample Space: A sample space $S$ is countably infinite if its outcomes can be put into a one-to-one correspondence with the positive integers:
    • $S = \{e_1, e_2, \dots\}$
  • Discrete Sample Space: A sample space $S$ is defined as a discrete sample space if it is either finite or countably infinite (i.e., countable).
    • The coin toss experiment ($S = \{1, 2, 3, \dots\}$) and the light bulb failure time rounded to the nearest hour ($S^* = \{0, 1, 2, 3, \dots\}$) are examples of discrete sample spaces.
  • Continuous Sample Space: A sample space is continuous if it consists of a continuum, such as all the points of a line segment or all the points in a plane (representing an uncountable infinite number of outcomes).
    • A discrete sample space is not an appropriate model when outcomes can assume any value in a real interval.
    • Continuous sample spaces arise in practice whenever experimental outcomes are measurements of physical properties measured on continuous scales, such as temperature, speed, pressure, and length.
    • The conceptual light bulb failure time ($S = \{t : 0 \le t < \infty\}$) is an example of a continuous sample space.

Events and Event Operations

• Definition of an Event
  • An event $A$ is a collection of some of the possible outcomes of a random experiment. Expressed in set theory notation, $A$ is a subset of the sample space $S$:
    • $A \subseteq S$
  • Elementary Events: Individual outcomes can each be regarded as an event, because each outcome constitutes a subset of $S$.
  • Impossible Event: The null set $\phi$ (the set containing no outcomes) is a subset of $S$ and is defined as the impossible event.
  • Certain Event: The complete sample space $S$ itself is a subset of $S$ and is defined as the certain event.
  • Die Roll Examples: In a die-tossing experiment with sample space $S = \{1, 2, 3, 4, 5, 6\}$, typical events include:
    • $A_1$: Score is 1, i.e., $A_1 = \{1\}$
    • $A_2$: Score is even, i.e., $A_2 = \{2, 4, 6\}$
    • $A_3$: Score is less than 5, i.e., $A_3 = \{1, 2, 3, 4\}$

• Occurrence of an Event
  • An event $A$ is said to have occurred if the outcome of the experiment belongs to the set A$.\n * In the die roll example, if the outcome is a score of 6, then event A_2$occurred, while$A_1$did not occur and$A_3 did not occur.\n\n* **Set Operations and Event Relationships**\n * **Union of Events (A \cup B$)**: Given two events$A$and$B$, the union$A \cup B$is the set of outcomes belonging to$A$,$B$, or both. The event$A \cup B$occurs whenever$A$occurs,$B occurs, or both occur.\n * **Intersection of Events (A \cap B$)**: Given two events$A$and$B$, the intersection$A \cap B$is the set of outcomes belonging to both$A$and$B$. The event$A \cap B$occurs when both$A$and$B occur simultaneously.\n * **Mutually Exclusive Events**: Two events A$and$B are mutually exclusive if both cannot occur at the same time:\n * A \cap B = \phi\n * Mutually exclusive events are disjoint sets.\n * **Complementary Event (\bar{A}$or$A'$)**: Given an event$A$, the complementary event$\bar{A}$(or$A'$) is the subset of$S$containing all outcomes that do not belong to$A$. The event$\bar{A}$occurs whenever$A does not occur and vice versa.\n * **Equally Likely Events**: Outcomes of a random experiment are defined as equally likely if no outcome is any more likely to occur than any other.\n\n\n\n# Approaches to Probability\n\n* **Frameworks of Objective Probability**\n * Probability can be categorized as an objective phenomenon derived from objective processes under two main headings:\n 1. Classical, or *a priori*, probability.\n 2. Relative frequency, or *a posteriori*, probability.\n\n* **Classical (a priori) Probability**\n * **Definition**: If a random experiment can result in N$mutually exclusive and equally likely outcomes,$n(A)$of which correspond to the occurrence of event$A$, the probability$P(A) is defined as the ratio:\n * P(A) = \frac{n(A)}{N} = \frac{\text{number of outcomes belonging to } A}{\text{total number of outcomes belonging to } S}\n\n* **Relative Frequency (a posteriori) Probability**\n * **Definition**: Depends on the repeatability of a process and the ability to count repetitions as well as the occurrences of an event.\n * If a process is repeated a large number of times N$, and an event with characteristic$A$occurs$n$times, the relative frequency of occurrence$\frac{n}{N}$is approximately equal to the probability of$A:\n * P(A) \approx \frac{n}{N}\n * **Fair Die Application**: In throwing a fair (unbiased) die, there are 6 outcomes. Reasoning establishes that outcomes are equally likely because no face is more likely to turn up than any other. On any throw, exactly one face turns up, making the outcomes mutually exclusive.\n\n* **Classical Probability Worked Examples**\n * **Example 1.4 (Coin Tossing Sequence)**: An unbiased coin is thrown three times, and the sequence of heads and tails is observed.\n * Sample Space: S = {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}$, total outcomes$N = 8\n * Since outcomes are equally likely and mutually exclusive, the probability of each outcome is \frac{1}{8}.\n * Let A$be the event that two or more heads appear consecutively:$A = {HHH, HHT, THH}$,$n(A) = 3\n * Let B$be the event that all tosses are the same:$B = {HHH, TTT}$,$n(B) = 2\n * P(A) = \frac{n(A)}{N} = \frac{3}{8}\n * P(B) = \frac{n(B)}{N} = \frac{2}{8} = \frac{1}{4}\n * Intersection A \cap B = {HHH}:\n * P(A \cap B) = \frac{n(A \cap B)}{N} = P({HHH}) = \frac{1}{8}\n * Union A \cup B = {HHH, HHT, THH, TTT}:\n * P(A \cup B) = \frac{n(A \cup B)}{N} = P({HHH, HHT, THH, TTT}) = \frac{4}{8} = \frac{1}{2}\n * **Example 1.5 (Selection of Numbered Balls)**: There are 15 balls, numbered 1 to 15, in a bag. One ball is selected at random (N = 15).\n * *Part i*: Probability that the number printed on the ball is a prime number greater than 5.\n * A_1 = {7, 11, 13}$,$n(A_1) = 3\n * P(A_1) = \frac{n(A_1)}{N} = \frac{3}{15} = \frac{1}{5}\n * *Part ii*: Probability that the number printed on the ball is an odd number less than 9.\n * A_2 = {1, 3, 5, 7}$,$n(A_2) = 4\n * P(A_2) = \frac{n(A_2)}{N} = \frac{4}{15}\n * *Intersection (A_1 \cap A_2$)*:$A_1 \cap A_2 = {7}$,$n(A_1 \cap A_2) = 1\n * P(A_1 \cap A_2) = \frac{n(A_1 \cap A_2)}{N} = \frac{1}{15}\n * *Union (A_1 \cup A_2$)*:$A_1 \cup A_2 = {1, 3, 5, 7, 11, 13}$,$n(A_1 \cup A_2) = 6\n * P(A_1 \cup A_2) = \frac{n(A_1 \cup A_2)}{N} = \frac{6}{15} = \frac{2}{5}\n * **Example 1.6 (Tossing Two Fair Dice)**: A fair die is tossed twice. What is the probability that the sum of the upturned faces is 9?\n * Sample Space: S = {(i, j) : i = 1, 2, \dots, 6; j = 1, 2, \dots, 6}$, total outcomes$N = 36\n * Each of the 36 outcomes is equally likely.\n * Let A$represent the event that the sum of upturned faces is 9:$A = {(3, 6), (4, 5), (5, 4), (6, 3)}$,$n(A) = 4\n * P(A) = \frac{4}{36} = \frac{1}{9}\n\n# Axiomatic Foundations of Probability\n\n* **Probability Function / Measure Definition**\n * For a given experiment with sample space S$, a set real function that associates a real value$P(A)$with each event$A$is called a probability function (or probability measure), and$P(A)$is called the probability of$A, if it satisfies three axioms.\n\n* **Axioms of Probability**\n * **Axiom I**: P(A) \ge 0$, for every event$A \subseteq S\n * **Axiom II**: P(S) = 1\n * **Axiom III**: If A_1, A_2, A_3, \dots$is a finite or infinite sequence of mutually exclusive events of$S, then:\n * P(A_1 \cup A_2 \cup A_3 \cup \dots) = P(A_1) + P(A_2) + P(A_3) + \dots\n\n* **Special Case for Two Mutually Exclusive Events**\n * If A$and$B$are two mutually exclusive events of$S, Axiom III simplifies to:\n * P(A \cup B) = P(A) + P(B)\n\n* **Worked Example 1.7 (Horse Race Probabilities)**\n * **Problem**: Three horses A, B, and C are in a race. A is twice as likely to win as B, and B is twice as likely to win as C. Find their respective winning probabilities P(A)$,$P(B)$, and$P(C).\n * **Solution**:\n * Let P(C) = p\n * Since B is twice as likely to win as C, P(B) = 2p\n * Since A is twice as likely to win as B, P(A) = 2 P(B) = 4p\n * By Axiom II, the sum of the probabilities over the sample space must be 1:\n * P(A) + P(B) + P(C) = 1\n * 4p + 2p + p = 1\n * 7p = 1 \implies p = \frac{1}{7}\n * Therefore:\n * P(A) = 4p = \frac{4}{7}\n * P(B) = 2p = \frac{2}{7}\n * P(C) = p = \frac{1}{7}\n\n# Probability in Discrete Spaces\n\n* **Probability Mass Assignment**\n * If A$is an event in a discrete sample space$S$, then$P(A)$equals the sum of the probabilities of the individual outcomes comprising$A$.
  • Given a sample space $S = \{e_1, e_2, e_3, \dots\}$, assign to each elementary event $\{e_i\}$ a real number $p_i$ such that $P(\{e_i\}) = p_i$.
  • The probabilities $p_i$ must satisfy:
    • $p_i \ge 0$ for all $i$
    • $\sum_i p_i = 1$
  • The probability of an event $A$ is calculated as:
    • $P(A) = \sum_{i : e_i \in A} p_i$
    • The summation is taken over all indices $i$ such that outcome $e_i$ is in A$.\n\n* **Worked Example 1.8 (Loaded Die)**\n * **Problem**: A die is loaded in such a way that each odd number is twice as likely to occur as each even number. Find P(G)$, where$G is the event that a number greater than 3 occurs on a single roll of the die.\n * **Solution**:\n * Sample Space S = {1, 2, 3, 4, 5, 6}\n * Let w$be the probability of getting a specific even number:$P({2}) = P({4}) = P({6}) = w\n * Each odd number is twice as likely as an even number: P({1}) = P({3}) = P({5}) = 2w\n * Applying the normalization condition \sum_i p_i = 1:\n * P({1}) + P({2}) + P({3}) + P({4}) + P({5}) + P({6}) = 1\n * 2w + w + 2w + w + 2w + w = 1\n * 9w = 1 \implies w = \frac{1}{9}\n * The individual outcome probabilities are:\n * P({1}) = P({3}) = P({5}) = \frac{2}{9}\n * P({2}) = P({4}) = P({6}) = \frac{1}{9}\n * The event G$(rolling a number greater than 3) is$G = {4, 5, 6}.\n * Summing probabilities for outcomes in G:\n * P(G) = P({4}) + P({5}) + P({6}) = \frac{1}{9} + \frac{2}{9} + \frac{1}{9} = \frac{4}{9}$$