Notes on Sample Space, Sets, and Probability

Sample Space and Probability (Chapter 1)

  • Sample Space: A sample space is the set of all possible outcomes of an experiment.

Sets

  • Union and Intersection:

    • Union: ( S \cup T = {x; x \in S \text{ or } x \in T} )
    • Intersection: ( S \cap T = {x; x \in S \text{ and } x \in T} )

  • Example:

    • If ( S = {♣, ♦, ♥} ) and ( T = {♦, ♥, ♠} ), then:
      • ( S \cup T = {♣, ♦, ♥, ♠} )
      • ( S \cap T = {♦, ♥} )

  • Graphical Representation: Shade separate graphs for ( S \cap T ) and ( S \cup T ).

  • Let ( S ) be the set of strictly positive even integers and ( T ) be integers ( \leq 9 ). Then:

    • ( S \cup T = {…, -2, -1, …, 8, 9, 10, 12, 14, 16, …} )
    • ( S \cap T = {2, 4, 6, 8} )

Complement of Sets

  • Complement:

    • Complement of ( S) is written ( S^c = {x; x \notin S} )
    • Properties:
      • ((S^c)^c = S)
      • ( S \cap S^c = \emptyset )
      • ( S \cup S^c = \Omega )

  • Example:

    • If ( \Omega = {♣, ♦, ♥, ♠} ) and ( S = {♦, ♥} ), then ( S^c = {♣, ♠} ).

Basic Set Operations

  • Set Difference:

    • ( S \setminus T = {x; x \in S \text{ and } x \notin T} = S \cap T^c )

  • Symmetric Difference:

    • ( S \Delta T = (S \setminus T) \cup (T \setminus S) )

Properties of Sets

  • Algebra of Sets:
    • ( S \cup (T \cap U) = (S \cup T) \cap (S \cup U) )
    • ( S \cap (T \cup U) = (S \cap T) \cup (S \cap U) )

Probabilistic Models

  • Probability Axioms:
    • Axiom 1: ( P(A) \geq 0 )
    • Axiom 2: ( P(\Omega) = 1 )
    • Axiom 3: If events are disjoint, then:
      • ( P(\bigcup{n=1}^\infty An) = \sum{n=1}^\infty P(An) )

Conditional Probability

  • Formula:

    • ( P(A|B) = \frac{P(A \cap B)}{P(B)} )
    • Valid if ( P(B) \neq 0 )

  • Properties:

    • If ( B \subset A ), then ( P(A|B) = 1 )
    • If ( A \subset B^c ), then ( P(A|B) = 0 )

Independence

  • Independent Events:
    • ( P(A \cap B) = P(A)P(B) )

Discrete Random Variables

  • Definition:

    • A random variable ( X ) is a function mapping outcomes to real numbers, taking finite or countably infinite values.

  • Probability Mass Function (PMF):

    • ( p_X(x) = P[X = x] )

Basic Probability Distributions

  • Bernoulli Distribution:
    • PMF: ( p(x) = \begin{cases} 1 - p & \text{if } x = 0 \ p & \text{if } x = 1 \end{cases} )
  • Binomial Distribution:
    • ( p(x) = \binom{n}{x} p^x (1-p)^{n-x} )( x = 0, 1, …, n )

Expectation and Variance

  • Expectation:
    • ( E[X] = \sum_{x} x P[X=x] )
  • Variance:
    • ( var(X) = E[(X - E[X])^2] = E[X^2] - (E[X])^2 )