Chapter 5 – Probability: An Introduction to Modeling Uncertainty

5.1 Events and Probabilities

• Random Experiment: A process that generates well-defined outcomes.
• Sample Space: The set of all possible outcomes of a random experiment.
  • Example 1: Tossing a coin.
    • Outcomes: Head, Tail.
    • Sample space: $S = {Head, Tail}$
  • Example 2: Rolling a die.
    • Outcomes: 1, 2, 3, 4, 5, 6.
    • Sample space: $S = {1, 2, 3, 4, 5, 6}$
• Event: A collection of outcomes.
  • Example: CP&L Capacity Expansion Project
    • Possible project durations: 8, 9, 10, 11, or 12 months.
    • Event $C$: Project completed in 10 months or less $= {8, 9, 10}$
• Probability of an Event: The sum of probabilities of outcomes for the event.
  • $P(C) = P(8) + P(9) + P(10) = 0.15 + 0.25 + 0.30 = 0.70$
  • $P(L)$: Project completed in less than 10 months $= {8, 9}$
    • $P(L) = P(8) + P(9) = 0.15 + 0.25 = 0.40$
  • $P(M)$: Project completed in more than 10 months $= {11, 12}$
    • $P(M) = P(11) + P(12) = 0.15 + 0.15 = 0.30$

5.2 Some Basic Relationships of Probability

• Complement of an Event: The event consisting of all outcomes not in $A$, denoted by $A^c$.
  • $P(A) + P(A^c) = 1$
  • $P(A) = 1 - P(A^c)$
• Union of Two Events: The event containing all outcomes in $A$ or $B$ or both, denoted by $A \cup B$.
• Intersection of Two Events: The event containing all outcomes belonging to both $A$ and $B$, denoted by $A \cap B$.
• Addition Law: Computes the probability of the union of two events.
  • $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
  • Rationale: Outcomes in $A \cap B$ are counted twice when computing $P(A) + P(B)$, so we subtract $P(A \cap B)$ to correct for overcounting.
• Application of the Addition Law: HR study example.
  • Event $S$: Dissatisfaction with salary, $P(S) = 0.30$
  • Event $W$: Dissatisfaction with work assignments, $P(W) = 0.20$
  • Event $S \cap W$: Dissatisfaction with both salary and work assignments, $P(S \cap W) = 0.12$
  • Probability of leaving due to dissatisfaction with salary or work assignments:
    • $P(S \cup W) = P(S) + P(W) - P(S \cap W) = 0.30 + 0.20 - 0.12 = 0.38$
• Mutually Exclusive Events: Two events with no outcomes in common. If one occurs, the other cannot.
  • $P(A \cap B) = 0$
  • For mutually exclusive events, the addition law simplifies to: $P(A \cup B) = P(A) + P(B)$

5.3 Conditional Probability

• Studying mortgage defaults at Lancaster Savings and Loans (S&L).

• $S$ = event that a customer is single.

• $M$ = event that a customer is married.

• $D$ = event that a customer defaulted on their mortgage.

• $D^c$ = event that a customer did not default on their mortgage.

• Joint Probabilities: Intersection probabilities of events.

  • Probability of not defaulting and being married: $P(D^c \cap M) = \frac{64}{300} = 0.2133$
  • Probability of defaulting and being married: $P(D \cap M) = \frac{79}{300} = 0.2633$
  • Probability of not defaulting and being single: $P(D^c \cap S) = \frac{116}{300} = 0.3867$
  • Probability of defaulting and being single: $P(D \cap S) = \frac{41}{300} = 0.1367$

• Joint Probability Table: Summarizes joint probabilities.

• Marginal Probabilities: Probabilities of individual events, calculated as sums of joint probabilities.

  • Probability that a customer is married: $P(M) = \frac{143}{300} = 0.4766$

• Conditional Probability: The probability of event $A$ given that event $B$ has occurred.

  • Notation: $P(A | B) = \frac{P(A \cap B)}{P(B)}$
  • $P(B | A) = \frac{P(A \cap B)}{P(A)}$
  • Note: $P(A | B) \neq P(B | A)$

• Application of Conditional Probabilities:

  • Probability of defaulting given the customer is married:
    • $P(D | M) = \frac{P(D \cap M)}{P(M)} = \frac{0.2633}{0.4766} = 0.5524$
  • Probability of defaulting given the customer is single:
    • $P(D | S) = \frac{P(D \cap S)}{P(S)} = \frac{0.1367}{0.5234} = 0.2611$
  • Conclusion: Default probability is higher if the customer is married.

• Independent Events: Events $A$ and $B$ are independent if

  • $P(A | B) = P(A)$
  • $P(B | A) = P(B)$
  • Otherwise, $A$ and $B$ are dependent.
  • In the mortgage default case:
    • $P(D) = 0.40$
    • $P(D | M) = 0.5524$
    • $P(D | S) = 0.2611$
    • Since $P(D | M) \neq P(D)$, events $D$ and $M$ are dependent.

• Multiplication Law:

  • $P(A \cap B) = P(A | B) \cdot P(B)$
  • $P(B \cap A) = P(B | A) \cdot P(A)$
  • For independent events:
    • $P(A \cap B) = P(A) \cdot P(B)$
    • $P(B \cap A) = P(B) \cdot P(A)$
    • Events A and B are dependent when $P(A \cap B) \neq P(A) \cdot P(B)$

• Bayes’ Theorem:

  • For two events, $A<em>1$ and $A</em>2$:
    • $P(A<em>i | B) = \frac{P(A</em>i) \cdot P(B | A<em>i)}{P(A</em>1) \cdot P(B | A<em>1) + P(A</em>2) \cdot P(B | A_2)}$ with $i = 1, 2$
    • $P(A<em>1)$ and $P(A</em>2)$ are prior probabilities.
    • $P(B | A<em>1)$ and $P(B | A</em>2)$ are conditional probabilities.
    • Bayes’ theorem calculates revised (posterior) probabilities $P(A<em>1 | B)$ and $P(A</em>2 | B)$.

  5.4 Random Variables

• Random variable: A numerical description of an experimental outcome.

• Discrete random variable: May assume either a finite number of values or an infinite sequence of values.

  • Examples:
    • Flip a coin: Face of coin showing (1 if heads; 0 if tails)
    • Roll a die: Number of dots showing on top of die (1, 2, 3, 4, 5, 6)
    • Contact five customers: Number of customers who place an order (0, 1, 2, 3, 4, 5)
    • Operate a health care clinic for one day: Number of patients who arrive (0, 1, 2, 3, …)
    • Offer a customer the choice of two products: Product chosen by customer (0 if none; 1 if choose product A; 2 if choose product B)

• Continuous random variable: May assume any numerical value in an interval or collection of intervals.

  • Examples:
    • Customer visits a web page: Time customer spends on web page in minutes ( $x \geq 0$ )
    • Fill a soft drink can (max capacity = 12.1 ounces): Number of ounces ( $0 \leq x \leq 12.1$ )
    • Test a new chemical process: Temperature when the desired reaction takes place (min temperature = 150°F; max temperature = 212°F) ( $150 \leq x \leq 212$ )
    • Invest $10,000 in the stock market: Value of investment after one year ( $x \geq 0$ )

5.5 Discrete Probability Distributions

• The probability distribution for a random variable describes the range and relative likelihood of possible values for a random variable.

• For a discrete random variable $x$, the probability distribution is defined by a probability mass function, denoted by $f(x)$.

• The probability mass function $f(x)$ provides the probability for each value of the random variable $x$.

• Required conditions for a discrete probability function are: $f(x) \geq 0$ and $\sum f(x) = 1$

• Empirical Probability Distribution: A probability distribution generated from observations.

  • An empirical probability distribution is considered a custom discrete probability distribution if it is discrete and the possible values of the random variable have different probabilities.
  • The probabilities of a custom distribution can be generated using either the subjective method or the relative frequency method.

• Custom Discrete Probability Distribution

  • Consider a random variable, $x$, describing the number of payments made per year by 300 home mortgage customers:
    • 45 customers made quarterly payments ($x = 4$), 180 made monthly payments ($x = 12$), and 75 made two payments per month ($x = 24$).
      • Probability distribution as a function:
        $f(x) = \begin{cases} 0.15, & x = 4 \ 0.60, & x = 12 \ 0.25, & x = 24 \ 0, & \text{otherwise} \end{cases}$

• Expected Value: The expected value, or mean, is a measure of the central location of a random variable, $x$. It is calculated as the weighted average of the values of the random variable. The weights are the probabilities, $f(x)$.

  • $E(x) = \mu = \sum x f(x)$

• Variance: The variance is a measure of variability in the values of a random variable, $x$. It is calculated as the weighted average of the squared deviations of a random variable, $x$, from its mean, $\mu$. The weights are the probabilities, $f(x)$.

  • $Var(x)=\sigma^2 = \sum(x-\mu)^2f(x)$

• Standard Deviation: The standard deviation $\sigma$ is the positive square root of the variance.

  • $\sigma = \sqrt{\sigma^2} = \sqrt{Var(x)}$

• Discrete Uniform Probability Distribution: The simplest example of a discrete probability distribution given by a formula.

  • The discrete uniform probability mass function can be written as:
    • $f(x) = \frac{1}{n}$
    • Where $n$ is the number of values the random variable may assume.

• Binomial Probability Distribution: The binomial probability function determines the probability of observing $x$ successes in $n$ independent events when there are only two possible outcomes.

  • The binomial probability mass function is described by the equation:
    • $f(x) = \binom{n}{x} p^x (1-p)^{n-x}$
    • Where $p$ is the probability of a success in a single event, and $1-p$ the probability of a failure in a single event.

• Poisson Probability Distribution: A Poisson Probability Distribution describes the number of occurrences, $x$, over a specified interval of time or space, if the following two properties are met:

  • $1$. The probability of an occurrence is the same for any two intervals (time or space) of equal length.
  • $2$. The occurrence or nonoccurrence in any interval is independent of the occurrence or nonoccurrence in any other interval.

5.6 Continuous Probability Distributions

• Probability mass function provides the probability that the random variable assumes a particular value for Discrete random variables
• Probability density function is the counterpart of the probability mass function for Continuous random variables.
• Uniform Probability Distribution: A continuous random variable, $x$, is described by a uniform probability distribution whenever the probability is proportional to the interval’s length.
  • The uniform probability density function is
    $f(x) = \begin{cases} \frac{1}{b-a} & \text{for } a \leq x \leq b \ 0 & \text{elsewhere} \end{cases}$
• Triangular Probability Distribution: A continuous random variable, $x$, can be described by a triangular probability distribution when only the following subjective information is available:
  • the minimum possible value, $a$,
  • the maximum possible value, $b$, and
  • the most likely value (or mode) of the distribution, $m$.
• Normal Probability Distribution: The normal probability distribution assumes the characteristic bell-shaped curve and is the most common probability continuous distribution.
  • The normal probability density function is entirely described by two parameters: the mean, $\mu$, and the standard deviation, $\sigma$
    • $f(x) = \frac{1}{\sqrt{2 \pi \sigma^2}} e^{-\frac{1}{2} (\frac{x-\mu}{\sigma})^2}$
  • Where $\pi \approx 3.14159$ and $e \approx 2.71828$
• Exponential Probability Distribution: The exponential probability distribution is used for a random variable, $x$, that describes the length of an interval between occurrences.
  • The exponential probability density function is
    • $f(x) = \frac{1}{\mu} e^{-\frac{x}{\mu}} \quad \text{for } x \geq 0$