Probability Theory and Event Prediction Notes

Foundations of Probability Theory

• Definition of Probability: A probability is a quantitative value, expressed as either a number or a percentage, that ranges between $0$ and $1$. It serves as a measure indicating the likelihood of a specific event occurring.

• Backbone of Statistics: Probability theory is considered the fundamental prerequisite and structural basis for the field of statistics.

• Methods for Assigning Probabilities:     * Classical Method: Probabilities are assigned based on the theoretical ratio of outcomes. The formula is:         * $\frac{\text{Number of outcomes in which an event occurs}}{\text{Number of possible outcomes}}$     * Empirical Probability (Relative Frequency Method): This approach relies on historical data or observed results. The formula is:         * $\frac{\text{Number of outcomes in which an event has occurred in the past}}{\text{Number of opportunities for an event to occur}}$     * Subjective Probability Method: This method involves the use of individual judgment, experience, and other non-mathematical criteria to determine the likelihood of an event.

• Core Definitions:     * Experiment: A defined random process that generates specific results or data points (e.g., a data-collection procedure).     * Sample Space: The comprehensive set representing all possible outcomes of an experiment.     * Event: A specific set of outcomes derived from an experiment. An event can be categorized as containing no outcome (empty set), a single outcome, or multiple outcomes. Probability is specifically assigned to these events.

Data Structure and Contingency Tables

• Multi-dimensional Data: Data collections often involve multiple variables or dimensions. This is frequently organized using a contingency table (also known as a cross-tab).

• Phone Plan Example Data (Counts):     * The data tracks two random variables: Phone plan choice ($29$, $49$, or $79$ dollars) and the day of purchase (Monday-Friday vs. Saturday-Sunday).     * Monday-Friday (Weekdays):         * $29$ Plan: $10$ instances         * $49$ Plan: $120$ instances         * $79$ Plan: $250$ instances         * Total Weekday Purchases: $380$     * Saturday-Sunday (Weekends):         * $29$ Plan: $40$ instances         * $49$ Plan: $30$ instances         * $79$ Plan: $350$ instances         * Total Weekend Purchases: $420$     * Combined Totals:         * Total $29$ Plans: $50$         * Total $49$ Plans: $150$         * Total $79$ Plans: $600$         * Grand Total Outcomes: $800$

• Potential Outcomes: In this example, there are $6$ specific potential outcomes representing every possible combination of plan price and purchase day.

• Sample Space for Example: {$29 \, \text{on weekdays}, $29 \, \text{on weekends}, $49 \, \text{on weekdays}, $49 \, \text{on weekends}, $79 \, \text{on weekdays}, $79 \, \text{on weekends}}

Probability Concepts and Visualizations

• Relative Frequency Assignment: Outcomes are assigned probabilities based on their observed frequency relative to the total samples.     * $P(\text{outcome}_i) = \frac{\text{number of occurrences of outcome}_i}{\text{total number of occurrences of all outcomes}}$

• Joint vs. Marginal Probabilities:     * Joint Probability: This denotes the relative frequency of an event involving all dimensions simultaneously (e.g., the probability a customer bought the $49$ plan on a weekday). It describes outcomes associated with more than one random variable.     * Marginal Probability: This represents the relative frequency of an event when considering only a single dimension, regardless of other variables (e.g., the total probability of a customer buying a $49$ plan, ignoring which day it was purchased).

• Venn Diagrams: Introduced by John Venn in $1880$, these diagrams visualize logical relations between sets.     * External Rectangle: Represents the entire sample space.     * Internal Circle: Represents a specific event, such as event $A$.

• Mathematical Notation:     * Complement ($A'$): Pronounced "A prime," this refers to the event "not $A$." For example, if $A$ is the event of buying a $29$ dollar plan, $A'$ is the event of buying any plan except the $29$ dollar one.     * Intersection ($\cap$): This symbol represents "and," indicating the co-occurrence of events. $A \cap B$ refers to both $A$ and $B$ happening. This is visualized as the overlapping section in a Venn diagram.     * Union ($\cup$): Pronounced "union," this symbol represents "or." $A \cup B$ indicates that event $A$ or event $B$ (or both) occurs.

Essential Rules of Probability

• Complement Rule: The sum of the probability of an event and its complement is always equal to $1$.     * $P(A) + P(A') = 1$

• Law of Total Probability (Version 1): The sum of the joint probabilities of an event $A$ intersection with $B$ and $A$ intersection with not $B$ equals the marginal probability of $A$.     * $P(A \cap B) + P(A \cap B') = P(A)$

• General Rule of Addition: This calculates the probability of the union of two events.     * $P(A \cup B) = P(A) + P(B) - P(A \cap B)$     * Example calculation from data: Probability of selling a plan on a weekday OR selling it for $29$ dollars.         * Calculation 1: $\frac{10}{800} + \frac{40}{800} + \frac{370}{800} = 0.0125 + 0.05 + 0.4625 = 0.525$         * Alternative calculation: $P(29) + P(\text{weekday}) - P(29 \cap \text{weekday})$         * $0.0625 + 0.475 - 0.0125 = 0.525$

• Mutually Exclusive Events: Events that cannot occur at the same time. If $A$ and $B$ are mutually exclusive, then $P(A \cap B) = 0$. They do not intersect in a Venn diagram. Any event and its complement are inherently mutually exclusive.

• Collectively Exhaustive Events: A set of events where the occurrence of at least one covers the entire sample space. If $A$ and $B$ are collectively exhaustive, then $P(A \cup B) = 1$. Any event and its complement are collectively exhaustive.

Conditional Probabilities and Bayes Rule

• Conditional Probability ($P(A|B)$): Denotes the probability that event $A$ occurs given that event $B$ has already occurred.     * Formula: $P(A|B) = \frac{P(A \cap B)}{P(B)}$     * Formula: $P(B|A) = \frac{P(A \cap B)}{P(A)}$     * Example: The probability of a client choosing a $29$ dollar plan conditional on visiting the store during a weekday.         * Approach 1: $\frac{10}{380} = 0.026$         * Approach 2: $\frac{P(29 \cap \text{Weekday})}{P(\text{Weekday})} = \frac{0.0125}{0.475} = 0.026$

• General Law of Multiplication: Derived from the conditional probability formula.     * $P(A \cap B) = P(A|B)P(B) = P(B|A)P(A)$

• Bayes Rule: An algebraic rearrangement of the multiplication law.     * $P(B|A) = \frac{P(A|B)P(B)}{P(A)}$

• Law of Total Probability (Version 2): Using conditional probabilities to find a marginal probability.     * $P(A|B)P(B) + P(A|B')P(B') = P(A)$

Event Independence

• Defining Independence: Events are independent if the occurrence or non-occurrence of one event has no effect on the likelihood of the other event.     * Example: Flipping a coin twice. The outcome of the second toss is independent of the first; $P(\text{Heads on 2nd toss}) = 50\%$ regardless of the first result.

• Mathematical Tests for Independence:     * Version 1: $P(A|B) = P(A)$ and $P(B|A) = P(B)$     * Version 2: $P(A \cap B) = P(A) \times P(B)$

• Testing for Dependence: If $P(A|B) \neq P(A)$ or if $P(A \cap B) \neq P(A) \times P(B)$, the events are considered dependent (not independent).

• Independence Test Example: Plan choice vs. Day of purchase.     * $P(29) = 0.0625$     * $P(\text{weekdays}) = 0.475$     * $P(29) \times P(\text{weekdays}) = 0.0625 \times 0.475 = 0.0297$     * Observed $P(29 \cap \text{weekdays}) = 0.0125$     * Since $0.0297 \neq 0.0125$, the events are not independent.

Case Study: Titanic Sinking Data

• Data Set Summary:     * Survivors: $233$ Females, $109$ Males. (Total survivors: $342$\/$891$)     * Deceased: $81$ Females, $468$ Males. (Total deaths: $549$\/$891$)     * Gender Totals: $314$ Females, $577$ Males. (Grand Total: $891$)

• Probability Calculations:     * Marginal Probability of Surviving: $P(\text{Survived}) = \frac{342}{891} \approx 0.38$ ($38\%$)     * Joint Probability (Male and Surviving): $P(\text{Survived} \cap \text{Male}) = \frac{109}{891} \approx 0.12$ ($12\%$)     * Union Probability (Survived OR Male): $P(\text{Survived} \cup \text{Male}) = \frac{342}{891} + \frac{577}{891} - \frac{109}{891} = \frac{810}{891} \approx 0.91$     * Conditional Probability (Survived given Male): $P(\text{Survived}|\text{Male}) = \frac{0.12}{0.65} \approx 0.19$ ($19\%$)     * Probability of Not Surviving: $P(\text{Survived}') = 1 - 0.38 = 0.62$. Note that Surviving and Dying are complements, mutually exclusive, and collectively exhaustive.

• Testing Independence (Gender and Survival):     * Independence would require $P(\text{Survived} | \text{Male}) = P(\text{Survived})$.     * $P(\text{Survived} | \text{Male}) = 0.19$.     * $P(\text{Survived}) = 0.38$.     * Since $0.19 \neq 0.38$, survival was dependent on gender.

Questions & Discussion

• Q: Why use the word "may" when concluding lack of independence based on the calculation?     * A: We cannot be $100\%$ sure from raw data alone without performing formal statistical hypothesis tests. These tests determine if the difference observed in the sample is significant enough to represent the population. These tests will be studied later in the semester.

• Q: If we collect data to answer if plan choices and day of purchase are independent, is that a sample or a population?     * A: It is typically a sample. If you chose a different set of data, the numbers and resulting probabilities in the contingency table would likely vary.

Weekly Summary of Formulae

• a) $P(A) + P(A') = 1$

• b) $P(A \cap B) + P(A \cap B') = P(A)$

• c) $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

• d) $P(A|B) = \frac{P(A \cap B)}{P(B)}$ ; $P(B|A) = \frac{P(A \cap B)}{P(A)}$

• e) $P(A|B)P(B) + P(A|B')P(B') = P(A)$ ; $P(B|A)P(A) + P(B|A')P(A') = P(B)$

• f) $\text{Independence} \iff P(A|B) = P(A); P(B|A) = P(B)$

• g) $\text{Independence} \iff P(A \cap B) = P(A) \times P(B)$