Probability and Matrix Operations Study Guide

Introduction to Probability

Definition of Probability: Probability is a measure of the likelihood of a random phenomenon or chance behavior. It describes the long-term proportion with which a certain outcome will occur in situations with short-term uncertainty.
Example Simulation: Flipping a coin $100$ times. * Process: Plot the proportion of heads against the number of flips. * Repeat the simulation to observe behavior.
Long-term Predictability: Probability deals with experiments that yield random short-term results or outcomes, yet reveal long-term predictability. The probability of an outcome is defined as the long-term proportion with which that certain outcome is observed.
The Law of Large Numbers: As the number of repetitions of a probability experiment increases, the proportion with which a certain outcome is observed gets closer to the probability of the outcome.
Acknowledgements: Materials adapted from www.math.kent.edu/~honli/.../chap05_01.ppt and www.vipbg.vcu.edu/.../Matrix_Algebra.ppt.

Probability Experiments and Sample Spaces

Experiment: In probability, an experiment is any process that can be repeated in which the results are uncertain.
Simple Event: A simple event is any single outcome from a probability experiment. Each simple event is denoted as $e_i$ .
Sample Space ( $S$ ): The sample space of a probability experiment is the collection of all possible simple events. It is essentially a list of all possible outcomes of the experiment.
Event ( $E$ ): An event is any collection of outcomes from a probability experiment. * An event may consist of one or more simple events. * Events are denoted using capital letters, such as $E$ . * Notation: The probability of an event is denoted as $P(E)$ , which represents the likelihood of that event occurring.

Properties of Probabilities

The Probability Scale: The probability of any event $E$ , $P(E)$ , must be between $0$ and $1$ inclusive (0 < P(E) < 1).
Impossible Events: If an event is impossible, the probability of the event is $0$ .
Certain Events: If an event is a certainty, the probability of the event is $1$ .
Sum of Sample Space: If a sample space $S = {e_1, e_2, \dots, e_n}$ , then the sum of the probabilities of all simple events must equal one: $P(e_1) + P(e_2) + \dots + P(e_n) = 1$ .
Unusual Event: An unusual event is defined as an event that has a low probability of occurring.
Equally Likely Outcomes: An experiment has equally likely outcomes when each simple event has the exact same probability of occurring.

Computing Probability Using the Empirical Method

Empirical Probability Formula: The probability of an event $E$ is approximately calculated by the following ratio: * $P(E) \approx \frac{\text{number of times event E is observed}}{\text{number of repetitions of the experiment}}$

The Addition Rule and Mutually Exclusive Events

Event Combinations: * E and F: The event consisting of simple events that belong to both $E$ and $F$ . * E or F: The event consisting of simple events that belong to either $E$ or $F$ or both.
Mutually Exclusive (Disjoint) Events: Events are called mutually exclusive or disjoint if the occurrence of one event precludes the occurrence of the other. * Example 1: Two fair coins are tossed. Possible outcomes: ${HH, TH, HT, TT}$ . Define $E = {TH}$ and $F = {TT}$ . These are disjoint. * Example 2: Define $E = {TT, TH}$ and $F = {TT, HH}$ . These are not disjoint because they share the outcome $TT$ . * Disjoint Definition: If events $E$ and $F$ have no simple events in common or cannot occur simultaneously, they are disjoint.
General Addition Rule: For any two events $E$ and $F$ : * $P(E \text{ or } F) = P(E) + P(F) - P(E \text{ and } F)$
Addition Rule for Mutually Exclusive Events: If $E$ and $F$ are mutually exclusive: * $P(E \text{ or } F) = P(E) + P(F)$ * Extension: If $E, F, G, \dots$ are mutually exclusive, then $P(E \text{ or } F \text{ or } G \text{ or } \dots) = P(E) + P(F) + P(G) + \dots$
Visual Representation: In a Venn diagram where the area of the entire region $P(S) = 1$ , the overlap represents $E \text{ and } F$ .

Complements and Complement Rule

Complement of an Event: Let $S$ denote the sample space and $E$ denote an event. The complement of $E$ , denoted as $\bar{E}$ , consists of all simple events in the sample space $S$ that are not simple events in event $E$ .
Complement Rule: If $E$ represents any event and $\bar{E}$ represents its complement, then: * $P(E) = 1 - P(\bar{E})$

Conditional Probability and Independence

Conditional Probability: The notation $P(F | E)$ is read as "the probability of event $F$ given event $E$ ". It describes the probability of an event $F$ occurring given that event $E$ has already occurred.
Independence: Two events $E$ and $F$ are independent if the occurrence of event $E$ in a probability experiment does not affect the probability of event $F$ .
Dependence: Two events are dependent if the occurrence of event $E$ affects the probability of event $F$ .
Definition of Independent Events: Events $E$ and $F$ are independent if and only if: * $P(F | E) = P(F)$ OR $P(E | F) = P(E)$

The Multiplication Rule

General Multiplication Rule: The probability that two events $E$ and $F$ both occur is: * $P(E \text{ and } F) = P(E) \times P(F | E)$ * In words: The probability of $E$ and $F$ is the probability of event $E$ occurring times the probability of event $F$ occurring given that event $E$ has occurred.
Multiplication Rule for Independent Events: If $E$ and $F$ are independent: * $P(E \text{ and } F) = P(E) \times P(F)$
Multiplication Rule for $n$ Independent Events: If events $E, F, G, \dots$ are independent, then: * $P(E \text{ and } F \text{ and } G \text{ and } \dots) = P(E) \times P(F) \times P(G) \times \dots$

Matrix Operations: Addition and Subtraction

Matrix Addition Example 1: * $\begin{pmatrix} 1 & 2 \ 3 & 4 \end{pmatrix} + \begin{pmatrix} 5 & 6 \ 7 & 8 \end{pmatrix} = \begin{pmatrix} 6 & 8 \ 10 & 12 \end{pmatrix}$
Matrix Addition Example 2: * $A = \begin{pmatrix} 1 & 4 \ 2 & 3 \end{pmatrix}$ and $B = \begin{pmatrix} 5 & 8 \ 6 & 7 \end{pmatrix}$ * $A + B = C \rightarrow \begin{pmatrix} 6 & 12 \ 8 & 10 \end{pmatrix}$
Addition Conformability: To add two matrices $A$ and $B$ : * The number of rows in $A$ must equal the number of rows in $B$ . * The number of columns in $A$ must equal the number of columns in $B$ .
Matrix Subtraction Example 1: * $\begin{pmatrix} 5 & 6 \ 7 & 8 \end{pmatrix} - \begin{pmatrix} 1 & 2 \ 3 & 4 \end{pmatrix} = \begin{pmatrix} 4 & 4 \ 4 & 4 \end{pmatrix}$ * Calculated as $B - A = C$ .
Matrix Subtraction Example 2: * $\begin{pmatrix} 1 & 4 \ 2 & 3 \end{pmatrix} - \begin{pmatrix} 5 & 8 \ 6 & 7 \end{pmatrix} = \begin{pmatrix} 4 & 4 \ 4 & 4 \end{pmatrix}$ (Note: The visual indicates subtraction resulting in absolute values or specific element-wise differences as labeled in the slide).
Subtraction Conformability: Rules are identical to addition: * Number of rows in $A$ = number of rows in $B$ . * Number of columns in $A$ = number of columns in $B$ .

Matrix Operations: Multiplication

Multiplication Conformability: To multiply two matrices $A$ and $B$ (Regular Multiplication): * The number of columns in $A$ must equal the number of rows in $B$ . * Dimensions: If $A$ is $(m \times n)$ and $B$ is $(n \times p)$ , the resulting matrix is $(m \times p)$ .
General Multiplication Formula: * $C_{ij} = \sum_{k=1}^n A_{ik} \times B_{kj}$
Step-by-Step Multiplication Example: * Matrices: $A = \begin{pmatrix} 5 & 6 \ 7 & 8 \end{pmatrix}$ and $B = \begin{pmatrix} 1 & 2 \ 3 & 4 \end{pmatrix}$ . * Step 1: Calculate $C_{11}$ ; * $C_{11} = (A_{11} \times B_{11}) + (A_{12} \times B_{21})$ * $C_{11} = (5 \times 1) + (6 \times 3) = 5 + 18 = 23$ * Step 2: Calculate $C_{12}$ ; * $C_{12} = (A_{11} \times B_{12}) + (A_{12} \times B_{22})$ * $C_{12} = (5 \times 2) + (6 \times 4) = 10 + 24 = 34$ * Step 3: Calculate $C_{21}$ ; * $C_{21} = (A_{21} \times B_{11}) + (A_{22} \times B_{21})$ * $C_{21} = (7 \times 1) + (8 \times 3) = 7 + 24 = 31$ * Step 4: Calculate $C_{22}$ ; * $C_{22} = (A_{21} \times B_{12}) + (A_{22} \times B_{22})$ * $C_{22} = (7 \times 2) + (8 \times 4) = 14 + 32 = 46$ * Final Resulting Matrix $C$ : * $C = \begin{pmatrix} 23 & 34 \ 31 & 46 \end{pmatrix}$