CS540 Midterm 1 - Probability and Math

What is the definition of an outcome in probability?

set of results of an experiment

How do you mathematically define an outcome?

Ω = {1,2,3,4}

What are events?

subsets of outcomes we’re interested in essentially (powerset)

How do you mathematically define an event?

∅,{1},{2},{3},{4}

Axioms: For all events E, ___

P(E) ≥ 0

Axioms: Always P(∅) = P(Ω) = ___

0, 1

Axioms: For disjoint events, P(E₁ U E₂) =

P(E₁) + P(E₂)

Axioms: For nondisjoint events, P(E₁ U E₂) =

P(E₁) + P(E₂) - (P(E₁) ∩ P(E₂))

What is a Random Variable?

a function that maps random outcomes to real values

What is a CDF and what is it used for?

Cumulative Distribution Function, basically evaluating what is the chance of rolling a target value or anything lower?

CDF formula:

$$F_X(x) := P(X \le x)$$

What is a PDF/PMF and what is it used for?

Probability density function/ Probability mass function used to tell you how likely different outcomes are

What is the formula for a PDF/PMF?

Explain the Expectation of an RV formula : $E[X] = \sum_a a \cdot P(X = a)$

Take every possible outcome, multiply its value by the odds of it happening, and add them all together.

Explain the Expectation of a Function formula: $E[f(X)] = \sum_a f(a) \cdot P(X = a)$

Take every possible outcome, multiply its value (with whatever rule or modification applied first) by the odds of it happening, and add them all together

What is variance? $Var[X] = E[(X - E[X])^2]$

Variance is simply a measure of how spread out and wild the results are compared to the average.

What is a Joint Distribution? $P(X = a, Y = b)$

What are the chances of two (or more) things happening together at the exact same time?

What is Marginal Probability? $P(X = a) = \sum_b P(X = a, Y = b)$

to find the total chance of one variable, add across all possibilities of the other variable.

What does Independence mean? $P(X, Y) = P(X)P(Y)$

Independence means two events have zero effect on each other.

What is Conditional Probability? $P(X = a \mid Y = b) = \frac{P(X = a, Y = b)}{P(Y = b)}$

Conditional Probability is the chance of something happening after you already learn a clue.

Conditional Independence: $P(X, Y \mid Z) = P(X \mid Z) P(Y \mid Z)$

Example: Ice-cream sales and drownings are correlated, but conditionally independent given the weather.

Chain Rule explained: $P(A_1, A_2, \dots, A_n) = P(A_1) \cdot P(A_2 \mid A_1) \cdot P(A_3 \mid A_2, A_1) \dots P(A_n \mid A_{n-1}, \dots, A_1)$

a way to calculate the probability of a whole sequence of events happening by taking them one link at a time.

Bayes Rule: $P(A \mid B) = \frac{P(B \mid A) \cdot P(A)}{P(B)}$

Given that I see this clue, what are the chances the condition is actually true?

Law of Total Probability $P(A) = \sum_{i=1}^n P(A \mid B_i) P(B_i)$

The Law of Total Probability is a method for finding the overall chance of an event by slicing the world into separate scenarios, calculating the odds in each slice, and adding them all together.

Bayesian Inference: $P(H \mid E) = \frac{P(E \mid H) P(H)}{P(E)}$

Bayesian Inference is just using clues to update your confidence in a theory. H is hypothesis E is evidence. So you’re evaluating H given E.