Math 183 - Part 3 (Probability Theory)

Probability theory

A mathematical framework for modeling randomness and uncertainty.

Sample space

The set of all possible outcomes of a random experiment.

Event

A subset of the sample space; a collection of outcomes of interest.

Intersection of events

The set of outcomes common to both events (“A and B”).

Union of events

The set of outcomes in at least one of the events (“A or B or both”).

Disjoint events

Events that cannot both occur at the same time (no shared outcomes).

Complement of an event

The set of outcomes in the sample space not included in the event.

Probability distribution

A function assigning probabilities to outcomes or events, with probabilities summing to 1.

Probability mass function (pmf)

A function that gives the probability for each outcome in a discrete sample space.

Uniform probability

When all outcomes in the sample space are equally likely.

Additivity

If two events are disjoint, the probability of their union is the sum of their probabilities.

General addition rule

For any events A and B, P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

Law of total probability

P(A) = P(A ∩ B) + P(A ∩ Bᶜ).

Conditional probability

The probability of event A given that event B has occurred, P(A|B) = P(A ∩ B)/P(B).

Multiplication rule

P(A ∩ B) = P(A|B) × P(B).

Independence (events)

Events A and B are independent if P(A|B) = P(A), or equivalently, P(A ∩ B) = P(A) × P(B).

Bayes’ Theorem

P(A|B) = [P(B|A) × P(A)] / P(B); relates conditional probabilities in “reverse” order.

Random variable

A function assigning a numerical value to each outcome in the sample space.

Indicator random variable

A variable equal to 1 if an event occurs, 0 otherwise.

Distribution of a random variable

The assignment of probabilities to the values the random variable can take.

Expectation (mean)

The average value of a random variable over many trials, E(X) = Σ x P(X = x).

Linearity of expectation

E(aX + bY) = aE(X) + bE(Y), for any constants a and b.

Variance

A measure of how spread out the values of a random variable are, Var(X) = E[(X − E(X))²].

Standard deviation

The square root of the variance; a measure of typical distance from the mean.

Variance of a sum (independent)

If X and Y are independent, Var(X + Y) = Var(X) + Var(Y).

Chebyshev’s inequality

For any random variable X, P(|X − μ| ≥ tσ) ≤ 1/t², where μ is mean, σ is SD.

Law of large numbers

As the number of independent, identically distributed trials increases, the sample mean approaches the population mean.

Frequentist interpretation

Probability of an event is the long-run relative frequency of that event in repeated trials.

Discrete probability space

A probability model with a finite or countable sample space (e.g., dice, coin flips).

Continuous probability space

A model with uncountably infinite outcomes; probabilities assigned to intervals via a density function.

Probability density function (pdf)

A function f(x) for continuous variables; probability X falls in [a, b] is the integral of f(x) from a to b.

Covariance

A measure of how two random variables vary together.

Correlation

A standardized measure of linear association between two random variables, ranges from -1 to 1.

Indicator variable

A variable that is 1 if a certain event occurs, 0 otherwise.

General multiplication rule (chains)

For events A1, A2, ..., An, P(A1 ∩ A2 ∩ ... ∩ An) = P(A1) × P(A2|A1) × ... × P(An|A1...An−1).

Sample mean

The average of observed values from a sample, a random variable itself.

Variance of sample mean

For n independent variables, variance of sample mean is population variance divided by n.

Base rate fallacy

Misinterpreting conditional probabilities by neglecting base rates (overall frequencies).

Natural experiment

Randomization occurs due to an external process, not by the researchers.

Uniform distribution (finite)

Each outcome in the sample space is equally likely.