Joint Probability, Independence, and Sampling

Distributions, Normal, z-scores, joint probability, independence and covariance, sampling and CLT

Baye’s rule

P(A|B) = P(B|A)*P(A) / P(B)

Expected value E(X) - discrete case

∑X*P(X=x)

Expected value E(X) - continuous case

∫x*f(x)dx (-∞,∞)

Linearity of expectation E(aX + b) and E(aX + bY)

aE(X) + b and aE(X) + bE(Y)

Variance Var(X)

E(X²) - [E(X)]²

Variance of a linear transformation Var(aX +b)

a² * Var(X)

Variance of a sum Var(aX + bY)

a²Var(X) + b²Var(Y) - 2abCov(X,Y)

if X and Y are independent, what is covariance

Cov = 0

normal distribution

X ~ N(µ,∂²), symmetric about µ, spread controlled by s.d. (∂)

95% rule

95% of probability lies between µ±1.96∂

properties when X~N(µ,∂²)

E(X) = µ, Var(X) = ∂², SD(X) = ∂

Standardisation formula (Z)

Z = (x - µ)/∂

properties of z-score

Z~N(0,1)

X~N(2,1), find P(X≤2.5)

z=(2.5-2)/1=0.5, P(X≤2.5)=P(z≤0.5)=ø(0.5)

joint probability distribution P(X=x, Y=y)

P(X=x)*P(Y=y|X=x)

conditional expectation E(Y|X=x)

∑y*P(Y=y|X=x)

X and Y independent P(X=x,Y=y)

P(X=x)P(Y=y)

X and Y independent E(XY)

E(X)*E(Y)

Covariance Cov(X,Y)

E(XY) - µx*µy

Correlation p(X,Y)

Cov(X,Y)/(∂x*∂y)

p=0

does not prove independence, only rules out linear relationships

Variance of a sum Var(X+Y)

Var(X) + Var(Y) + 2Cov(X,Y)

if independent Var(X + Y)

Var(X) + Var(Y)

if X and Y are negatively correlated

Cov<0 so Var(X+Y)<Var(X)+Var(Y)

i.i.d.

independent and identically distributed

i.i.d. explanation

when experiment is repeated n times, each Yi is a random variable drawn from the same distribution and they dont influence each other

Sample mean Y^

is itself a random variable

Properties of sample mean

E(Y^) = µ, Var(Y^) = ∂²/n, SD(Y^) = ∂/√n

Law of large numbers (LLN)

as n→∞, Y^→µ, sample mean converges to true population mean

Central Limit Theorem (CLT)

for large n(≥30), Y^~N(µ,∂²/n)

difference between Y^ and E(X)

first is a random variable, the other is a constant

E(Y^) and Var(Y^)

µ and ∂²/n