M3. Moments

The expected value of a function of a variable, g(X), is defined as

E[a + bX] =

a + bE[X]

Addition: Let h(Y) be a (another) function of (another) variable. Then E[g(X) + h(Y)] =

E[g(X)] + E[h(Y)]

Multiplication: Let h(Y) be a (another) function of (another) variable. If X ⊥ Y then E[g(X)h(Y)] =

E[g(X)]E[h(Y)]

E(X), X~U(a, b)

Jensen’s inequality

The expected value of a nonlinear function of a variable is not equal to the nonlinear function of the expected value

The variance of a variable X is defined as

Let f (y|x) denote a conditional pdf. The conditional expectation is defined as

E[h(X)Y|X] =

h(X)E[Y|X]. Conditioning on X = treating it as known so this is akin to E[aX] = aE[X]

E[Y|X] = E[Y] if

X ⊥ Y. Intuitively, if two variables are independent then conditioning on X has no information about the mean of Y

The Law of Iterated Expectation: E[Y] =

E[E[Y|X]]

Cov(X, Y)

E [(X − E(X)) (Y − E(Y))] = E [XY] − E [X] E [Y]

(where a is a constant) Cov(X, a) =

0

(where a is a constant) Cov(aX, Y) =

aCov(X, Y)

Cov(X, Y + Z) =

Cov(X, Y) + Cov(X, Z)

Var(aX ± bY)

a2Var(X) + b2Var(Y) ± 2abCov(X, Y)

If X and Y are independent then, Cov(X, Y) =

0

Corr(X, Y) =

If two variables have a covariance of 0 or are uncorrelated must they be independent?

No, although if they are independent they must have a covariance of 0

Mean-independence is defined as

Independence implies mean-independence. However, the converse is not true