STAT INF - summary

An estimator is unbiased if

An estimator is unbiased if the mean of its sampling distribution is equal to the true value of the parameter being estimated.

Reduced Exponential Family

Regular Exponential Family

Invariance Property

Regular Estimation Problem

Fisher Score Function

Fisher Information

Two ways to Estimate Asymptotic Variance

Asymptotic variance allows you to compare different estimators to see which one is more "efficient" (uses the data better).

Once you know the asymptotic distribution, you can build confidence intervals.

Proving Estimator of Asymptotic Variance is Consistent

Asymptotic Distribution of MLE

“By asymptotic normality of the MLE, we would have …”

The Cramér-Rao Inequality

Optimal efficiency means that among all "well-behaved" estimators, the MLE has the smallest possible asymptotic variance.

Delta Method Theorem

“By asymptotic normality of the MLE, we would have …”

WLLN

“Note that X¯n →p λX by WLLN”

Sample mean →p population mean

CMT

“Therefore, it is consistent by CMT as g (x) = 1/x is continuous for x > 0.”

Convergence

O_p and o_p

Speed of convergence

Slutsky’s Theorem

Central Limit Theorem

Newton–Raphson Algorithm

Power of a test

Types of errors

Size of a test

= maximum of power function in case null hypothesis is true

Different Kinds of Tests

p-value

Neyman-Pearson Lemma

provides the most powerful test for comparing two simple hypotheses

Wald Statistic

Score Statistic

Complete Testing Procedure

To show consistency of test:

Size Distortion

Do we prefer estimator with low or high variance?

We want to have an estimate that is as robust as possible

= estimator performs well even when the model assumptions are violated or when there are outliers.

By choosing the estimator with the lower variance, there will be less variation in the obtained estimate and this leads to more precise inference.

Confidence Interval

Confidence Interval

General construction idea

Confidence Interval

Common formulas

p-value table

Example