Model Fitting

Probability

Long-run relative frequency from a random process

Probability characteristics

1. Between 0-1

2. Cannot be negative

3. Sum of all probabilities over all possible mutually exclusive outcomes must equal 1

4. If 2 events are mutually exclusive, the probability of observing either events is the sum of their individual probabilities

Independence

Knowledge of one event provides no information about the probability that another event will occur

Joint probability

The probability that 2+ different events will occur P(A,B), P(A & B), P(A intersection B)

Probability density function (pdf)

Describes the probability of an event occurring for a continuous distribution (Total area under the curve is 1)

Ex: Normal distribution is a common pdf

Maximum likelihood estimation (MLE)

Involves finding the parameters that maximize the probability of generating the observed data

Likelihood does NOT equal probability

MLE parameter estimation

Treat the data as fixed and find parameters that maximize the probability of observing that data

Properties of likelihoods

Do not need to sum to one

Relative measure of model fit

Calculating products is difficult, so logs often used

Properties of MLEs

Invariant to log transformations

Asymptotically efficient (lowest possible var)

Asymptotically normally distributed

Asymptotically unbiased

Same as OLS id errors are normal, additive, and with constant var

Nonlinear optimization

Term for trying to find the max/min of a function

Potential problems with nonlinear optimization

1. Improper starting values

2. Model specification/coding error

3. (-LL) function may be undefined for some parameter values

4. Parameters action problems

5. Uninformative data