Lecture 5: Data Quality

Non-sampling error

Any error in survey estimates not caused by sampling only a part of the population

Measurement error

Any deviation of the recorded answer from the truth in any direction

Measurement bias

Measurement error that is systematic in one direction

Selection bias

Part of the target population is missing from the data

• Coverage error or non-response

Coverage error

The frame excludes (or double counts) some of the target population

Recall bias

Respondents misremember (e.g. income, age, etc grouping on round numbers)

Social desirability bias

Respondents answer in a way that looks good to the interviewer

• Crime stats, hobbies etc.

Priming/ Question order-effect

Earlier questions change answers to later questions

Unreliable (proxy) respondents

A person answering for others who does not know the answer (e.g. earnings)

Attenuation bias

Classical measurement error in an X biases its coefficients toward zero

Imputation

Filling in missing values with plausible values

Cell-mean imputation

Replace missing values with the mean of respondents within the same cell (assumes MCAR within cell)

• Shrinks variance and distorts correlations

Hot-deck Imputation

Replace a missing value with the actual value of a similar respondent (donor) in the same cell

Regression Imputation

Replace a missing value with the regression prediction from observed x’s

• Inflates the x-y correlation (no residual noise)

Multiple Imputaiton

Create m > 1 completed datasets with random draws, analyse each, then combine.

• Point estimates are the average over m estimates

• Variance: within-imputation variance (average of m variances) + between-imputation variance (variance of the m estimates)

Why single Imputation understates uncertainty

It treats imputed values as if they were observed, ignoring the uncertainty about the true value

Mid-point estimation

Replace a bracketed earnings answer with the mid-point of the bracket