Econometrics and Research Design

Comprehensive practice flashcards defining core vocabulary, statistical concepts, assumptions, diagnostics, and testing methods across econometrics and research design lecture notes.

Data-Generating Process (DGP)

The true but unknown real-world mechanism or function that produces the observed data from explanatory variables, unobservables, and randomness, which econometric models aim to approximate.

Fundamental Research

Research aimed at understanding how something works based on data and grounded in existing literature and theories, focusing on uncovering the underlying mechanisms of change.

Solution-Oriented Research

Research focused strictly on solving a specific practical problem rather than understanding the underlying mechanisms of change.

Operationalisation

The process of transforming and breaking down an abstract or broad research question into specific, measurable components.

Statistical Inference

The process of using probability models to reason from an observed sample to the broader population and ultimately to the underlying Data-Generating Process under uncertainty.

Null Hypothesis ($H_0$)

A hypothesis tested in statistical procedures that most frequently posits that there is no effect or no relationship (e.g., $H_0: \beta = 0$).

Alternative Hypothesis ($H_a$)

The opposite of the null hypothesis in a statistical test, positing that a true effect or difference exists (e.g., $H_a: \beta \neq 0$).

p-value

The probability of observing a result as extreme as, or more extreme than, the observed result, under the assumption that the null hypothesis is true ($P(\text{data} \text{ or more extreme} \times H_0)$).

Significance Level ($\alpha$)

A predetermined threshold measuring how sure a researcher wants to be that observed results are not due to chance, commonly set at $0.10$ ($10\%$), $0.05$ ($5\%$), or $0.01$ ($1\%$).

Type I Error

The error committed when a true null hypothesis is incorrectly rejected (a false positive), with a probability denoted by $\alpha$.

Type II Error

The error committed when a false null hypothesis is failingly rejected (a false negative), with a probability denoted by $\beta$.

Practical Significance

The real-world meaningfulness and magnitude of an estimated effect, evaluated through critical thinking rather than relying solely on statistical significance.

Data Wrangling

The iterative process of turning messy, unstructured data into a clean, structured form through enriching and validating, cleaning, and structuring and transforming.

Stratified Sampling

A sampling technique where the population is divided into subgroups (strata) from which samples are drawn, useful when certain categories or sectors are underrepresented.

Cluster Sampling

A sampling method in which the population is divided into clusters, and entire clusters are randomly selected for analysis.

Linear Rescaling

Adding a constant to, or multiplying/dividing a variable by, a constant to simplify interpretation without altering statistical relationships.

Standardisation

A transformation ($z = \frac{x - \bar{x}}{s}$) that centres a variable on zero and scales it so that its standard deviation equals one, facilitating comparison across different scales.

Winsorising

An outlier treatment method where extreme values beyond chosen percentiles (such as the $5^{\text{th}}$ and $95^{\text{th}}$ percentiles) are replaced with the percentile cutoff values.

Tukey Method (Outlier Detection)

A non-parametric outlier identification technique that classifies an observation as an outlier if it falls below $Q_1 - 1.5 \times \text{IQR}$ or above $Q_3 + 1.5 \times \text{IQR}$.

Cook's Distance

An influence diagnostic that measures how much all fitted values in a regression model change when a single observation is deleted, using a threshold of $1$ or a conservative threshold of $\frac{4}{n}$.

Missing Completely at Random (MCAR)

A missing data mechanism where missingness is entirely independent of both observed and unobserved data values, with no systematic reason for missing entries.

Missing at Random (MAR)

A missing data mechanism where the probability of missingness depends systematically on observed variables in the dataset, but not on unobserved data values.

Missing Not at Random (MNAR)

A missing data mechanism where the probability of data being missing is systematically related to unobserved values or the missing values themselves.

Little's MCAR Test

A statistical test whose null hypothesis states that data are Missing Completely at Random; rejection ($p < 0.05$) indicates that data are not MCAR, though it cannot differentiate between MAR and MNAR.

Multiple Imputation by Chained Equations (MICE)

An iterative imputation algorithm that cycles through incomplete variables one at a time, predicting each variable using a tailored model based on the others to generate multiple completed datasets.

Random Forest Imputation

A non-parametric imputation technique that uses an ensemble of decision trees to predict missing values, automatically capturing non-linear relationships and interactions without requiring a specified functional form.

Econometrics

The application of statistical and mathematical techniques to economic problems by fitting regression lines through mathematical equations relating dependent and independent variables.

Least-Squares Criterion

The mathematical criterion used to select a regression line by minimising the sum of squared errors/residuals ($\sum e_i^2$) between observed values and model predictions.

Ordinary Least Squares (OLS)

A regression estimation method that estimates model parameters by minimising the sum of squared residuals ($\sum_{i=1}^n e_i^2$).

Intercept ($\beta_0$)

The constant term in a regression model representing the expected value of the dependent variable when all independent variables equal zero, where the line cuts the $Y$-axis.

Slope Coefficient ($\beta_1$)

A regression parameter that indicates the expected change in the dependent variable ($Y$) when the independent variable ($X$) increases by one unit, holding all other variables constant.

Total Sum of Squares (TSS)

The total sample variation of the dependent variable around its mean, calculated as $\text{TSS} = \sum_{i=1}^n (Y_i - \bar{Y})^2$.

Residual Sum of Squares (RSS)

The portion of total variation unexplained by the regression model, calculated as the sum of squared differences between observed and fitted values: $\text{RSS} = \sum_{i=1}^n e_i^2$.

Coefficient of Determination ($R^2$)

The fraction of the sample variation in the dependent variable explained by the independent variables, calculated as $R^2 = \frac{\text{ESS}}{\text{TSS}} = 1 - \frac{\text{RSS}}{\text{TSS}}$.

Adjusted $R^2$ ($\bar{R}^2$)

A modification of $R^2$ that penalises the addition of explanatory variables by adjusting for degrees of freedom: $\bar{R}^2 = 1 - \frac{\sum e_i^2 / (N - K - 1)}{\sum (Y_i - \bar{Y})^2 / (N - 1)}$.

Anscombe's Quartet

Four distinct datasets that share virtually identical descriptive statistics and regression fits ($r = 0.816$, $R^2 = 0.67$, $\hat{Y} = 3.00 + 0.500X$) but display completely different graphical structures.

Control Variable

An additional explanatory variable included in a multiple regression model to hold confounding factors constant while isolating the relationship of primary interest.

Confounding Variable

A variable that influences both the independent variable and the dependent variable, which can distort or create spurious connections between them if omitted.

Mediator Variable

A variable that lies along the causal pathway between the independent and dependent variables, transmitting part or all of the independent variable's effect indirectly.

Moderator Variable

A variable that alters the direction or strength of the relationship between an independent variable and a dependent variable.

Goodhart's Law (Strathern's Formulation)

The principle formulated by Marilyn Strathern (1997) stating: 'When a measure becomes a target, it ceases to be a good measure.'

Akaike Information Criterion (AIC)

An information criterion that assesses model quality by balancing goodness of fit against complexity with a light penalty for each additional parameter.

Bayesian Information Criterion (BIC)

An information criterion that penalises model complexity based on the sample size, generally favouring more parsimonious models than AIC.

Hannan–Quinn Information Criterion (HQIC)

An information criterion that penalises model complexity more heavily than AIC but less heavily than BIC.

Dummy Variable

A binary indicator variable that takes the value of $0$ or $1$ to denote the absence or presence of a specific qualitative category.

Reference Group

The omitted category in a dummy variable regression against which the coefficients of the included dummy categories are compared and interpreted.

Gauss-Markov Theorem

The mathematical theorem proving that if an OLS regression satisfies the classical assumptions 1 through 6, the OLS estimator is the Best Linear Unbiased Estimator (BLUE).

Best Linear Unbiased Estimator (BLUE)

A descriptor indicating that an estimator is linear, unbiased, and has the minimum variance (highest efficiency) among all linear unbiased estimators.

Best Unbiased Estimator (BUE)

An estimator that possesses the smallest variance among all unbiased estimators, including non-linear estimators, achieved when OLS residuals are normally distributed.

Endogeneity

The violation of the OLS assumption occurring when an explanatory variable is correlated with the error term, resulting in biased and inconsistent coefficient estimates.

Ramsey's RESET Test

A regression specification test that detects functional form misspecification or omitted variables by testing the statistical significance of adding powers of the fitted values ($\hat{Y}^2, \hat{Y}^3, \hat{Y}^4$).

Homoscedasticity

The classical OLS assumption requiring that the error terms have a constant variance across all observations.

Heteroscedasticity

The condition where error terms do not have constant variance, which does not bias coefficient estimates but leads to biased standard errors and invalid hypothesis tests.

Breusch-Pagan Test

A statistical procedure testing the null hypothesis of homoscedasticity against the alternative of a linear form of heteroscedasticity.

White's Test

A non-parametric test designed to detect general, non-linear forms of heteroscedasticity, which loses statistical power when many variables are included.

Variance Inflation Factor (VIF)

A diagnostic metric measuring the degree of collinearity between an explanatory variable and all other predictors in a regression: $\text{VIF} = \frac{1}{1 - R^2}$.

Serial Correlation (Autocorrelation)

A violation of OLS assumptions, primarily occurring in time-series and panel data, where error terms are correlated with their own lagged values.

Omitted Variable Bias

The bias that arises when a variable affecting the dependent variable and correlated with one or more included predictors is excluded from the regression model.

Upward Bias (Omitted Variable)

The systematic overestimation of an included variable's effect ($\text{Bias} = \beta_2 \times \text{Corr}(X_1, X_2)$) occurring when the effect of the omitted variable on $Y$ and its correlation with the predictor share the same sign.

Downward Bias (Omitted Variable)

The systematic underestimation of an included variable's effect occurring when the omitted variable's effect on $Y$ and its correlation with the predictor have opposite signs.

Irrelevant Variable

An exogenous variable added to a regression that does not affect the true underlying relationship, which does not introduce bias but inflates coefficient variances and lowers adjusted $R^2$.

Interaction Term

An independent variable formed by multiplying two or more predictors, used when the effect of one predictor on the dependent variable depends on the level of another predictor.

Dummy-Numeric Interaction

A regression model specification ($Y_i = \beta_0 + \beta_1 X_i + \beta_2 D_i + \beta_3 X_i D_i + \varepsilon_i$) where an interaction between a continuous variable and a dummy variable allows both the intercept ($\beta_2$) and the slope ($\beta_3$) to differ across groups.

Conditional Effect (Interaction)

The specific marginal effect of an independent variable ($\beta_1 + \beta_3 Z$) calculated and tested at an assigned value of the interacting variable ($Z$).

Generalised Variance Inflation Factor (GVIF)

A diagnostic measuring how strongly a set of variables (such as an interaction term and its component main effects) overlaps with the remaining predictors, adjusted by degrees of freedom as $\text{GVIF}^{\frac{1}{2 \times \text{Df}}}$.