Regression Assumptions

This chapter delves into regression assumptions within the context of the life expectancy regression model established in Chapter 17.
Prerequisites for following along in R:
- Load the datasets: countries2 and states20.
- Load the libraries: DescTools, lmtest, sandwich, and stargazer.

The confidence in regression estimates (slopes and significance levels) is contingent upon adhering to specific assumptions, which are summarized below and applied to the life expectancy model through diagnostic tests.
Many of these assumptions involve the characteristics of the error term in the population (denoted as ϵ) which cannot be directly observed. Instead, evaluations are based on the sample residuals (denoted as e).

The assumption of linearity posits that the relationship between independent variables (x) and the dependent variable (y) is linear. This means the rate of change in y for any unit change in x remains constant across all values of x.
A key example is the curvilinear relationship observed between the number of doctors per capita and life expectancy, discussed in Chapter 17. Here, applying a linear model leads to poor fit as it does not yield the least squared error, highlighting the necessity of using scatterplots to identify curvilinear relationships.
The scatterplots for life expectancy against the five original independent variables are illustrated in Figure 18.1:
- From the visual assessment, docs10k appears to breach the linearity assumption. This concern was mitigated by logging its values, so further model modifications regarding non-linearity were unnecessary.

This assumption states that there should be no correlation between the error term and the independent variables.
Understanding this assumption is challenging; the main concern is omitted variable bias. The error term reflects variables not included in the model. If these omitted variables correlate strongly with any included variable, the slope for that variable may be biased.
In experimental data, this assumption is not typically an issue since independent variable outcomes are randomly assigned and unrelated to others.
For observational data (the type used in this text), careful consideration of other possible influences on the dependent variable is essential to mitigate omitted variable bias:
- For instance, food_def, a measure of average daily calorie supply as a percentage of needs, is crucial since nutrition likely impacts life expectancy and correlates with other model variables.
Example Code to Plot Relationship:
- To visualize the relationship, scatterplot generated using:
  R plot(countries2$food_def, countries2$lifexp) abline(lm(countries2$lifexp ~ countries2$food_def))
Pearson's correlation results:
- Show a significant positive relationship between food_def and lifexp with outputs indicating:
- t = 10, df = 172, p-value < 2e-16
- Confidence interval: [0.480, 0.677]
- Sample estimate: cor = 0.587

The assumption for multiple regression suggests that no independent variable can be a perfect linear combination of others.
R and other software generally handle perfect multicollinearity by excluding the variable causing the issue.
However, multicollinearity is often present at some degree, which could inflate the slope standard errors, diminishing the relevance of t-scores and p-values.
Following the model adjustment to include food_def, it's important to evaluate the Variance Inflation Factor (VIF) statistics:
- Example VIF Statistics:
- For fert1520: 3.63
- For mnschool: 3.84
- For log10(pop19_M): 1.08
- For urban: 1.84
- For log10(docs10k): 5.18
- For food_def: 1.59
Upon checking the VIF for the revised model (fit2), there were no major collinearity issues.
Note: There is no strict threshold defining what constitutes high collinearity; it’s more problematic in smaller samples.

To avoid bias, the expected or average value of the error term should equal zero.
This can be tested using sample residuals:
- Result: Mean error = 2.81e-17, which is very close to 0.
If the mean of the error term deviates from zero, it may indicate specification errors requiring reconsideration of variable transformations or the inclusion of new variables.

Assessing the normal distribution of the error term is critical for confidence in statistical significance estimates.
In larger samples, significant deviations from normality tend to be a lesser concern unless they are severe.
Methods to evaluate normality include:
- Visual plotting of residuals in a histogram.
- Utilization of the Shapiro-Wilk normality test:
  R shapiro.test(residuals(fit2))
If the p-value obtained is less than 0.05, the null hypothesis on normality is rejected.
Result: p-value = 0.3, suggesting no significant deviation from normal distribution in residuals.

Assumption states that the variance of prediction errors should remain consistent across all values of independent variables.
This is important because statistical significance estimates depend on the prediction error variance.
Heteroscedasticity occurs when error variation is not constant, potentially misrepresenting statistical significance levels.
Visualization of homoscedasticity can be performed by plotting residuals against predicted outcomes.
- Classical pattern violation: Cone-like distribution of residuals.
Formal testing can be done using the Breusch-Pagan test:
- Example Code:
  R bptest(fit2)
Result:
- Breusch-Pagan Test output: BP = 20, df = 6, p-value = 0.002, indicating we reject the null hypothesis of constant variance.

Assumes that errors generated by different independent variable values are unrelated, also referred to as serial correlation.
Primarily a concern in time-series data, leading to inflated t-scores due to suppressed standard errors, which may cause incorrect significance conclusions.

Congratulations on progressing through the book! This journey enhanced your data analysis skills, particularly with R.
Next steps include:
- Engaging in further study, emphasizing regression analysis.
- Opt for courses focused on regression methods suitable for your comfort level.
- Utilize the R language in additional data analysis courses to strengthen both statistical understanding and R programming skills.
- Seek hands-on data analysis opportunities utilizing real-world datasets.

Apply the regression model from Chapter 17 using the states20 dataset with the following specifications:
- Dependent variable: infant_mort
- Independent variables: PCincome2020, teenbirth, lowbirthwt, south
Tasks include:
- Create a scatterplot matrix to assess the linearity assumption among independent variables.
- Evaluate and conduct tests to determine if the mean of the error term equals zero.
- Use a histogram and shapiro.test to check for normality in error terms.
- Plot residuals against predicted values to analyze constant variance assumption.
- Conduct a Breusch-Pagan test for a formal homoscedasticity assessment.
- Identify and justify an influential variable in the states20 data set potentially omitted from your model.
- Include the new variable in the model, display using stargazer, and discuss differences between the models for potential biases.