Interactions Between Independent Variables and Nonlinear Regression Models

Interaction Effects in Regression Models

• Concept of Interaction Effects:

  • An interaction occurs when the effect of a change in one independent variable on the dependent variable ($Y$) depends on the value of another independent variable.
  • Interaction terms allow regression specifications to capture conditional relationships and structural differences across subgroups or varying continuous scales.

• Three Categorical Cases:

  • Interactions between two binary variables.
  • Interactions between a continuous variable and a binary variable.
  • Interactions between two continuous variables.

Interactions Between Two Binary Variables

• Baseline Model Without Interactions:

  • Consider a population linear regression model relating log earnings to two binary explanatory variables:         $Y_i = \beta_0 + \beta_1 D_{1i} + \beta_2 D_{2i} + u_i$
  • Variable definitions:
    • $Y_i = \ln(\text{Earnings}_i)$: Natural logarithm of earnings for the $i^{\text{th}}$ individual.
    • $D_{1i}$: Binary indicator where $D_{1i} = 1$ if the $i^{\text{th}}$ person graduated from college, and $D_{1i} = 0$ otherwise.
    • $D_{2i}$: Binary indicator where $D_{2i} = 1$ if the $i^{\text{th}}$ person is female, and $D_{2i} = 0$ if male.
  • Interpretation of baseline coefficients:
    • $\beta_1$: The effect on log earnings of graduating from college, holding sex constant.
    • $\beta_2$: The mean difference in log earnings between females and males, holding college schooling constant.
  • Limitation of the baseline model:
    • The specification assumes that the return to having a college degree is identical for men and women.
    • It does not allow the effect of $D_{1i}$ on $Y_i$ to depend on the state of $D_{2i}$.

• Binary Interaction Regression Model:

  • To allow the market return of a college degree to differ between men and women, the model is augmented with an interaction term formed by the product of both binary regressors:         $Y_i = \beta_0 + \beta_1 D_{1i} + \beta_2 D_{2i} + \beta_3 (D_{1i} \times D_{2i}) + u_i$
  • Terminology:
    • The regressor $D_{1i} \times D_{2i}$ is termed the interaction term or interacted regressor.
    • The complete equation is designated a binary variable interaction regression model.
  • Interpretation of interaction coefficients:
    • For men ($D_{2i} = 0$): The effect of acquiring a college degree (a unit change in $D_{1i}$) is $\beta_1$.
    • For women ($D_{2i} = 1$): The effect of acquiring a college degree is $\beta_1 + \beta_3$.
    • $\beta_3$: The difference in the effect of acquiring a college degree for women compared to men.

Application: Student-Teacher Ratio and English Learners (Binary Variables)

• Binary Variable Definitions:

  • $HiSTR_i$: Binary variable equal to $1$ if the student-teacher ratio is $20$ or more, and $0$ otherwise.
  • $HiEL_i$: Binary variable equal to $1$ if the percentage of English learners is $10\%$ or more, and $0$ otherwise.

• Interacted Regression Results:

  • Estimated regression of test scores:         $\text{TestScore} = 664.1 - 1.9\,HiSTR - 18.2\,HiEL - 3.5(HiSTR \times HiEL)$
  • Standard errors:
    • Constant ($664.1$): $(1.4)$
    • $HiSTR$ ($-1.9$): $(1.9)$
    • $HiEL$ ($-18.2$): $(2.3)$
    • $HiSTR \times HiEL$ ($-3.5$): $(3.1)$
  • Summary statistic: $\bar{R}^2 = 0.290$.

• Predicted Marginal Effects:

  • Formula for predicted effect of moving from a low to high student-teacher ratio ($HiSTR = 0$ to $HiSTR = 1$), holding English learner status constant:         $\text{Effect} = -1.9 - 3.5\,HiEL$
  • In districts with low English learners ($HiEL = 0$):
    • Test scores decline by $1.9$ points.
  • In districts with high English learners ($HiEL = 1$):
    • Test scores decline by $1.9 + 3.5 = 5.4$ points.

• Subgroup Sample Average Test Scores:

  • Low student-teacher ratio, low English learners ($HiSTR = 0$, $HiEL = 0$):         $\text{Average} = 664.1$
  • High student-teacher ratio, low English learners ($HiSTR = 1$, $HiEL = 0$):         $\text{Average} = 664.1 - 1.9 = 662.2$
  • Low student-teacher ratio, high English learners ($HiSTR = 0$, $HiEL = 1$):         $\text{Average} = 664.1 - 18.2 = 645.9$
  • High student-teacher ratio, high English learners ($HiSTR = 1$, $HiEL = 1$):         $\text{Average} = 664.1 - 1.9 - 18.2 - 3.5 = 640.5$

Interactions Between a Continuous and a Binary Variable

• Theoretical Framework:

  • Dependent variable: Log earnings, $Y_i = \ln(\text{Earnings}_i)$.
  • Continuous explanatory variable: Work experience in years ($X_i$).
  • Binary explanatory variable: College degree status ($D_i = 1$ if college graduate, $0$ otherwise).
  • Interactions between $X_i$ and $D_i$ give rise to three structural forms for the population regression function.

• Specification 1: Different Intercepts, Same Slope:

  • Population regression function:         $Y_i = \beta_0 + \beta_1 X_i + \beta_2 D_i + u_i$
  • Regression line for nongraduates ($D_i = 0$):         $Y = \beta_0 + \beta_1 X$
    • Intercept: $\beta_0$
    • Slope: $\beta_1$
  • Regression line for graduates ($D_i = 1$):         $Y = (\beta_0 + \beta_2) + \beta_1 X$
    • Intercept: $\beta_0 + \beta_2$
    • Slope: $\beta_1$
  • Substantive meaning:
    • $\beta_1$ is the marginal effect on log earnings of an additional year of work experience, which is identical regardless of college graduation status.
    • $\beta_2$ is the intercept shift, representing the mean effect of a college degree on log earnings holding years of experience constant.

• Specification 2: Different Intercepts and Different Slopes:

  • Population regression function:         $Y_i = \beta_0 + \beta_1 X_i + \beta_2 D_i + \beta_3 (X_i \times D_i) + u_i$
  • Regression line for nongraduates ($D_i = 0$):         $Y = \beta_0 + \beta_1 X$
    • Intercept: $\beta_0$
    • Slope: $\beta_1$
  • Regression line for graduates ($D_i = 1$):         $Y = (\beta_0 + \beta_2) + (\beta_1 + \beta_3)X$
    • Intercept: $\beta_0 + \beta_2$
    • Slope: $\beta_1 + \beta_3$
  • Substantive meaning:
    • $\beta_2$ is the difference between the two intercepts.
    • $\beta_3$ is the difference between the two slopes.
    • $\beta_1$ is the return to an additional year of work experience for nongraduates.
    • $\beta_1 + \beta_3$ is the return to an additional year of work experience for college graduates.

• Specification 3: Same Intercept, Different Slopes:

  • Population regression function:         $Y_i = \beta_0 + \beta_1 X_i + \beta_2 (X_i \times D_i) + u_i$
  • Regression line for nongraduates ($D_i = 0$):         $Y = \beta_0 + \beta_1 X$
    • Intercept: $\beta_0$
    • Slope: $\beta_1$
  • Regression line for graduates ($D_i = 1$):         $Y = \beta_0 + (\beta_1 + \beta_2)X$
    • Intercept: $\beta_0$
    • Slope: $\beta_1 + \beta_2$
  • Substantive meaning: Both groups share a common intercept at zero work experience, but accumulate earnings at different rates per year of experience.

Application: Continuous Student-Teacher Ratio and Binary English Learners

• Model Framework:

  • Continuous independent variable: Student-teacher ratio ($STR_i$).
  • Binary independent variable: High percentage of English learners ($HiEL_i = 1$ if percentage of English learners is greater than $10\%$, and $0$ otherwise).
  • Structure: Different intercepts and different slopes.

• Estimated Regression Results:

  • Fitted equation:         $\text{TestScore} = 682.2 - 0.97\,STR + 5.6\,HiEL - 1.28(STR \times HiEL)$
  • Standard errors:
    • Constant ($682.2$): $(11.9)$
    • $STR$ ($-0.97$): $(0.59)$
    • $HiEL$ ($5.6$): $(19.5)$
    • $STR \times HiEL$ ($-1.28$): $(0.97)$
  • Summary statistic: $\bar{R}^2 = 0.305$.

• Subgroup Regression Functions:

  • Districts with low fraction of English learners ($HiEL_i = 0$):         $\text{TestScore} = 682.2 - 0.97\,STR_i$
  • Districts with high fraction of English learners ($HiEL_i = 1$):         $\text{TestScore} = (682.2 + 5.6) - (0.97 + 1.28)STR_i = 687.8 - 2.25\,STR_i$

• Interpretation:

  • A reduction of the student-teacher ratio by $1$ unit increases test scores by $0.97$ points in districts with low fractions of English learners.
  • A reduction of the student-teacher ratio by $1$ unit increases test scores by $2.25$ points in districts with high fractions of English learners.
  • The difference in the marginal effect of student-teacher ratio between district types is $1.28$ points, corresponding to the interaction coefficient.

Interactions Between Two Continuous Variables

• Model Formulation:

  • Let $X_{1i}$ and $X_{2i}$ both be continuous independent variables.
  • Example: $Y_i$ is log earnings of the $i^{\text{th}}$ worker, $X_{1i}$ is years of work experience, and $X_{2i}$ is years of education.
  • Standard linear regression restriction: Without interaction terms, the marginal effect of an additional year of experience is assumed independent of the level of education.
  • Interaction model specification:         $Y_i = \beta_0 + \beta_1 X_{1i} + \beta_2 X_{2i} + \beta_3 (X_{1i} \times X_{2i}) + u_i$
  • Interaction term: The product regressor $X_{1i} \times X_{2i}$.

• Marginal Effects Interpretation:

  • Marginal effect of $X_{1i}$ on $Y_i$: $\beta_1 + \beta_3 X_{2i}$.
  • Marginal effect of $X_{2i}$ on $Y_i$: $\beta_2 + \beta_3 X_{1i}$.
  • The return to work experience depends on the quantity of education received, and the return to education depends on total accumulated work experience.

Logarithmic Transformations and Percentage Change Approximations

• Exact Formula vs. Logarithmic Difference:

  • Exact percentage change calculation:         $\text{Percentage Change} = 100 \times \frac{\text{Sales}_t - \text{Sales}_{t-1}}{\text{Sales}_{t-1}}$
  • Logarithmic approximation formula:         $\text{Log Approximation} = 100 \times [\ln(\text{Sales}_t) - \ln(\text{Sales}_{t-1})]$

• Numerical Comparisons (Base Sales = 196):

  • When sales change from $196$ to $198$:
    • Exact change: $100 \times \frac{198 - 196}{196} = 1.0204\%$
    • Approximation: $100 \times [\ln(198) - \ln(196)] = 1.0152\%$
  • When sales change from $196$ to $205$:
    • Exact change: $100 \times \frac{205 - 196}{196} = 4.5918\%$
    • Approximation: $100 \times [\ln(205) - \ln(196)] = 4.4895\%$
  • When sales change from $196$ to $250$:
    • Exact change: $100 \times \frac{250 - 196}{196} = 27.551\%$
    • Approximation: $100 \times [\ln(250) - \ln(196)] = 24.335\%$
  • When sales change from $196$ to $500$:
    • Exact change: $100 \times \frac{500 - 196}{196} = 155.1\%$
    • Approximation: $100 \times [\ln(500) - \ln(196)] = 93.649\%$

• Behavior of Approximation:

  • The logarithmic approximation is highly accurate for small percentage changes.
  • The quality of the approximation systematically deteriorates as the percentage change becomes larger.

• Alternative Exercise Specification (Base Sales = 243):

  • Sales in 2018: $\$243\text{ million}$.
  • Sales in 2019 options evaluated: $\$250\text{ million}$, $\$255\text{ million}$, $\$260\text{ million}$, and $\$265\text{ million}$.

Hedonic Housing Price Model Analysis

• Interpretation Principles for Semi-Log and Log-Log Regressions:

  • Regressors in levels ($X$): The estimated coefficient gives an approximate percentage effect on the dependent variable for a one-unit change in $X$.
  • Regressors in logarithms ($\ln(X)$): The estimated coefficient represents an elasticity.
  • Binary regressors ($D$): The exact percentage effect is computed via the formula:         $\text{Exact Percentage Effect} = 100 \times (e^{\beta} - 1)\%$

• Variable Definitions:

  • $\text{Price}$: Sale price in dollars ().\n * \text{Size}: House size measured in square feet.\n * \text{Bedrooms}: Number of bedrooms.\n * \text{Pool}$: Binary variable equal to$1$if house has a swimming pool, and$0 otherwise.\n * \text{View}$: Binary variable equal to$1$if house has a nice view, and$0 otherwise.\n * \text{Condition}$: Binary variable equal to$1$if real estate agent reports house is in excellent condition, and$0 otherwise.\n\n* **Regression Results Summary Table:**\n\n| Regressor | (1) | (2) | (3) | (4) | (5) |\n| :--- | :--- | :--- | :--- | :--- | :--- |\n| \text{Size}$|$0.00042$($0.000038) | — | — | — | — |\n| \ln(\text{Size})$| — |$0.69$($0.054$) |$0.68$($0.087$) |$0.57$($2.03$) |$0.69$($0.055) |\n| [\ln(\text{Size})]^2$| — | — | — |$0.0078$($0.14) | — |\n| \text{Bedrooms}$| — | — |$0.0036$($0.037) | — | — |\n| \text{Pool}$|$0.082$($0.032$) |$0.071$($0.034$) |$0.071$($0.034$) |$0.071$($0.036$) |$0.071$($0.035) |\n| \text{View}$|$0.037$($0.029$) |$0.027$($0.028$) |$0.026$($0.026$) |$0.027$($0.029$) |$0.027$($0.030) |\n| \text{Pool} \times \text{View}$| — | — | — | — |$0.0022$($0.10) |\n| \text{Condition}$|$0.13$($0.045$) |$0.12$($0.035$) |$0.12$($0.035$) |$0.12$($0.036$) |$0.12$($0.035) |\n| \text{Intercept}$|$10.97$($0.069$) |$6.60$($0.39$) |$6.63$($0.53$) |$7.02$($7.50$) |$6.60$($0.40) |\n| **Summary Statistics** | | | | | |\n| \text{SER}$|$0.1026$|$1.023$| — | — |$1.020 |\n| \bar{R}^2$|$0.0710$|$0.0761$| — | — |$0.0814 |\n\n* **Empirical Questions Evaluated:**\n * Using column (1), expected change in price of adding 1500\text{ square feet}$and constructing a$99\% confidence interval.\n * Using column (2), interpreting the elasticity coefficient on \ln(\text{Size}) and evaluating the price impact of doubling house size.\n * Using column (2), estimating the effect of \text{View}$on price, its$99\% confidence interval, and testing whether it is statistically different from zero.\n * Using column (3), evaluating the effect of adding two bedrooms, assessing its statistical significance, and comparing the relative importance of house size versus number of bedrooms.\n * Using column (4), evaluating whether the coefficient on \text{Condition} is statistically significant.\n * Using column (5), assessing whether the interaction term \text{Pool} \times \text{View} is statistically significant, and computing the marginal effect of adding a view for a house with a pool versus a house without a pool.\n\n# Academic Journal Demand and Elasticity Analysis\n\n* **Determinants of Journal Demand:**\n * Age interaction with price:\n * The demand for older journals is less elastic than for younger journals because the estimated interaction term between the natural log of journal age and price per citation is positive.\n * Linearity of demand curve:\n * The relationship between log price and log quantity is strictly linear because the estimated regression coefficients on log price squared and log price cubed are both statistically insignificant.\n * Effect of journal length:\n * Journals with more characters have higher overall demand, as evidenced by a positive and statistically significant coefficient estimate on the natural log of characters.\n\n* **Computing Price Elasticity for an 80-Year-Old Journal:**\n * General marginal effect equation:\n        \text{Effect of } \ln(\text{Price per citation}) = [-0.899 + 0.141 \times \ln(\text{Age})] \times \ln(\text{Price per citation})\n * Evaluating at \text{Age} = 80:\n        \text{Elasticity} = -0.899 + 0.141 \times \ln(80) = -0.28\n\n* **Standard Error Calculation:**\n * The standard error of the estimated elasticity is derived by dividing the absolute value of the point estimate (0.28$) by the square root of the$F-statistic testing the linear hypothesis restriction:\n        \beta_{\ln(\text{Price per citation})} + \ln(80) \times \beta_{\ln(\text{Age}) \times \ln(\text{Price per citation})} = 0\n\n* **Invariance Under Variable Rescaling:**\n * Log transformation property when dividing by a scaling constant a:\n        \ln\left(\frac{\text{Characters}}{a}\right) = \ln(\text{Characters}) - \ln(a)\n * Impact on model estimates:\n * The estimated slope parameter associated with \text{Characters}$$ remains identical.
    • The intercept (constant term) absorbs the added term and changes accordingly.