General Linear Regression Models and Dichotomous Variables

The General Linear Model (GLM) and Regression with Categorical Variables

• The General Linear Model (GLM) serves as an overarching theoretical framework or "big umbrella" that encompasses various statistical analyses used in psychological research and social sciences.

• The common themes across all models within the GLM framework are:

  • A numerical dependent variable (DV).

  • The assumption of normally distributed residuals ($\epsilon \sim N(0, \sigma^2)$).

  • The use of a linear function to connect the independent variables (IVs) with the DV.

• The basic linear function for all GLM analyses can be expressed as:

  • $DV = \text{Intercept} + \text{Slope} \times \text{Predictor} + \text{Error}$

• Analyses such as Correlation, Simple Linear Regression, Multiple Linear Regression, T-tests, and Analysis of Variance (ANOVA) are all diverse expressions of the GLM.

• ANOVA and regression share deep mathematical similarities, specifically in how they partition variance in a numerical DV:

  • In Linear Regression: Variance is divided into the component captured by the regression model versus the residuals.

  • In ANOVA: Variance is divided into the component captured by between-group differences versus within-group differences (errors).

Linear Regression with Dichotomous Independent Variables

• A dichotomous variable (also known as a binary variable) is a categorical variable that contains exactly two levels, conditions, or groups.

• Examples of dichotomous variables include:

  • Age categories (e.g., adolescents vs. adults).

  • Enrolment status (e.g., internal vs. external students).

  • Task performance (e.g., students who attempted quizzes vs. those who did not).

• While independent samples t-tests are the traditional choice for comparing two groups on a numerical DV, linear regression is mathematically equipped to handle dichotomous predictors.

• Geometry of the Model:

  • A straight line can always be drawn between any two points. Even if an IV only has two observations or levels, the regression model fits a best-fitting line through those two cluster points.

• Standard regression commands in software (e.g., regress in Stata) work identically whether the IV is numerical or categorical.

Dummy Coding and Coefficient Interpretation

• Dummy coding is a special numerical representation where the two categorical levels are coded as $0$ and $1$.

• This specific coding is essential for the literal interpretation of the regression output:

  • The Intercept (Constant Term/$b_0$): Represents the predicted value of the DV when the IV is zero ($X = 0$). Consequently, the intercept equals the mean of the DV for the group coded as $0$.

  • The Slope (Regression Coefficient/$b_1$): Represents the change in the DV for every one-unit increase in the IV. Moving from $0$ to $1$ is a one-unit increase; therefore, the slope represents the mean difference between the two groups.

  • Slope Calculation: $Slope = \text{Mean of Group 1} - \text{Mean of Group 0}$.

• If a variable is not coded as $0$ and $1$ (e.g., coded as $1$ and $2$), the software will still run the analysis and identify which group is higher or lower, but the constant term loses its meaningful interpretation as a group mean.

• Comparison between T-test and Regression results:

  • Both provide identical $p$-values and degrees of freedom ($df$).

  • The absolute value of the $t$-statistic is identical. In a t-test, the sign depends on which group is subtracted from which. In regression, a positive beta indicates Group 1 has a higher mean, while a negative beta indicates Group 1 has a lower mean than Group 0.

Statistical Control in Multiple Regression

• Multiple regression allows for "statistical control," which is the ability to account for the unique effect of each predictor while holding others constant.

• This is particularly useful when experimental control is impossible, such as in non-experimental research involving correlated predictors (e.g., using both 'hours of study' and 'quiz completion' to predict 'final grades').

• Variability Tracking:

  • Model as a Whole: Captures how predictors jointly or collectively explain variability in the DV (indicated by the $F$-test and R-squared/$R^2$).

  • Predictor Coefficient Table: Shows the unique contribution of each predictor after partialling out the shared variance between independent variables.

• Interpreting Categorical Variables in Multiple Regression:

  • When included in a multiple regression model, the slope for a dichotomous IV is no longer just the simple difference between group means. It is the group difference after controlling for the other predictors in the model.

  • If students who complete quizzes also tend to study more hours, the simple group difference in grades (t-test) overestimates the effect of quizzes. The multiple regression coefficient identifies the effect of quizzes assuming every student studied for the same number of hours.

  • The intercept in a multiple regression model represents the predicted value of $Y$ when all predictors equal zero.

Assumptions of the Linear Model

• Regression with dichotomous variables must still satisfy the four primary assumptions:

  1. Independence of Observations: Participants must be independent of one another. This is typically met through study design.

  2. Normality of Residuals: Residuals ($Y - \hat{Y}$) should be approximately normally distributed. Since residuals are calculated by subtracting the group mean from individual scores, a combined histogram of these errors should center around zero in a normal curve.

  3. Homoscedasticity (Homogeneity of Variance): The spread of residuals should be roughly equal across the levels of the IV. In an rvfplot (residual-versus-fitted plot), this appears as cluster groups with equal vertical spread.

  4. Linearity: There should be no obvious non-linear trend. For a two-level predictor, a straight line is the only possible fit between the two points, so this assumption is inherently satisfied.

Case Study: Wealth Redistribution and Social Sampling

• Research Context: Identifying why wealthier people tend to view the current distribution of wealth as fair and are less supportive of redistribution.

• Theory (Social Sampling): Wealthier people perceive society as wealthier because their social circles consist mainly of other wealthy individuals. This biases their perception of the status quo.

• Participants: $305$ US adults recruited via the Amazon Merckchuk online survey platform.

• Measures:

  • Social Circle Mean Income (IV): Estimated income of people in the participant's social circle.

  • Fairness (IV): Perceived fairness of household income distribution ($1$ to $9$ scale).

  • Satisfaction (IV): Satisfaction with current wealth distribution ($1$ to $9$ scale).

  • Political Preference (Covariate): One's political orientation ($1$ = more liberal to $9$ = more conservative).

  • Gender (Covariate): Categorical variable coded as male ($0$) and female ($1$).

  • Support for Redistribution (DV): A composite score derived from four items (e.g., "The government should redistribute wealth through heavy taxes on the rich").

• Hypothesis: Higher social circle income, higher perceived fairness, and higher satisfaction will predict lower support for wealth redistribution, even after controlling for political orientation and gender.

Data Management and Practical Implementation

• Reverse Scoring: Essential when survey items are phrased in opposite directions. For a $1$ to $6$ scale, a score of $x$ is reversed using the formula $7 - x$.

  • Example: Item 2 ("The government should not help the poor") is reversed so that a high score consistently indicates high support for redistribution across all four items.

• Composite Variables: Created in Stata using the egen (extended generate) command with the rowmean function to calculate the average of multiple items per participant.

• Labeling and Recoding:

  • Use the label define and label values commands in Stata to assign text labels (e.g., "Male," "Female") to numeric dummy codes ($0, 1$).

• Interpreting Case Study Results:

  • Political Preference: Negative beta coefficient indicates that more conservative individuals are less supportive of redistribution (p < .05).

  • Gender: Positive beta coefficient indicates that females (coded as $1$) are statistically significantly more supportive of redistribution than males ($0$).

  • Social Sampling: Higher income in social circles predicted lower support for redistribution.

  • Effect Size Comparison: To compare predictors measured on different scales (e.g., income in dollars vs. fairness on a $1-9$ scale), researchers use Standardized Beta Coefficients (regress ..., beta). The absolute value of the standardized beta indicates which predictor has a stronger relative effect on the DV.

Point-Biserial Correlation

• A correlation between a numerical variable (DV) and a dichotomous variable (IV) is specifically called a Point-Biserial Correlation.

• It is used to determine the association between a binary state and a continuous outcome.

• Example: Correlating whether a student answered a specific exam question correctly (Yes/No) with their total grade for the unit to measure the question's discriminative power.