Week 5 Lecture Recording

Overview of Regression Analysis

• Focused on understanding simple and multiple regression analysis.

• Aims to predict outcomes from one or more predictors.

• Key concepts: hierarchy and stepwise regression.

Simple Regression

• Definition: Analyzing the relationship between two variables to predict an outcome.

• Y = a + bX:

  • Y: Criterion variable (outcome).

  • a: Intercept (value when X=0).

  • b: Slope (changes in Y for each unit change in X).

• Example: Salary (predictor) and happiness (criterion).

• Residuals: Error terms representing the distance of data points from the regression line.

  • Vertical distance from points to the line indicates error.

  • Can be assessed using the Durbin-Watson statistic.

Multiple Regression

• Definition: Expands simple regression to analyze multiple predictors simultaneously.

• Y = a + b1X1 + b2X2 + ... + bnXn:

  • b1, b2, ... bn: Slopes for each predictor variable.

• Outcome Example: Statistics anxiety predicted by personality traits (perfectionism) and effort (revision hours).

Assumptions of Multiple Regression

1. Continuous Criterion Variable: Must be interval-level.

2. Normal Distribution of Residuals: Residuals should be normally distributed.

3. Independence of Errors: Residuals must be independent.

4. Homogeneity of Variance: Variance should be consistent across levels of predictors.

5. No Multicollinearity: Predictors should not be highly correlated.

6. Linear Relationship: A linear connection must exist between predictors and criterion variable.

Regression Models

Hierarchical Regression

• Known predictors based on previous research are entered first (theory-driven).

• Exploratory predictors can enter in subsequent steps.

• Benefits: Establishes unique predictive influence while holding other variables constant.

Forced Entry Regression

• All predictors entered simultaneously.

• Suitable when all predictors are thought to have equal importance.

• Cannot determine which predictor explains more variance beforehand.

Stepwise Regression

• Variables entered based on statistical criteria.

• Algorithm selects variables iteratively to maximize explained variance.

• Useful for exploring significant predictors without prior knowledge.

• Not ideal for definitive conclusions due to reliance on mathematical selection process.

Example of Multiple Regression Analysis

• Research Question: What enhances exam performance?

• Predictors: Revision hours, alcohol consumption, coffee intake.

• Model Summary: Multiple correlation coefficient (R) and adjusted R-squared values represent shared variance with the criterion.

• ANOVA Table: Determines if the regression model is significant (predicts above chance level).

• Coefficients Table: Shows the significance of each predictor variable.

  • Only revision hours and alcohol consumption were significant predictors; coffee was not.

Coefficients Interpretation

• Unstandardized beta coefficients represent the effect of a one-unit change in the predictor on the outcome.

  • For example, each additional revision hour increases exam scores, while each unit of alcohol decreases scores.

• Standardized coefficients allow comparison across predictors.

Reporting Findings

• Report coefficients only if the ANOVA is significant.

• Example interpretation: "For each additional hour of revision, a student's score increases by 1.53%, while each unit of alcohol consumed drops performance by 2.3%."

• Confidence intervals help assess reliability of coefficients (e.g., intervals not crossing zero indicate significance).

Conclusion

• Understand and apply regression techniques through practical exercises during computer lab sessions.

• Aim for competence in designing, running, and interpreting regression tests in various research contexts.