Regression

Regression Analysis

A statistical technique for finding the best-fitting straight line for a set of data and making predictions based on correlations.

Linear Relationship

A relationship between two variables that allows for the computation of an equation describing the relationship, expressed as Y = bX + a.

Y-intercept

The value of Y when X is equal to 0, represented as 'a' in the equation Y = bX + a.

Slope Constant

Indicates how much Y changes when X increases by 1 point, represented as 'b' in the equation Y = bX + a.

Best Fit Line

The line through the data points that minimizes the distance from each point to the line, providing the best prediction of Y.

Standard Error of Estimate (SEE)

A measure of the accuracy of predictions, indicating the standard distance between the regression line and actual data points.

Coefficient of Determination (r²)

The proportion of variability in the dependent variable (Y) that can be explained by its relationship with the independent variable (X).

Simple Linear Regression

A regression analysis that predicts a dependent variable from one independent variable.

Multiple Linear Regression

A regression analysis that predicts a dependent variable from two or more independent variables.

Homoscedasticity

The assumption that the variance of residuals should remain constant across levels of the dependent variable.

Normal Distribution of Residuals

The assumption that residuals (errors) should be normally distributed in regression analysis.

Multicollinearity

A situation where independent variables are too highly correlated, which can skew results; ideally, r should be less than .80.

Unstandardized Coefficients (B)

Coefficients used in a regression equation to predict Y scores; cannot be compared across predictors unless IVs use the same scale.

Standardized Coefficients (β)

Coefficients that indicate the relative strength of each predictor, allowing comparison across different independent variables.

Durbin-Watson Test

A test used to check the independence of residuals, with values close to 2 being ideal.

Shapiro-Wilk Test

A test used to check for normality in the distribution of residuals.

Adjusted r²

An adjusted version of r² that accounts for the number of predictors in the model, providing a more accurate measure of explanatory power.

Independent Variable (IV)

A variable that is manipulated or categorized to determine its effect on the dependent variable.

Dependent Variable (DV)

The outcome variable being predicted or explained in a regression analysis.

Stepwise Multiple Regression

A regression method where independent variables are entered based on software-driven analysis of data.

Pearson's r

A measure of the linear correlation between two variables, indicating the strength and direction of their relationship.

Assumptions of Regression Analysis

Key assumptions include:
linearity,
independence of residuals,
homoscedasticity (constant variance),
normal distribution of residuals
absence of multicollinearity.

Linearity

Linearity assumes that the relationship between the independent variable (IV) and dependent variable (DV) is a straight line. This means that changes in the IV would lead to proportional changes in the DV.

Independence of Residuals

This assumption states that residuals (the differences between observed and predicted values) should be independent from each other. This means that the value of one residual should not influence the value of another, indicating that all observations should be collected independently.

Homoscedasticity

Homoscedasticity refers to the assumption that the variance of the residuals should remain constant across all levels of the independent variable. If the variance changes (heteroscedasticity), it can lead to inefficiencies in the predictions and estimates.

Normal Distribution of Residuals

This assumption posits that residuals should follow a normal distribution. Normally distributed residuals help ensure the validity of hypothesis tests and confidence intervals, as many statistical tests depend on this condition.

Absence of Multicollinearity

Multicollinearity occurs when independent variables are highly correlated with each other, which can inflate the variance of coefficient estimates and make them unstable. The assumption of absence of multicollinearity suggests that while IVs can be correlated, they should not be so highly correlated that it skews the regression results.