regression

Covariance

A way to see if there is a relationship between two or more variables.

Cov(x, y) formula

Cov(x, y) = ∑(xi - x̄)(y - ȳ) / N-1.

Regression

Predicting Y from someone’s score on X based on knowledge of the relationship between X and Y.

Residuals/Errors

Differences between the actual values of the dependent variable and the values predicted by the regression model.

Intercept in regression

The starting point of the dependent variable when all independent variables are zero.

Regression Coefficient

Shows the strength and direction of the relationship between each independent variable and the dependent variable.

Assumptions of Regression

Linearity, independence of errors, constant variance, and normality of residuals.

Regression Equation

Y (predicted value) = b0 + b1x + e.

R squared

Coefficient of determination; measures the proportion of variance predictable from the independent variables.

F-test in regression

Tests whether independent variables significantly explain the variation in the dependent variable.

Standard Error of the Estimate (SEE)

Measure of the accuracy of predictions made by a regression model.

Residual variance

Measures the average squared difference between observed and predicted values.

Correlation vs. Regression

Correlation checks if two things are linked, while regression explores and uses that link for predictions.

Method of Least Squares

A technique used in regression analysis to find the line of best fit by minimizing the sum of squared residuals.

Degrees of Freedom for Residuals

Calculated as n - k - 1, where n is number of observations and k is number of predictors.

Predictive Modeling

Using regression to predict outcomes based on explanatory variables.

Independent Variables (IV)

Variables that are manipulated to observe their effect on the dependent variable.

Dependent Variable (DV)

The outcome variable that researchers are trying to predict or explain.

Linear Regression

A method for modeling the relationship between a dependent variable and one or more independent variables.

Non-linear relationship

When points follow a curved pattern suggesting the use of a non-linear regression model.

Clustered relationship

A distribution of points that form distinct groups or categories, needing further analysis.

Residual Sum of Squares (RSS)

The sum of the squares of the residuals; a measure of how well the regression model fits the data.

Coefficient of Determination

Another term for R squared, indicating how well data fits a statistical model.

Baseline in Regression

The intercept value of the regression model when all independent variables are set to zero.

Change in Y due to 1 unit Change in X

Reflected in the slope of the regression equation, b1.

Statistical Significance

Indicates whether the results of the regression model are likely due to chance.

Limitations of Regression

Regression equations should not be used for predictions outside the range of original data.

Relationship Type based on Scatter Plot

Determining the nature of relationships (linear, non-linear, clustered) through visual data representation.

Analysis of Variance (ANOVA) in Regression

Used to compare means among groups to see if the regression model significantly explains variability.

Predictor variable

The independent variable that is used to predict the dependent variable.

Strength and Direction of Relationship

Described by regression coefficients indicating how one variable affects another.

Approach for Estimating Coefficients

Finds the line that best fits data by minimizing differences between observed and predicted outcomes.