DSCI 4520 - Linear Regression 1

Vocabulary flashcards covering the concepts, equations, and diagnostic evaluations of simple and multiple linear regression from DSCI 4520.

Residual

The error in a regression model, calculated as the difference between the observed value and the predicted value ($e = y - \hat{y}$).

Sum of Squared Errors (SSE)

A metric used to quantify the overall error in a model by calculating the sum of all squared residual terms ($\sum e^2$).

Linear Relationship

A relationship where increasing or decreasing one variable $n$ times is associated with a corresponding increase or decrease of $n$ times in the other variable, such that change is consistently present in the same proportion.

Scatterplot Matrix

A visual display of relationships between variables that is symmetric across the main diagonal and features univariate histograms along the main diagonal.

Pearson's Correlation Coefficient

A metric ranging from $-1$ to $1$ that measures the strength and direction of the linear relationship between two variables.

Linear Regression

A statistical method that uses ordinary least squares to predict a target (dependent) variable based on one or more input (independent) variables.

Simple Linear Regression

A regression model that predicts a target variable based on a single predictor variable using the equation $y = \beta_0 + \beta_1 x + \epsilon$.

Multiple Linear Regression

A regression model that predicts a target variable based on multiple predictor variables using the equation $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \beta_k x_k + \epsilon$.

Target (Dependent) Variable

The output variable ($y$) being predicted in a regression model, which must be measured at least at the interval level.

Predictor (Independent) Variable

An input variable ($x$) used to predict the dependent variable in a regression model; it can be interval-level or nominal (via dummy coding).

Dummy Coding

A technique used to convert nominal independent variables into numerical values so they can be included in a regression model.

Ordinary Least Squares (OLS)

A mathematical approach that identifies the best-fitting line through data by finding coefficient values that minimize the sum of squared errors ($SSE$).

Slope Coefficient ($\beta_1$)

A parameter in simple linear regression calculated as $\beta_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}$, representing the change in the dependent variable for a one-unit increase in the independent variable, all else held equal.

Intercept ($\beta_0$)

The point where the regression line crosses the y-axis, calculated as $\beta_0 = \bar{y} - \beta_1 \bar{x}$, representing the baseline value when the independent variable is zero.

Naïve (Average) Model

A baseline model that predicts the mean of the dependent variable for every observation, serving as the benchmark that a linear regression model seeks to outperform.

Significance of the Model ($F$-test)

An evaluation of overall model performance that compares the $SSE$ of the null (average) model to that of the full regression model to determine if the model adds value over the mean.

Significance of the Predictors ($t$-test)

A statistical test that determines the likelihood that a specific predictor's slope coefficient ($\beta$) is equal to zero, indicating whether the predictor influences the dependent variable.

Coefficient of Determination ($R^2$)

A metric indicating the proportion of total variation in the dependent variable that is explained by the regression model.

Adjusted $R^2$

A modified version of $R^2$ that adjusts downward as additional predictor variables are added to penalize unnecessary model complexity.