DEFENITIONS

Simple Linear Regression

A very simple approach for predicting a quantitative response Y on the basis of a single predictor variable X.

Response

The quantitative variable that is being predicted.

Predictor

The variable used to predict the response.

Intercept (β₀)

The value of Y when X = 0.

Slope (β₁)

The average increase in Y associated with a one-unit increase in X.

Error Term (ε)

A catch-all for what we miss with the simple model.

Population Regression Line

Y = β₀ + β₁X.

Least Squares Regression Line

The line obtained by minimizing the Residual Sum of Squares.

Prediction

The estimate ŷᵢ = β̂₀ + β̂₁xᵢ.

Residual

The difference between the ith observed response value and the ith response value predicted by the linear model.

Residual Sum of Squares (RSS)

A measure of the discrepancy between the data and the estimation model.

Least Squares Coefficient Estimates

The values β̂₀ and β̂₁ that minimize RSS.

Standard Error

A measure of the average amount that an estimate differs from the actual value.

Confidence Interval

A range of values that can be thought of as containing the true value of the parameter with a stated probability.

t-Statistic

The coefficient estimate divided by its standard error.

Null Hypothesis

H₀: There is no relationship between X and Y.

Alternative Hypothesis

Hₐ: There is some relationship between X and Y.

Residual Standard Error (RSE)

An estimate of the standard deviation of ε.

R² vs cor(x,y)

R² is used for single and large number of variables

cor(x,y) is used for single variable

R² Statistic

Measures the proportion of variability in Y that is explained using X

Total Sum of Squares (TSS)

The total variance in the response.