Least Squares Regression Concepts

These flashcards cover key concepts and terminology related to Least Squares Regression as discussed in the lecture notes.

Regression

A technique used for prediction when the variable being predicted is numeric.

Dependent Variable

The variable being predicted, also known as the response, target or output variable, denoted as Y.

Independent Variable

Variables used to predict the value of the dependent variable, also called predictor, explanatory or input variables, denoted as X1, X2, X3…

Simple Regression

Regression involving one independent variable.

Multiple Regression

Regression involving two or more independent variables.

Least Squares Regression Method

A procedure that estimates the parameters of a straight-line regression equation using sample data.

Regression Equation

A simple regression equation has the form: y = b0 + b1*x.

Slope (b1)

The average change in the dependent variable y given a one unit increase in the independent variable x.

Intercept (b0)

The value where the regression line intersects the Y axis; the estimated value of y when x=0.

Residual

The error or variability in y that is not explained by the linear relationship between x and y.

Sum of Squared Errors (SSE)

A measure used in LSR that minimizes the sum of the squared errors.

Coefficient of Determination (R2)

The proportion of the variation in Y that is explained by the regression model.

Correlation Coefficient (r)

Measures the strength of a linear relationship between two variables.

Point Estimate

A single value estimate of a population parameter.

Interval Estimate

A range of values where a population parameter is expected to lie.

Confidence Interval (CI)

Specifies a range within which a population parameter is expected to lie with a given confidence level.

Hypothesis Testing

Involves resolving a conflict between two competing hypotheses about a population parameter.

Null Hypothesis (Ho)

Represents the presumed status quo in hypothesis testing.

Alternative Hypothesis (Ha)

Contradicts the null hypothesis and represents what the researcher is surmising.

P-value

The probability of observing a sample statistic as extreme as the one observed when the null hypothesis is true.

Confidence Interval for Slope Coefficient

A range around the slope coefficient within which we expect the true slope coefficient to lie.

Testing Significance of Slope

Involves assessing whether a slope coefficient is significantly different from zero.

Prediction Interval

An interval estimate for the predicted y for one single observation with given x values.

Confidence Interval vs Prediction Interval

Confidence intervals are narrower than prediction intervals because they estimate the average for a population while prediction intervals account for individual variability.

Statistical Inference

Drawing conclusions about population characteristics through analysis of sample data.

Confidence Level

The probability that the computed confidence interval will contain the true population parameter.