Linear Regression

Linear Regression Prediction Formula

• ^y = ^B0 + ^B1x1 + ^B2x2= forces changes in predicted values to occur in only a single direction(a vector in the space of inputs with elements equal to the parameter estimates)
  • ^y = prediction estimate = formed from a simple linear combination of inputs
  • ^B0 = intercept estimate = centers the range of predictions
  • ^B1 = parameter estimate =determine the trend strength(or slope) between each input and the target.
  • Intercept and parameter estimates are chosen to minimize the squared error between the predicted and observed target(or response) values (least squares estimation).
  • Prediction estimates can be viewed as a linear approximation to the expected(average) value of a target conditioned on observed input values.
• Simple Linear Regression Model - The relationship between the response variable (target) and the predictor variable (input) can be characterized by the equation that fits a line that minimizes the sum of the squared vertical distances between the observations and the fitted line. In other words, the fitted or regression line is as close as possible to all the data points.
• R-squared - is a statistical measure of how close the data are to the fitted regression line. It is also known as the coefficient of determination, or the coefficient of multiple determination for multiple regression. … 100% indicates that the model explains all the variability of the response data around its mean.
• Why does a binary target need a sigmoid function when working with linear regression?
  • To make Y values stay within a specific range (between 0 and 1) which allows us to determine one of two classes (dependent variable).
• Sigmoid function - A general mathematical function that has an S-shaped curve, or sigmoid curve, which is bounded, differentiable, and real.
• Logistic function -  A certain sigmoid function that is widely used in binary classification problems using logistic regression.
• (sigmoidal) logistic function
  • P = e^logit(P)/1+e^logit(P)

Supervised Machine Learning Method w/ Linear Regression

• What is Linear Regression used to predict?
  • Linear regression is used to predict the continuous dependent variables using a given set of independent variables.
• What is Linear Regression used to solve for?
  • Regression problem.
• What are we able to predict in Linear Regression?
  • the value of continuous variables
• When we find the line of best fit what does that help us do?
  • Easily predict the output
• What does the output of linear regression have to be?
  • a continuous value (ex: price, age, etc.)
• What is required between the relationship between the dependent variable and the independent variable?
  • Must be linear

\